SQS Message🔗

Overview🔗

SQS Message lets you post a message to an Amazon Simple Queue Service (SQS) queue. Other applications can read those messages and perform further processing. FIFO Queues can be used through this component where available.

The AWS API for SQS list queues has a hard limit of 1000, meaning the retrieval of queue names is limited to the first 1000 that are defined. However, users can add additional queues, which will still validate and function, provided the user enters the queues manually.

Outgoing SQS messages that exceed the 256 KB message size limit will be trimmed so that they fit within the queue size constraints. Messages are trimmed to prioritize sending useful information about job completion, to ensure proper tracking in the success/failure queue. Specifically, task information will be discarded first, ensuring that the job summary is always sent. The job summary message will only be trimmed if absolutely necessary to meet the size limit.

Properties🔗

Name = string

A human-readable name for the component.

Region = drop-down

The AWS region that the queue is located in. This must be an existing SQS queue.

Queue Name = drop-down

The name of the queue to write to. Available queues for the selected region are listed.

Message = string

The message to post to the designated queue. Can contain variables to be resolved at runtime, in the same way that other components can use variables. Read Variables to learn more.

Multi-line messages can be created. In the Message editor, use Shift + Enter to create a new line.

Message Format = drop-down

Messages may be optionally Base64 encoded, or compressed with 'gzip' compression. If compression is chosen, then the compressed data is also Base64 encoded, so it can be transported as plain text.

Message Group ID = string

Visible only when the above properties specify a valid First-In-First-Out (FIFO) queue. The tag that specifies that a message belongs to a specific message group.

Message Duplication ID = string

Visible only when the above properties specify a valid First-In-First-Out (FIFO) queue. The token used for deduplication of sent messages.

Snowflake	Delta Lake on Databricks	Amazon Redshift	Google BigQuery	Azure Synapse Analytics
✅	✅	✅	❌	❌