Outboxer.Relay.SQS (Outboxer v0.1.0)

Relays messages to Amazon SQS through ex_aws_sqs.

This module only exists when :ex_aws_sqs is in your dependencies. ex_aws also needs an HTTP client and, to read SQS responses, an XML parser:

{:ex_aws, "~> 2.5"},
{:ex_aws_sqs, "~> 3.4"},
{:sweet_xml, "~> 0.7"},   # or {:saxy, "~> 1.1"}
{:hackney, "~> 1.20"}     # or configure ex_aws to use Req

Without an XML parser ex_aws cannot tell Outboxer which entries of a batch were accepted, and the relay reports every batch as failed.

Usage

{Outboxer,
 repo: MyApp.Repo,
 outboxes: [
   orders: [
     relay: {Outboxer.Relay.SQS, queue_url: "https://sqs.eu-west-1.amazonaws.com/123456789012/orders.fifo"}
   ]
 ]}

Credentials and region come from your ex_aws configuration.

Options

:queue_url (String.t/0) - Required. The queue URL. A URL ending in .fifo is treated as a FIFO queue.
:fifo - Whether the queue is FIFO. Detected from the .fifo suffix of the URL when omitted. The default value is nil.
:batching - FIFO queues only. :group_safe never puts two messages of the same group in one SendMessageBatch request, so a partial failure cannot reorder a group. :fast fills every request with up to 10 messages regardless of group. The default value is :group_safe.
:encoding - :raw sends the payload as the message body and requires it to be valid UTF-8. :base64 encodes it first. The default value is :raw.
:ex_aws_config (keyword/0) - Overrides passed to ExAws.request/2, such as [region: "eu-west-1"]. The default value is [].

How messages map to SQS

SQS	Outboxer
`MessageBody`	the encoded payload (see `:encoding`)
`MessageGroupId` (FIFO)	the group key; messages without one get a unique id, so they do not queue behind each other
`MessageDeduplicationId` (FIFO)	`Outboxer.Envelope.dedup_id/1`, stable across attempts
message attributes	the message headers, plus `outboxer-id` and `outboxer-type`

SQS allows 10 message attributes per message, so keep headers to 8 or fewer.

Ordering

Only FIFO queues keep order. A standard queue will happily deliver messages out of order whatever Outboxer does; use one when you want the transactional guarantee without ordering.

On a FIFO queue, messages of a group are sent in order. With the default batching: :group_safe a request never carries two messages of one group, because SQS can accept some entries of a batch and reject others. The cost is throughput for a single very busy group (one request per message). batching: :fast lifts that at the price of a small reordering window when SQS partially rejects a batch.

Duplicates

Delivery is at-least-once. On FIFO queues the deduplication id makes SQS drop a repeated send within its five-minute window. On standard queues the consumer has to deduplicate, for example on the outboxer-id attribute.

Limits

SQS rejects a message, or a whole batch, above its size limit. A message that is too large fails on every attempt, so set max_attempts on the outbox if your payloads can get near the limit.