What's a queue?
* Fully Managed
* Scales from I message per second to 15,000s per second
* Default retention of messages: 4 days, maximum of 14 days
* No limit to how many messages can be in the queue
* Low latency (<10ms on publish and receive)
* Horizontal scaling in terms of number of consumers
* Can have duplicate messages (at least once delivery, occasionally)
* Can have out of order messages (best effor ordering)
1. SQS - Producing Messages
* Define Body
* Add messages attributes
* Provide Delay Delivery
Get back
* Message identifier
* MD5 hash of the body
2. SQS - Consuming Messages
* Poll SQS for messages (receive up to 10 messages at a time)
* Process the message within the visibility timeout
* Delete the message using the message ID & receipt handle
3. AWS SQS - FIFO Queue
* First in First out
* must end in .fifo
* Lower throughput (up to 3,000 per second with batching, 300/s without)
* Messages are sent exactly once
* 5 minutes interval de-duplication using "Duplication ID"
4. SQS - Extended Client
* Message size limit is 256KB, how to send large message?
* Using the SQS Extended Client (Java Library)
Kinesis Data Stream vs SQS
| Kinesis Data Stream | SQS |
| Data can be consumed many times | Queue, decouple application |
| Data is deleted after the retention period | One application per queue |
| Ordering of records is preserved (at the shard level) |
Records are deleted after consumption |
| Streaming MapReduce | Messages are processed independently for standard queue |
| KCL with DynamoDB | Capability to 'delay' messages |
| provisioned mode or on-demand mode | Dynamic scailing of load |
