Getting Started with Amazon Simple Queue Service (AWS SQS)

James Kenny -
What is Amazon Simple Queue Service

Amazon Simple Queue Service (Amazon SQS) offers reliable and scalable hosted queues for storing messages as they travel between computers.

By using Amazon SQS, you can move data between distributed components of your applications that perform different tasks without losing messages or requiring each component to be always available.

Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. A queue is a temporary repository for messages that are awaiting processing.

Like other Message bus systems using Amazon SQS, you can decouple the components of an application so they run independently, with Amazon SQS easing message management between components. Any component of a distributed application can store messages in a fail-safe queue. Messages can contain up to 256 KB of text in any format. Any component can later retrieve the messages programmatically using the Amazon SQS API. Messages larger than 256 KB can be managed using the Amazon SQS Extended Client Library for Java, which uses Amazon S3 for storing larger payloads.

Pricing

As of this post, these applied to all regions except Asia Pacific (Tokyo) and GovCloud (US)

  • First 1 million Amazon SQS Requests per month are free
  • $0.50 per 1 million Amazon SQS Requests per month thereafter ($0.00000050 per SQS Request)
  • All SQS API calls count as a Request
  • A single Request can have from 1 to 10 messages, up to a maximum total payload of 256KB.
  • Each 64KB ‘chunk’ of payload is billed as 1 Request. For example, a single API call with a 256KB payload will be billed as four Requests.

Asia Pacific (Tokyo) region only

  • First 1 million Amazon SQS Requests per month are free
  • $0.476 per 1 million Amazon SQS Requests per month thereafter ($0.000000476 per SQS Request)
  • All SQS API calls count as a Request
  • A single request can have from 1 to 10 messages, up to a maximum total payload of 256KB.
  • Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests.

GovCloud (US) region only

  • No free tier
  • $0.60 per 1 million Amazon SQS Requests per month thereafter ($0.00000060 per SQS Request)
  • All SQS API calls count as a Request
  • A single request can have from 1 to 10 messages, up to a maximum total payload of 256KB.
  • Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests.
Get Started

First thing we need to do is make sure we have a user set up in the Identity and Access Management. We need a user that has access keys see here I cover creating a new user with access keys.

Now in Services search for SQS

Select "Create new Queue"

I had already changed my region to EU Ireland( see the top right of the AWS Console you can change the region)

I just left things at the defaults for the moment we'll play with them later. Go ahead and create a queue.

Once your queue is created you will see some information in the panel at the bottom of the screen, mainly the Name followed by a URL, for me it's

https://sqs.eu-west-1.amazonaws.com/54XXXXXXX57/serversncodeq

We need that URL. The next part and this caught me first time around is the Permissions, There is no default set so we need to do that, you can click the permission tab or Right click the queue and select "Add a Permissions"

For the moment I select Everybody and All actions but you can see how we can use user accounts to control access to the queues, who can write and read.

That's it we have a queue and user account so we can access the queue

Let's write some code

The below sample is in C# for the code see Github

AWS SQS Read Console Sample Application

AWS SQS Write Console Sample Application

Write a message to the queue.

AWS SQS Write Console Sample Application

In a C# console app, in Package manager run

PM> Install-Package AWSSDK.SQS

This will install the SDK for SQS so we can get going. Here is the code for that will write a message to the queue.

        string sQueue = "https://sqs.eu-west-1.amazonaws.com/{CODE}/{QUEUENAME}";

        try
        {
            var config = new AmazonSQSConfig()
            {
                ServiceURL = "https://sqs.eu-west-1.amazonaws.com/"
            };

            var _messageRequest = new SendMessageRequest();

            _messageRequest.QueueUrl = sQueue;

            _messageRequest.MessageBody = "This is a test message";

            AmazonSQSClient _client = new AmazonSQSClient("ACCESSKEY", "ACCESSSECRET", config);

            SendMessageResponse sendMessageResponse = _client.SendMessage(_messageRequest);

        }
        catch (Exception ex)
        {
            Console.WriteLine("Error: " + ex.ToString());
        }

So lets take a wander through what we have just done.

First up we have the Queue URL we looked at earlier, I store it off in a string.

We then set the config.

            var config = new AmazonSQSConfig()
            {
                ServiceURL = "https://sqs.eu-west-1.amazonaws.com/" 
            };

The Config Service URL should be the service, region and then amazonaws.com This will tell the SQS Client where we want to connect.

Next we create our MessageRequest

            var _messageRequest = new SendMessageRequest();

We set the Message Request Queue URL to the string we set at the top ("https://sqs.eu-west-1.amazonaws.com/{CODE}/{QUEUENAME}")

            _messageRequest.QueueUrl = sQueue;

            _messageRequest.MessageBody = "This is a test message";

Now we want to set the Amazon SQS client. We pass our Access key and Access Secret we setup in the IAM and then pass the config which will set everything to the region we want.

            AmazonSQSClient _client = new AmazonSQSClient("ACCESSKEY", "ACCESSSECRET", config);

We now send the message.

            SendMessageResponse sendMessageResponse = _client.SendMessage(_messageRequest);
Read a message to the queue.

Sample Code here

In a C# console app, in Package manager run

PM> Install-Package AWSSDK.SQS

To read the message we just created our code will look like this

        string sQueue = "https://sqs.eu-west-1.amazonaws.com/{CODE}/{QUEUENAME}"; 

        try
        {
            var config = new AmazonSQSConfig()
            {
                ServiceURL = "https://sqs.eu-west-1.amazonaws.com/" 
            };
            
            AmazonSQSClient _client = new AmazonSQSClient("ACCESSKEY", "ACCESSSECRET", config);

            ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest();

            receiveMessageRequest.QueueUrl = sQueue;

            ReceiveMessageResponse receiveMessageResponse = _client.ReceiveMessage(receiveMessageRequest);
            
            foreach (var oMessage in receiveMessageResponse.Messages)
            {
                Console.WriteLine(oMessage.Body);

                // Delete the message from the queue
                DeleteMessageRequest deleteMessageRequest = new DeleteMessageRequest();

                deleteMessageRequest.QueueUrl = sQueue;
                deleteMessageRequest.ReceiptHandle = oMessage.ReceiptHandle;

                DeleteMessageResponse response = _client.DeleteMessage(deleteMessageRequest);
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("Error: " + ex.ToString());
        }

Now lets take a wander through what we just did.

        string sQueue = "https://sqs.eu-west-1.amazonaws.com/{CODE}/{QUEUENAME}";

As with the writing a message I like to load the URL into a string.

            var config = new AmazonSQSConfig()
            {
                ServiceURL = "https://sqs.eu-west-1.amazonaws.com/" 
            };

We create the config with a ServiceURL that will point to the SQS in Dublin (eu-west-1)

            AmazonSQSClient _client = new AmazonSQSClient("ACCESSKEY", "ACCESSSECRET", config);

We create the AmazonSQSClient with the access key, Secret and the config.

            ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest();

            receiveMessageRequest.QueueUrl = sQueue;

We set the Queue URL to the URL from our Queue

             ReceiveMessageResponse receiveMessageResponse = _client.ReceiveMessage(receiveMessageRequest);

We create a receive Message response and call the Client and ask for what messages are on the queue.

            foreach (var oMessage in receiveMessageResponse.Messages)
            {
                Console.WriteLine(oMessage.Body);

We can now do a foreach on the ReceiveMessageResponse Messages this will allow us to read each message.

AWS SQS needs you to delete the message once it's read.

                // Delete the message from the queue
                DeleteMessageRequest deleteMessageRequest = new DeleteMessageRequest();

                deleteMessageRequest.QueueUrl = sQueue;
                deleteMessageRequest.ReceiptHandle = oMessage.ReceiptHandle;

Each message has a receipt handle, we set the Queue URL to our Queue we are reading from.

                DeleteMessageResponse response = _client.DeleteMessage(deleteMessageRequest);

That's it we've now created and read a message from the Amazon Simple Queue Service.