: Messages Queue services like Amazon SQS, IronMQ, what are they exactly? I need to understand message queue services. Available services I know out there are Amazon SQS and IronMQ. What are

@Martha676

Pretty old post but worth adding an answer to.

check out the similar question on quora www.quora.com/Message-Queuing/What-are-some-use-cases-for-message-queues-in-real-life
Here are some usecases for the queue from Iron's Blog blog.iron.io/2012/12/top-10-uses-for-message-queue.html
Some information about popular queues

Redis, message queue open sourced, is great for a simple message broker but messages can be lost.

RabbitMQ, self hosted message queue open sourced, use a different protocol 'AMQP', is very popular but requires you to adapt the protocol and manage your own nodes. CloudAMQP is the only hosted rabbit solution I know of.

Amazon SQS, hosted message queue expected message latency is pretty high and has the possibility of messages being delivered twice. It is quite stable but lacks feature development from Amazon's end in my opinion.

IronMQ, a hosted message queue is pretty great for it's single message delivery, fifo, and use the HTTP protocol over AMQP and is elastically scalable, private clusters, error/alert queues, gui, yadda yadda awesome!

Celery, self hosted task queue open sourced, task queue, has support for scheduling, host your own node. primarily python support

Resque, task queue open sourced, built on the above mentioned redis, requires you host your own nodes and manage their scaling, ruby based, works well with sidekiq.

IronWorker, hosted task queue AND the processing - gives you the complexity of being a task queue AND your task processing layer in one package. supports scheduling, supports all languages, and is elastically scalable, yadda yadda awesome!

Delayed_job, task queue open sourced, uses local database as the queue, not very flexible or efficient but can get the job done. ruby based.

Vote Up Vote Down

@BetL925

A message queue service allows multiple servers to communicate, especially when they perform different roles. Amazon has a whitepaper about SQS. It gives the following example:


Example: Online Photo Processing Service for Consumers

Assume you want to offer an online photo processing service for consumers. The services lets consumer specify the operations they want performed on their photos. Operations could include redeye reduction, cropping, thumbnail creation, customization, re-coloring, teeth whitening, etc.

Users upload the photos to your website and specify the tasks to be performed on the photos. Users can submit as few as one or as many as hundreds of photos in a single upload session. After submitting the photos, users can come back and check on the status of their photos. When the photos have been processed, they can download the photos from the web site. Let's assume that different operations take different processing times, ranging from a few seconds to several minutes. Therefore, the time to complete the user's request depends on the number of photos, the size of the photos, and the processing operations to be performed.


It goes on to show how the message queue service can be used to communicate between the web server and the photo processing server. The web server puts messages in the queue stating that work needs to be done. The photo processing server sends messages back stating that the work has been performed.

When properly implemented, this allows web sites to scale more easily. In this example, it would be fairly easy to add more photo processing servers and have all of them listen to the same queue for work to be performed.

Vote Up Vote Down