How to throttle message delivery to consumers?

Question

I am looking for a message queue with an in-built throttling feature. Use case is that the recipient worker pool may accept a lot of messages but a service that workers depend on may not be able to handle the load. It's not possible to reduce the worker pool since the worker instances handle different types of messages.

So the feature I am looking for is throttling based on a topic. Say a topic T, I want the queue to accept as many messages from the producers, but throttle the demand from consumers on topic T to say deliver only 5 messages per minute.

Answer 1

For Java following solutions may work, They should be also available for node

• You can throttle messages by controlling your consumer's poll() calls.

• Try to sleep thread between call to poll()

• Try to use more timeout in poll() and use following property
  MAX_POLL_RECORDS_CONFIG to control how many messages you will
  receive in a single poll.

Answer 2

If you are using IronMQ, you can throttle messages if you are using "pull" queues. This feature will need to be done manually from the user's code. If you were to use "push" queues, you would not be able to throttle messages. However, your consumers will receive the messages at the highest rate. Here is a link describing push and pull queues: https://dev.iron.io/mq/3/reference/push_queues/index.html

If you have additional questions, please reach out to Iron.io support via chat, email, or phone.