RabbitMQ: Bi-directional federated brokers, how do you make all queues pop the message?

Question

I have two brokers configured[1] with federation plugin. Both are pointing to each other as upstream.

My test is:

• publish a message on broker A
• consume on broker B

The result is:

• consuming on broker B works
• < good > the queue on broker B pops the message
• < not good > the queue on broker A still has the message
  • < reason why this is not good > The issue I see is: if I always publish on one broker and then always consume on the other --> then the queue on the publishing broker will grow until it's full and start dropping messages.

The result I would like is:

• both queues on broker A and B pop their messages when the consumer consumes on broker B

How do I configure RabbitMQ to pop the message from all queues when a consumer consumes the message on broker B? Right now I am trying to do so with RabbitMQ Federation plugin.

---

[1] The two brokers point to each other as upstreams and I configure them the same way as described in the "simple example" given by the documentation except that there are two brokers each pointing to each other as upstream. The code for the publisher looks like this and the code for the consumer looks like this.

Answer 1

I created this environment:

Server A, Server B.

Created a bidirectional federated in this way:

Federation Upstream: Server_B = amqp://servera

Federation Upstream: Server_A = amqp://serverb

Then created the same policy on both the servers:

Pattern : ^fed\.
Apply to: all   
federation-upstream-set:all

Created one queue to the Server A called: fed.test1 then created a consumer to the Server B as:

 ConnectionFactory factory = new ConnectionFactory();
 factory.setHost("localhost");
 factory.setPort(5673);
 Connection connection = factory.newConnection();
 Channel channel = connection.createChannel();


Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
                        throws IOException {
String message = new String(body, "UTF-8");
System.out.println(" [x] Message   '" + message );
                }
            };
channel.basicConsume("fed.test1",  true, consumer);

Then published a message to the Server A ---> fed.test1

The message has been consumed to the Server B and the queues messages count is zero to both the queues (Server A, Server B).

This works as you expected.

Hope it helps.

Answer 2

@Trevor Boyd Smith, probably option 2 or 3 as shown below are something you could consider.

Option 1: Bidirectional federated exchange

A message will end up in the both broker A and B, one copy each broker, independent of each other. In other words, e.g., even after broker B has delivered the message to its consumer, the other copy of the message still remains in broker A.

Advantage: You will always have two copies of the message, one in each broker, which is highly available.

Disadvantage: You need to have a consumer connected to each broker.

Option 2: Bidirectional federated queue

A message will end up in one of the two brokers. By default, the broker where the message has been published will have the priority to enqueue the message; however if only the other broker has got a consumer, the message will move to the other broker.

It does not matter which broker the message ends up in, the message will be delivered once and once only by a consumer connected to either of the brokers.

Advantage: The message will be delivered once and once only to a consumer connected to either of the brokers.

Disadvantage: There will be only one copy of the message. If the broker that has got the message goes down, the other broker can not get the message. But if you are fine with eventual consistency, this option is OK. The reason is when the problematic broker comes back up running, the message would be available, eventually.

Option 3: Bidirectional federated exchange and queue

In this case, a message will end up in both brokers, one copy each broker. The same message will be delivered to a consumer connected to either of the brokers, twice! Once the message has been delivered twice, it will be gone in both brokers. (If there are two consumers, one connected to each broker, each broker will deliver the same message to its consumer once.)

Advantage: The consumer can connect to either of the brokers, and the message in each broker will be delivered and dequeued.

Disadvantage: The same message will be delivered twice. The workaround would be, before handling the message, check if the same message has already handled.

Note:

It does not mean which option is better than the other one or ones. It all depends on your use case, and there are many other configurations that can come into play, which could change the behaviour.