Retrieving all past messages from a apache pulsar topic

Question

I think a simple example would describe my question better.

For example, let's say there is a topic named "A" and I have produced 100 messages(message1...message100). I have already consumed and acknowledged up to message 50 using subscription "A_1" with type exclusive. For some reason, my application shuts down, so when restarting the application, I need to read from message 1 again. Can this be achieved? I was thinking it would be possible to create a new subscription("A_2") and start reading messages again but i was unsure whether "A_2" would start reading from message1 or message51.... any directions or hints would be great!

Thanks in advance

Answer 1

Yes this can be achieved, all that is required is to create a new subscription, e.g. "A_2", and use the subscriptionInitialPosition parameter to specify that you want to start consuming messages from the earliest available message as shown:

return getClient().newConsumer()
        .topic(topic)
        .subscriptionName("A_2") 
        .subscriptionType(SubscriptionType.Exclusive)
        .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
        .subscribe();

This assumes that the messages haven't been deleted due to message retention policies.

Answer 2

Keeping the receiving of the message and the acknowledgement separate in Pulsar means its possible to write your application so that it only acknowledges message when they are not needed anymore.

It might be useful to consider only acknowledging the message when you're sure the application doesn't need it anymore.

Otherwise, what @david-kjerrumgaard said.