When I install a Mosquitto Broker and publish messages to a topic and subscribe the the messages using an Apache Beam MQTTIO pipeline and print the message in the console, I am able to get all the messages.
Even after a gap of 5 minutes if I publish a message, I am able to see the message in the console of the Beam application
Now I changed nothing but the ServerUri in the ConnectionConfiguration of MQTTIO.Read() and gave a corresponding topic that is available in the that broker. Refer Documentation
This change made the application work differently, only the topics that have higgh frequency of messages are being printed in the console. When the message frequency if 1 message per minute or somewhere near to that range of frequency, only the first message is getting printed in the console.
I even tried out withMaxNumRecords and withMaxReadTime but still it just lists the first message.
Is there Some timeout that is happening in the broker configuration because of its idlenesss between messages?
The same behaviour was observed in runners - DirectRunner and FlinkRunner
Changing the options to --streaming=true for the pipeline using the arguments did not work as well