Managing MQTT Connections: To Close or Not to Close After MQTT Message Publish?

Question

A message is sent to the broker from an IoT device every 15 minutes. Currently, after each message is published, the MQTT connection is closed. Is this a suitable approach for a network of 70k IoT devices, or would it be more appropriate to maintain a persistent mqtt connection with a keepalive setting?

Additional Info:

1. These are AC powered IoT gateways having NBIoT Modem which sends periodic electricity meter data over MQTT.
2. All the devices sends message at fixed interval of 15 minutes starting from 12:00 onwards.
3. Device remains active throughout to receive any incoming messages from server (head end system) over CoAP.
4. EMQX broker is used

Answer 1

All the devices sends message at fixed interval of 15 minutes starting from 12:00 onwards.

So this basically means you are creating a 70k client connection storm every 15mins.

I'm not surprised that devices are failing to connect.

I would suggest your best option is to keep the connection open all the time.

Since power is not a problem the only factor to consider here is how much bandwidth the keep alive pings will cause on the NBIoT connection, but this can be mitigated by increasing the ping time to something like 5mins.

The other option (in addition to keeping connected) is if the message payload contains a timestamp the devices could add some random jitter to when the devices publish their message, this way the broker is not also dealing with 70k publishes all happening in the same second. Given the data is only at 15min resolution +/-2 (or more) mins across the devices would also help reduce load on the broker.

Answer 2

It's not asked, but NB-IoT in combination with TCP isn't really a good solution. Search in the forums of modem manufacturer and you will find quite a lot of "pain stories". If you use NB-IoT with TCP you will mostly not be able to benefit from the "deep building penetration", so when it works in your lab, it doesn't proof, that it works in a buildung base. And I don't get the part with CoAP. If you have also CoAP, why do you send the data over MQTT/TCP? May be a CoAP/MQTT Proxy works better.

Response: We agree with your point, but this solution design was finalized by a third-party consultant, and as solution providers, we are bound by the agreed-upon project scope.

Now our focus is mainly on improving the stability of the already finalized solution.

Regarding the 'deep building penetration', as we have observed, the NB-IoT network provided by the operator is quite good. In addition to this, in cases where low network coverage is observed, the operator is installing high-gain antennas, repeaters, etc., to ensure good network availability. In fact, the NB-IoT network operator is the project owner; hence, worry on the network side is minimal.

Regarding the point 'why we have MQTT/TCP when we already have CoAP,' again, we have to stick to the agreed project scope and solution.