My team and I are trying to build an in-house instant messaging solution using websockets. We've designed it to be horizontally scalable using pub/sub technology. Currently, we're using a single-node software load balancer and we are aware that there's only a limited number ephemeral ports available from the load balancer to any of the websocket servers (~64k). Is it possible to load balance websocket connections without running out of ephemeral ports? We are trying to get more than 100k concurrent connections.
Ideally, we want something like the following if it is at all possible.
The handshake is load balanced across the servers and subsequently a direct websocket connection is established between the client and a server. No TCP connections going through the load balancer at all.
We tried using TCP/UDP load balancing with Nginx, expecting that a direct connection would be established between the client and a server. But upon checking the remote address of the connection from the server, it was still coming from the load balancer node. To further confirm this, we turned off the load balancer and observed from the server that the connection was dropped immediately as a result. So, there were still TCP connections going through the load balancer node.
We are aware that DNS-level load balancing can solve this problem but it's simply not an option for us. Also, hardware load balancers are out of the picture.
We are eager to hear your answers.