How to scale http requests horizontally?

Question

Ok, say I have simple web-server instance which always responses "Hello world" and this simple server can handle only 1K connections simultaneously.

I have to handle 1M http-requests simultaneously, so I clone 1K nodes with my web-server instance.

But I have only one entry point. All requests come through one point.

So what should be placed as entry point to delegate requests? What software can pipe 1M requests to 1K web-servers?

Answer 1

If you use round-robin DNS, the requests can come through multiple points.

If you want software for reverse proxy on Linux, Nginx is the way to go.

node-http-proxy also seems quite popular.

Golang has a ReverseProxy built into its httputil package.

(If I'm not mistaken, all three above will use epoll on Linux. But there is no guarantee any software can always handle 1M connections. It depends on many factors.)

Load balancing is a broad topic. Here are a few links for reading:

https://en.wikipedia.org/wiki/Round-robin_DNS

https://en.wikipedia.org/wiki/C10k_problem

https://en.wikipedia.org/wiki/Clustered_web_hosting

https://en.wikipedia.org/wiki/Reverse_proxy

https://en.wikipedia.org/wiki/Load_balancing_(computing)