How to set the maximum simultaneous requests that can be handled by a servlet using Tomcat

Question

I wanted to know how many simultaneous requests a web application built using a servlet can handle (using Tomcat 7). I have configured maxThreads, acceptCount, minSpareThreads attributes of server.xml file.

My question is: Do we have to take the underlying OS into consideration as well?

For example:

maxThreads= 5;
acceptCount= 1;
OS= 1 processor with 4 cores; 

So at maximum 4 requests can be handled simultaneously with 1 request waiting? Assuming that each request will take some-time to complete. Do we also have to take into consideration hyperthreading?

Answer 1

Good Question, but there is no straight answer. You need to tweek the server parameters yourself and test it against the load.

Few question that arises my mind.

1.) How much time does each request takes to process.

2.) How many threads are being used/configured in thread pool.

3.) How many server connections have been configured.

OS, server will not play much bigger role in this. Once you are done setting parameters, pushing them to there limits, then we might even to think of hardware.

In server.xml file you specify maxThreads which specifies maximum number of simultaneous requests that can be handled.If not specified, this attribute is set to 200.

From stats/blogs

A single Tomcat server with default settings on modest hardware should easily handle 2k requests/second, assuming it doesn't have too much work to do per request. If processing one request takes 500+ ms, you'll probably need to bump up the number of threads in the thread pool, and you might start pushing the limits.

Answer 2

There is no easy way to come up with these numbers by just guessing. Usually, you don't have to specify anything at the risk of the machine getting clogged. But at the same time, the network interface is so slow that it's hard to congest a web server (unless each requests takes a long time to process) before you congest the network.

My suggestion is to leave these values alone and monitor the server to see how it behaves under load. If you can, do some load tests - there are some great tools out there which can simulate thousands of users using your server at the same time.

Also from experience: The database is the bottleneck. There will be a couple of queries that will bring everything to a halt. It's hard to tell which ones in advance because the performance depends on so many things: Amount of data in the database, indexes, actual SQL being executed, disk speed, network between DB and web server to name just the most important ones.