How to handle multiple threads in ```Executor Service ``` java

Question

I am working on a solution, where for a single customer, I need to invoke an API to get the employee list of that customer. But this API (third party) returns only 100 employees at once. So, I need to invoke the same API by changing the offset every time to get the next set of 100 employees. This is currently handled using single thread, as a result the time for retrieving all the employees (lets say 30k) increases as per the count of employees. And the application has many such customers.

As a solution for this, I am trying to implement multithreading using ExecutorService. With this solution the time taken for retrieving all the employees for a customer has come down.

Question:

1. How does multiple threads (from multiple customers) work with Executor Service which in turn has multiple threads for invoking the API?
2. Does the below logic in mutlithreaded environments result in incorrect data for enduser? Sample code:

  ExecutorService executor = Executors.newFixedThreadPool(2);

  ServiceExecutor executor0 = new ServiceExecutor("0");
  ServiceExecutor executor1 = new ServiceExecutor("100");

  Future result0 = executor.submit(executor0);
  Future result1 = executor.submit(executor1);
   
  List<String> s1 = new ArrayList<>();
  s1.add(result0.get());
  s2.add(result1.get());

Answer 1

How does multiple threads (from multiple customers) work with Executor Service which in turn has multiple threads for invoking the API?

An executor can be backed by zero, one, or more threads.

You need no other threads. The job of an executor service is to manage the threads for you.

To get the employees for each of multiple customers, use the same logic as seen below.

Make an executor service backed by virtual threads. Loop those customers, passing each to the constructor of a task that executes the code seen below. Post the results to a thread-safe Map implementation.

Map< Customer , Collection < Employee > > eachCustomerAndTheirEmployees = new … ;

Does the below logic in mutlithreaded environments result in incorrect data for enduser? Sample code:

No such class as ServiceExecutor seen in your code is bundled with Java.

Regarding Executors.newFixedThreadPool(2), your situation calls for virtual threads in Java 21+.

ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor() ;

In modern Java, an executor service is AutoCloseable. So we can use convenient try-with-resources syntax.

Collection < Employee > employees = new CopyOnWriteArrayList<>() ; // Collection must be thread-safe!
try 
(
    ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor() ;
)
{
    int pages = … ;  // determine page count. 
    for( int page = 1 ; page <= pages ; page ++ )
    {
        executorService.submit( new PageProcessor( page , employees ) ) ;
    }
}
// At this point execution blocks until all tasks are done (or a day has passed — see source code of `close` method). 
… Use the now-populated collection of Employee objects. 

Write your task as a class that implements Runnable or Callable. Here we have such a class named PageProcessor. We pass to its constructor the page number (the nth batch of 100 rows to be retrieved). Our task adds the retrieved Employee objects to the thread-safe collection passed in to its constructor.

Answer 2

How does multiple threads (from multiple customers) work with Executor Service which in turn has multiple threads for invoking the API?

The answer for this highly depends on how your application is set-up to receive requests from customers. Is a new executor service created for every incoming request or is the executor service shared across all requests? Creating a new executor service per request would be very wasteful, as you'd have to allocate space for new threads that aren't going to exist for very long. Thus it is much more efficient to re-use the same ExecutorService for multiple requests.

It's probably more flexible in this instance to use the Executors.newCachedThreadPool() instead of the newFixedThreadPool() as this allows the ExecutorService to dynamically increase the number of threads in the pool as necessary.