Forkjoinpool behavior vs ExecutorService

177 Views Asked by At

We have a scenario to insert ~4M records into database. We receive these records in batches of 56 files, each containing ~80k records.

Used Forkjoinpool to insert the records in batches of 1000. Once all the threads are completed we proceed further.

Noticed that Forkjoinpool.get() is not blocking. It doesn't wait for the thread to complete. This is causing some of the records to be missed(~5k) saving in the database.

Replaced Forkjoinpool with ExecutorService which works as expected. All the ~4M records are saved.

Wanted to understand why Forkjoinpool.get() is not blocking.

Both the approaches take ~5 mins to complete. Performance wise no issues.

ForkJoinpool

//Partition 80000 records into chunks of 1000 records
List<List<EntityObject>> entitiesPartitonList = Lists.partion(EntityObjectsList);

ForkJoinPool filesFJP= new ForkJoinPool(80);
filesFJP.submit(()->entitiesPartitonList.parallelStream().foreach(entitesBatch->{
            jdbctemplate.batchupdate(query,entitesBatch.toArray(new Map[entitesBatch.size]))
})).get();

ExectorService

List<Future<Boolean>> futures = new ArrayList<>();
ExecutorService es=Executors.newFixedThreadPool(80)

entitiesPartitonList.foreach(entitesBatch->futures.add(es.submit(()->{
        jdbctemplate.batchupdate(query,entitesBatch.toArray(new Map[entitesBatch.size]))
})));

futures.foreach(fut->fut.get());
0

There are 0 best solutions below