Simultaneous http post request spring boot

Question

Hi, I have a list with size of 500k and I have to make a request to a server with hash parameters.

The server accepts JSON Array of 200 objects. so I could send 200 items each time. But still I need to split the list each time and send that part to server.

I have a method for this which makes the http post request. and I want to use spring boot options (if available) to call the method with different threads and get the response back and merge them into one.

Answer 1

I did it using java CompletableFuture class without any springboot tags. but you Could use @async for your method too. sample code :

        var futures = new ArrayList<CompletableFuture<List<Composite>>>();
        var executor = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());

        for (List<CompositeRecord> records : Lists.partition(recordsList, 200)) {
            var future = CompletableFuture.supplyAsync(() -> /* call your method here */, executor);
            futures.add(future);
            Thread.sleep(2000);
        }

        CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).exceptionally(ex -> null).join(); // avoid throwing an exception in the join() call
        var futureMap = futures.stream().collect(Collectors.partitioningBy(CompletableFuture::isCompletedExceptionally));
        var compositeWithErrorList = new ArrayList<Composite>();
        futureMap.get(false).forEach(l -> {
            try {
                compositeWithErrorList.addAll(l.get());
            } catch (InterruptedException | ExecutionException e) {
                e.printStackTrace();
            }
        });

after the code is executed you will have a map of done and undone futures.