got 'CancellationException: Request execution cancelled' always when throwing an exception in httpasyncclient callback

Question

I use HttpAysnClient to do http requests, and I found when I throw an exception in the failed callback, the next request always be failed, how to fix it?

I use maven dependency: 'org.apache.httpcomponents:httpasyncclient:4.1.5'.

my java test code:

CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();

try {

    httpclient.start();

    AtomicBoolean fireException = new AtomicBoolean(false);

    while (true) {

        try {

            String url;

            if (fireException.compareAndSet(false, true)) {

                url = "http://localhost:8080"; // throw Connection refused

            } else {

                url = "http://www.apache.org/";

            }

            final HttpGet request2 = new HttpGet(url);

            httpclient.execute(request2, new FutureCallback<HttpResponse>() {

                public void completed(final HttpResponse response2) {

                    System.out.println("completed, " + request2.getRequestLine() + "->" + response2.getStatusLine());

                }

                public void failed(final Exception ex) {

                    System.out.println("failed, " + request2.getRequestLine() + "->" + ex);

                    throw new RuntimeException();

                }

                public void cancelled() {

                    System.out.println(request2.getRequestLine() + " cancelled");

                }

            });

            TimeUnit.SECONDS.sleep(1);

        } catch (Exception e) {

            e.printStackTrace();

            TimeUnit.SECONDS.sleep(1);

        }

    }

} finally {

    httpclient.close();

}

exception in the next requests: java.util.concurrent.CancellationException: Request execution cancelled

Answer 1

you can see doc in https://hc.apache.org/httpcomponents-client-5.1.x/migration-guide/migration-to-async-simple.html

and if you want to use CloseableHttpClient you must start it client.start();

Answer 2

I can confirm same behavior with version 4.1.5.

I must confess it is quite surprising to see an application uncontrolled exception shutting down the whole client unexpectedly. In the context of an application reusing same client instance in multiple places, means the application client gets completely unsuable, with catastrophic consequences for the service.

You can use the "isRunning" method to evaluate if the client is under this situation, and potentially try to recreate the client again. But it is definately incovenient to see the client being shutdown like this.

After exercising the client with different conditions (error responses, slow responses...), the only way to reproduce this is to point to an invalid endpoint where no server is running. This is the condition presented in the original example.

I think I found the issue here https://jar-download.com/artifacts/org.apache.httpcomponents/httpasyncclient/4.1.5/source-code/org/apache/http/impl/nio/client/InternalIODispatch.java You can see onException doesn't have a try/catch block to properly handle exceptions from the application.

I have confirmed this issue is fixed in Httpclient5 5.1.3. So other than fixing your application code to avoid uncontrolled exceptions, the solution is to migrate into the new Httpclient5 lib version.