Apache 5 HttpClient Retry Strategy Not Working

Question

I am trying to add a custom retry strategy to my HttpClient such that it retries on SocketTimeoutException and NoHttpResponseException. However, the custom retry strategy is not getting invoked in case of these errors. Please find the code snippets below:

Creating HttpClient:

    val requestConfig = RequestConfig.custom()
        .setConnectTimeout(Timeout.ofMilliseconds(20))
        .setResponseTimeout(Timeout.ofMilliseconds(1000))
        .setConnectionRequestTimeout(Timeout.ofMilliseconds(200))
        .build()

    val retryStrategy = CustomRetryStrategy(
        3, // Retry 3 times
        TimeValue.ofMilliseconds(10) // Retry very quickly
    )
    

    val client = HttpClientBuilder.create()
        .setDefaultRequestConfig(requestConfig)
        .setRetryStrategy(retryStrategy)
        .build()

Retry Strategy Code:

class CustomRetryStrategy(private val maxRetries: Int, private val defaultRetryInterval: TimeValue) :
    DefaultHttpRequestRetryStrategy(maxRetries, defaultRetryInterval) {

    override fun retryRequest(request: HttpRequest, exception: IOException, execCount: Int, context: HttpContext?): Boolean {
        if (execCount > this.maxRetries) {
            // Do not retry if over max retries
            return false
        }

        if (exception is SocketTimeoutException ||
            exception.cause is SocketTimeoutException ||
            exception.cause is NoHttpResponseException ||
            exception is NoHttpResponseException
        ) {
            println("Retryable Exception Encountered. Retrying")
            return true
        }
        return super.retryRequest(request, exception, execCount, context)
    }
}

Execute Request Code:

val response = client.execute(request)

May I know what am I missing or doing wrong which is leading to retries not getting executed?

Answer 1

The Default retry strategy tests if the call is Idempotent (can make multiple calls with the same result as if it had made 1) most likely your call fails this test. I am working with the 5 version not the 4 but should be similar. In 5 you need a custom HttpRequestRetryStrategy class that implements HttpRequestRetryStrategy, if you base this off the DefaultHttpRequestRetryStrategy you can update the function handleAsIdempotent to return true for cases you want to retry.

There may be a better way but this is what I found tracing through the library code to fix an issue after not finding a solution posted somewhere.

Answer 2

In case someone else find themself in this page and like to see some java code doing custom retry strategy for all errors/exceptions because default one is excluding some.

public static void main(String[] args) throws IOException {
    final ClassicHttpRequest httpGet = ClassicRequestBuilder.get("https://httpbin.org/status/500")
            .build();

    CloseableHttpClient httpClient = HttpClientBuilder
            .create()
            .setRetryStrategy(new CustomRetryStrategy(3, TimeValue.ofSeconds(3)))
            .build();

    httpClient.execute(httpGet, response -> {
        System.out.println(response.getCode() + " " + response.getReasonPhrase());
        return null;
    });
}

static class CustomRetryStrategy implements HttpRequestRetryStrategy {
    private final int maxRetries;
    private final TimeValue retryInterval;

    public CustomRetryStrategy(final int maxRetries, final TimeValue retryInterval) {
        this.maxRetries = maxRetries;
        this.retryInterval = retryInterval;
    }

    @Override
    public boolean retryRequest(
            final HttpRequest request,
            final IOException exception,
            final int execCount,
            final HttpContext context) {
        Args.notNull(request, "request");
        Args.notNull(exception, "exception");

        if (execCount > this.maxRetries) {
            // Do not retry if over max retries
            return false;
        }
        return true;
    }

    @Override
    public boolean retryRequest(
            final HttpResponse response,
            final int execCount,
            final HttpContext context) {
        Args.notNull(response, "response");

        return execCount <= this.maxRetries;
    }

    @Override
    public TimeValue getRetryInterval(HttpResponse response, int execCount, HttpContext context) {
        System.out.println("Retrying HTTP request after " + retryInterval.toString());
        return retryInterval;
    }
}