HttpClient asyncRequest and exponential backoff

Question

I need to implement an exponential backoff on a request that might fail. However, it's implemented as an async request. Had this been done synchronously, I'd have a better idea on where to put the delay. Roughly, I'm thinking it'd work something like this:

// These would be configurable in application.yml
currentAttempt = 0;
maxAttempts = 3;
timeoutGrowth = 2;
currentDelayTime = 5ms;
repeatNeeded = false;
while(repeatNeeded && currentAttempt < maxAttempts) {
    httpStatusCode = makeRequest(someService)
    if(httpStatusCode == 503) {
        repeatNeeded=true;
        currentAttempt++;
        currentDelayTime*=timeoutGrowthRate;
        sleep(currentDelayTime)
    }
}

However, with an async call, the caller to the function is given the time back to do something else until the Future is has something. Do I code the backoff within the getObservations() method below, or do I code this in the caller of that getObservations() method? Below is the call as it currently is:

public CompletableFuture<ToolResponse> getObservations(String text, Map<String, Object> bodyParams) throws URISyntaxException {
        URI uri = getUri(text);
        HttpRequest request = getRequest(uri, text, bodyParams);
        Map<String, String> contextMap = Optional.ofNullable(MDC.getCopyOfContextMap()).orElse(Collections.emptyMap());

        Instant startTime = Instant.now();

        return httpClient.sendAsync(request, BodyHandlers.ofString())
                .exceptionally(ex -> {
                    throw new ExternalToolException(externalServiceConfig.getName(), ex);
                })
                .thenApply(response -> {
                    long toolRequestDurationMillis = ChronoUnit.MILLIS.between(startTime, Instant.now());
                    if (HttpStatus.valueOf(response.statusCode()).is2xxSuccessful()) {
                        ToolResponse toolResponse = processResponse(response, toolRequestDurationMillis);
                        logToolResponse(toolResponse);
                        return toolResponse;
                    }

                    log.error("{} returned non-200 response code: {}", externalServiceConfig.getName(), response.statusCode());
                    throw new ExternalToolException(externalServiceConfig.getName(), response.statusCode());
                });
}

Answer 1

If you could consider using reactive java that has very powerful API including retries. For example,

request()
  .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));

there are more options like retries for the specific exceptions only or defining max backoff

request()
  .retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
  .maxBackoff(5)
  .filter(throwable -> isRetryableError(throwable))

You could use WebClient that is a non-blocking client exposing a fluent, reactive API over underlying HTTP client libraries such as Reactor Netty

webClient.get()
        .uri("/api")
        .retrieve()
        .bodyToMono(String.class)
        .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));

if for some reason, you still want to use HttpClient you can wrap CompletableFuture

Mono.fromFuture(httpClient.sendAsync(request, BodyHandlers.ofString()))
  .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));