CircuitBreaker Not Changing State from HALF_OPEN to CLOSED

Question

I have this circuit breaker configuration in my spring-boot reactive application -

CircuitBreakerConfig.custom().failureRateThreshold(5)
            .slowCallDurationThreshold(Duration.ofMillis(5000))
            .slidingWindowType(SlidingWindowType.COUNT_BASED)
            .slidingWindowSize(5)
            .permittedNumberOfCallsInHalfOpenState(5)
            .waitDurationInOpenState(Duration.ofMillis(30000))
            .slowCallRateThreshold(5)

then I am calling the upstream API like this -

return Mono.from(invokeUpstream(body, headers))
                .compose(CircuitBreakerOperator.of(circuitBreaker)).onErrorResume(this::fallback);

the invokeUpstream method looks like this -

Mono<ResponseEntity<String>> response = webClient.post()
                                                   .body(BodyInserters.fromValue(body))
                                                   .retrieve()
                                                   .onStatus(HttpStatus::is5xxServerError, clientResponse -> Mono.error(new BadGatewayException()))
                                                   .toEntity(String.class);

And the fallback method simply throwing exception -

private Mono<ResponseEntity<String>> fallback(Throwable ex) {
    throw new BadGatewayException();
}

Now, when the upstream API returns 500 response 5 times, I can see the circuitBreaker state is moving to OPEN state from CLOSED state, which is expected. In OPEN state it stays for 30 seconds as per the configuration. After that it moves to HALF_OPEN state and the problem begins here. Even if the upstream API returns success response, it never moves to CLOSED state, it stays in HALF_OPEN state forever.

I am using these dependencies in my application -

    <dependency>
        <groupId>io.github.resilience4j</groupId>
        <artifactId>resilience4j-circuitbreaker</artifactId>
        <version>1.3.1</version>
    </dependency>
    <dependency>
        <groupId>io.github.resilience4j</groupId>
        <artifactId>resilience4j-reactor</artifactId>
        <version>1.3.1</version>
    </dependency>

I am not sure what I am doing wrong even after going through all the documentation.

Answer 1

I found the issue. I was checking the circuit breaker state using

circuitBreaker.tryAcquirePermission()

method even before making the actual call which was causing this issue. After removing this, it's working perfectly.