4xx/5xx count metric by exception type

Question

We have setup a spring-boot service & we want to plot quantum of 4xx/5xx by Exception-type.

Below is the Prom-QL we are using :-

sum by (exception,uri,status) (increase(http_server_requests_seconds_count{kubernetes_namespace=~"$namespace",uri!~".*health.*|.*actuator.*|.*error.*",status=~"5.*"})[1m])

The issue we are getting is, exception-type us always coming as None. in the metrics.

Controller :-

@PostMapping(APIPath.Endpoint)
public ResponseEntity<Response> predictRTO(@RequestBody @Valid request, Errors errors) {
    try {
        if (errors.hasErrors()) {
            throw new RequestParamsException(errors.getAllErrors());
        }
        Response response = service.getResponse(request);
        return ResponseEntity.ok(response); 
    } catch (RequestParamsException e) {
   return ResponseEntity.status(HttpStatus.BAD_REQUEST).body(Response.builder().errors(e.getErrors()).build());
    } catch (Exception e) {             
        return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(Response.builder().errors(Collections.singletonList(
                CommonConstants.INTERNAL_SERVER_ERROR)).build());
    }
}

I have tried hitting the actuator/prometheus, there also we are getting 5xx/4xx with None as an exception.

http_server_requests_seconds_bucket{exception="None",method="POST",outcome="SERVER_ERROR",status="500",uri="/endpoint",le="0.111848106",} 0.0
http_server_requests_seconds_bucket{exception="None",method="POST",outcome="CLIENT_ERROR",status="400",uri="/v2/prod/predict-rto-preorder-prod",le="3.22122547",} 19.0

We are catching all exceptions at Controller layer, maybe those are not propogated to micro-meter, Is there a way to fix this ?

Answer 1

You might be hitting this issue: https://github.com/spring-projects/spring-framework/issues/29848

If that's the case, you should handle the error on Spring side and signal the error there, e.g.:

@ControllerAdvice
public class CommonExceptionHandler extends ResponseEntityExceptionHandler {
    @ExceptionHandler(Throwable.class)
    ProblemDetail onThrowable(HttpServletRequest request, Throwable error) {
        ServerHttpObservationFilter.findObservationContext(request)
            .ifPresent(context -> context.setError(error));
        return ...;
    }
}

Here is a full example: CommonExceptionHandler. (You can also return the view as a String if that's your use-case.)

Also, the instrumentation of Spring MVC contains the outcome tag which is directly mapped to HTTP status codes (see HttpStatus):

INFORMATIONAL: 1xx
SUCCESSFUL:    2xx
REDIRECTION:   3xx
CLIENT_ERROR:  4xx
SERVER_ERROR:  5xx

So you can do this and see the different outcomes on different graphs:

sum(rate(http_server_requests_seconds_count[5m])) by (outcome)