Ktor response negotiation error in case of unexpected response

Question

Using Ktor's ContentNegotiation is very convenient and easy way, how to de-/serialize text into one's preferred format.

In our case it is JSON.

All the guides on the internet (server or client) will tell you how to set it up and how to de-/serialize the contents using it.

But it was nowhere said how to deal with cases when the server responds with unexpected payload, e.g. in case of an error.

Say we are doing a POST https://apis.google.com/some-service with a payload. It returns a response that may look like this:

{
    "data": {
        "response": [{
            "foo": "bar",
            "gee": "zuz"
        }, ...
    }
}

In your app, you therefore have your data classes defined using @Serializable annotation and the ContentNegotation plugin is installed in your HttpClient and all is working fine unless for example Google service's database is down and the service responds with an error response, that may look like this:

{
    "error": {
        "code": 500,
        "message": "The DB is down, retry after 30 seconds.",
        ...
    }
}

Now your serialization doesn't work anymore and instead your code throws an exception (io.ktor.serialization.JsonConvertException to be precise).

How to deal with these cases?

Answer 1

What nobody says or shares is how to be prepared for cases like this.

Status pages

Here is one approach I came across that can be extended for our case. Following a Ktor documentation and applying all the necessary plugins, one may have already installed and configured the StatusPages for their server. Knowing the serialization failure throws io.ktor.serialization.JsonConvertException, we can add a status page exactly for it, following is an example code:

install(StatusPages) {
    exception<Throwable> { call, cause ->

        val log = call.application.environment.log

        when (cause) {
            // ...

            is JsonConvertException -> {
                log.error(cause.message)
                call.respond(
                    message = "Error serializing response from JSON; original error: ${cause.message}",
                    status = HttpStatusCode.InternalServerError,
                )
            }

            is Exception -> {
                // ...
            }
        }

Custom Exception handling

However, this only logs and responds back the original conversion error, which says approximately this:

ERROR Application - Illegal input: Field 'data' is required for type with serial name 'here.your.SerializableClass', but it was missing at path: $

In order to know what the server actually responded, no matter what format the response body has, we have to do a little change to our code calling the Google service.

Instead of the previous call, which may be:

private suspend fun postToGoogleService(data: String) : SerializableClass {
    // setup your HttpClient

    val response = client.post("$apiUrl?key=$apiKey") {
        contentType(ContentType.Application.Json)
        setBody(data)
    }

    return response.body()
}

we add a try-catch block, something like this:

private suspend fun postToGoogleService(data: String) : SerializableClass {
    // setup your HttpClient

    val response = client.post("$apiUrl?key=$apiKey") {
        contentType(ContentType.Application.Json)
        setBody(data)
    }

    val responseClass : SerializableClass
    try {
        responseClass = response.body()
    } catch (e : JsonConvertException) {
        throw Exception(
            "Failed serializing Class from Google API; the response was: ${response.bodyAsText()}",
            e.cause,
        )
    }
    return responseClass
}

This way we take the response body as a string and put it into the Exception message. If you prefer not to pollute your exceptions with potentially weird text, you may only log.Error(response.bodyAsText()) and throw the original Exception(e.message, e.cause).

This way, I was able to identify what is wrong with my request, and especially, to identify that the cause for the error response was in my call.