ElasticSearch is unable to recognize Context-Type header with encoding defined

Question

I've spent some time trying to fix the elastic search bulk upload warning:

Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header

My request is below:

POST http://elasticserver/_bulk HTTP/1.1
Authorization: xxx
Content-Type: application/x-ndjson; charset=utf-8
Host: elasticserver
Content-Length: 8559

... new line delimited json content ...

And my valid response with 200 status is below:

HTTP/1.1 200 OK
Warning: 299 Elasticsearch-5.5.1-19c13d0 "Content type detection for rest requests is deprecated. Specify the content type using the [Content-Type] header." "Mon, 14 Aug 2017 00:46:21 GMT"
content-type: application/json; charset=UTF-8
content-length: 4183

{"took":5538,"errors":false,...}

By experimenting I discovered that the issue is in content type charset definition Content-Type: application/x-ndjson; charset=utf-8 and if I change it to Content-Type: application/x-ndjson I get no warning.

Is it an elastic search issue or I'm forming the request incorrectly?

Answer 1

The official documentation explicitly states that

When sending requests to this endpoint the Content-Type header should be set to application/x-ndjson.

The RestController source code also shows that they are ignoring the charset:

final String lowercaseMediaType = restRequest.header("Content-Type").toLowerCase(Locale.ROOT);
// we also support newline delimited JSON: http://specs.okfnlabs.org/ndjson/
if (lowercaseMediaType.equals("application/x-ndjson")) {
    restRequest.setXContentType(XContentType.JSON);
    return true;
}