Spring's AsyncRestTemplate not working with zipped content, Getting compressed content instead of json object

Question

I have enabled content compression on my spring boot app and response of every API is getting compressed but if I hit those APIS using RestTemplate I get content like

"\u001F�\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000�}��8��¨OtD���1��]�m�mo��v�_LlP\u0014J�4E��(�����C�:\u0012<D\u0010����\b۲��\u0004\u0012@\"

However if I use HttpClientBuilder.create().build() to create my RestTemplate object and add headers.set("Content-Encoding", "gzip"); headers.set("Accept-Encoding", "gzip"); while setting headers to my exchange request, I receive proper json object in a string format because API itself returning a String.

But I am facing the same problem while accessing those APIs using AsyncRestTemplate, even If I create AsyncRestTemplate object using my already present RestTemplate object.

One way I have is to implement thread pool on my own and then hit the request using RestTemplate, but I want to know is there any way of getting json object instead of compressed content without me implementing threading mechanism.

Below is the complete code

public class RestUtil {

    @Resource RestTemplate restTemplate;
    @Resource AsyncRestTemplate asyncRestTemplate;

    private final Log log = LogFactory.getLog(getClass());

    public String getSyncResponse(HttpServletRequest request) {
        final String URL = "not including url";

        UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(URL);

        HttpEntity<String> entity = new HttpEntity<>(null, getHeaders(request));
        try {
            ResponseEntity<String> response = restTemplate.exchange(builder.toUriString(), GET, entity, String.class);
            System.out.println("==== sync response ====");
            System.out.println(response.getBody());
            System.out.println("========");
            return response.getBody();
        } catch (Exception ex) {
            log.error("Error in connecting to codebook server ", ex);
        }
        return null;
    }

    public List<String> getAsyncResponse(HttpServletRequest request) {

        List<String> urls = new ArrayList<>();
        urls.add("not including url");
        urls.add("not including url");

        HttpEntity<String> entity = new HttpEntity<>(null, getHeaders(request));
        try {

            ArrayList<Future<ResponseEntity<String>>> futures = new ArrayList<>();

            for (String url : urls) {
                ListenableFuture<ResponseEntity<String>> listenableFuture = asyncRestTemplate.exchange(url, GET, entity, String.class);
                listenableFuture.addCallback(response -> log.info("Success"), ex -> log.error("Request failed", ex));
                futures.add(listenableFuture);
            }

            List<String> responses = new ArrayList<>();
            for (Future<ResponseEntity<String>> future : futures) {
                responses.add(future.get().getBody());
            }

            System.out.println("==== async response ====");
            System.out.println(responses);
            System.out.println("========");

            return responses;

        } catch (Exception ex) {
            log.error("Error in connecting to server ", ex);
        }
        return null;
    }

    private HttpHeaders getHeaders(HttpServletRequest request) {
        HttpHeaders headers = new HttpHeaders();
        headers.set("Accept", APPLICATION_JSON_UTF8_VALUE);
        headers.set("Content-Type", APPLICATION_JSON_UTF8_VALUE);
        headers.set("Content-Encoding", "gzip");
        headers.set("Accept-Encoding", "gzip");
        Enumeration<String> headerNames = request.getHeaderNames();
        while (headerNames.hasMoreElements()) {
            String header = headerNames.nextElement();
            Enumeration<String> headerList = request.getHeaders(header);
            headers.put(header, Collections.list(headerList));
        }

        return headers;
    }
}


@Configuration
class HttpConfig {

    @Bean
    public RestTemplate restTemplate() {
        RestTemplate restTemplate = new RestTemplate(httpRequestFactory());

        MappingJackson2HttpMessageConverter jsonHttpMessageConverter = new MappingJackson2HttpMessageConverter();
        jsonHttpMessageConverter.getObjectMapper().configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);
        restTemplate.getMessageConverters().add(jsonHttpMessageConverter);

        return restTemplate;
    }

    @Bean
    public ClientHttpRequestFactory httpRequestFactory() {
        return new HttpComponentsClientHttpRequestFactory(httpClient());
    }

    @Bean
    public CloseableHttpClient httpClient() {
        return HttpClientBuilder.create().build();
    }

    @Bean
    public AsyncRestTemplate asyncRestTemplate(AsyncClientHttpRequestFactory asyncHttpRequestFactory, RestTemplate restTemplate) {
        return new AsyncRestTemplate(asyncHttpRequestFactory, restTemplate);
    }

    @Bean
    public AsyncClientHttpRequestFactory asyncHttpRequestFactory() {
        SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
        requestFactory.setTaskExecutor(new SimpleAsyncTaskExecutor());
        return requestFactory;
    }

}

Answer 1

I would like to recommend you don't do compressions manually, rather just tell Spring do this for you (actually compressing will be done by web server (e.g. Tomcat), but Spring will do this for you). Usually in Spring such things like compressions, encoding can be turned on just by single property

Please look at http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#how-to-enable-http-response-compression

Answer 2

A simple workaround is adding a ResponseInterceptor with AsyncRestTemplate, that automatically decompresses the response. The ListenableFuture that you receive back will have the decompressed response already. Here's an example:

AsyncRestTemplate getAsyncRestTemplate() {
    final HttpComponentsAsyncClientHttpRequestFactory httpComponentsAsyncClientHttpRequestFactory =
            new HttpComponentsAsyncClientHttpRequestFactory();

    final AsyncRestTemplate asyncRestTemplate = new AsyncRestTemplate(httpComponentsAsyncClientHttpRequestFactory);
    asyncRestTemplate.setInterceptors(Collections.singletonList((httpRequest, bytes, asyncClientHttpRequestExecution) -> {
        if (!httpRequest.getHeaders().containsKey(HttpHeaders.ACCEPT_ENCODING)) {
            httpRequest.getHeaders().set(HttpHeaders.ACCEPT_ENCODING, "gzip");
        }
        final ListenableFuture<ClientHttpResponse> future = asyncClientHttpRequestExecution.executeAsync(httpRequest, bytes);
        return new ListenableFutureAdapter<ClientHttpResponse, ClientHttpResponse>(future) {
            @Override
            protected ClientHttpResponse adapt(ClientHttpResponse clientHttpResponse) throws ExecutionException {
                return new InflatedClientHttpResponse(clientHttpResponse);
            }
        };
    }));
    return asyncRestTemplate;

}

public class InflatedClientHttpResponse implements ClientHttpResponse {
    private final ClientHttpResponse clientHttpResponse;
    private final Boolean isCompressed;

    public InflatedClientHttpResponse(ClientHttpResponse clientHttpResponse) {
        this.clientHttpResponse = clientHttpResponse;
        final HttpHeaders httpHeaders = clientHttpResponse.getHeaders();

        final List<String> contentEncoding = httpHeaders.get(HttpHeaders.CONTENT_ENCODING);
        if (contentEncoding != null && contentEncoding.contains("gzip")) {
            isCompressed = true;
        } else {
            isCompressed = false;
        }

        httpHeaders.remove(HttpHeaders.CONTENT_ENCODING);
        httpHeaders.remove(HttpHeaders.CONTENT_LENGTH);
    }

    @Override
    public HttpStatus getStatusCode() throws IOException {
        return clientHttpResponse.getStatusCode();
    }

    @Override
    public int getRawStatusCode() throws IOException {
        return clientHttpResponse.getRawStatusCode();
    }

    @Override
    public String getStatusText() throws IOException {
        return clientHttpResponse.getStatusText();
    }

    @Override
    public void close() {
        clientHttpResponse.close();
    }

    @Override
    public InputStream getBody() throws IOException {
        if (isCompressed) {
            return new GZIPInputStream(clientHttpResponse.getBody());
        }
        return clientHttpResponse.getBody();
    }

    @Override
    public HttpHeaders getHeaders() {
        return clientHttpResponse.getHeaders();
    }
}