Using google-http-client and google-http-client-apache-v2 behind a proxy produces NonRepeatableRequestException

Question

I'm using google-http-client and google-http-client-apache-v2 libraries to make a POST request behind a proxy.

// 1.- Setting ssl and proxy
HttpClientBuilder builder = HttpClientBuilder.create();
            
SSLContext sslContext = SslUtils.getTlsSslContext();
SslUtils.initSslContext(sslContext, GoogleUtils.getCertificateTrustStore(), SslUtils.getPkixTrustManagerFactory());
builder.setSSLSocketFactory(new SSLConnectionSocketFactory(sslContext));
            
builder.setProxy(new HttpHost(host, port));
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(host, port), new UsernamePasswordCredentials(user, pass));
builder.setDefaultCredentialsProvider(credentialsProvider);

// 2.- Build request
HttpTransport httpTransport = new ApacheHttpTransport(builder.build());
HttpRequestFactory factory = httpTransport.createRequestFactory(credential);

HttpContent httpContent = new ByteArrayContent("application/json", "{}")
HttpRequest request = factory.buildRequest("POST", new GenericUrl(url), httpContent);

// 3.- Execute request
HttpResponse httpResponse = request.execute();

That request produces a NonRepeatableRequestException:

org.apache.http.client.ClientProtocolException
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.13.jar!/:4.5.13]
    at com.google.api.client.http.apache.v2.ApacheHttpRequest.execute(ApacheHttpRequest.java:73) ~[google-http-client-apache-v2-1.39.2.jar!/:?]
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) ~[google-http-client-1.39.2.jar!/:1.39.2]
    at 
    ...
Caused by: org.apache.http.client.NonRepeatableRequestException: Cannot retry request with a non-repeatable request entity.
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:225) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.13.jar!/:4.5.13]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.13.jar!/:4.5.13]
    at com.google.api.client.http.apache.v2.ApacheHttpRequest.execute(ApacheHttpRequest.java:73) ~[google-http-client-apache-v2-1.39.2.jar!/:?]
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) ~[google-http-client-1.39.2.jar!/:1.39.2]

        

It seems like ApacheHttpRequest wraps ByteArrayContent that is repeatable (see JavaDoc) inside a ContentEntity that is non-repeatable.

Debuging execution inside google libraries, proxy is returning "407 Proxy Authentication Required", then it tries to repeat the request (guess that including the credentials) and that exception arises because ContentEntity used by google library is non-repeatable.

Is there any way to avoid handshake with proxy including credentials in first request to avoid reuse of the entity?

Is there any way to tell google libraries that uses a repeatable entity?

Tryed with follwing library versions:

• google-api-client-1.31.5
• google-http-client-jackson2-1.39.2
• google-oauth-client-1.31.5
• google-http-client-apache-v2-1.39.2
• google-http-client-1.39.2
• httpclient-4.5.13
• httpcore-4.4.14

Answer 1

Workaround I posted on github in case it helps someone:

As workaround what I'm trying is to wrap ApacheHttpTransport in CustomApacheHttpTransport, which delegate the result of methods to ApacheHttpTransport except for buildRequest method.

This buildRequest method in CustomApacheHttpTransport builds a custom request of type CustomApacheHttpRequest.

public final class CustomApacheHttpTransport extends HttpTransport {
    
    private ApacheHttpTransport apacheHttpTransport;
    
    public CustomApacheHttpTransport (HttpClient httpClient) {
        this.apacheHttpTransport = new ApacheHttpTransport(httpClient);
    }
    
    @Override
    protected LowLevelHttpRequest buildRequest (String method, String url) {
        HttpRequestBase requestBase;
        if (method.equals("DELETE")) {
            requestBase = new HttpDelete(url);
        } else if (method.equals("GET")) {
            requestBase = new HttpGet(url);
        } else if (method.equals("HEAD")) {
            requestBase = new HttpHead(url);
        } else if (method.equals("PATCH")) {
            requestBase = new HttpPatch(url);
        } else if (method.equals("POST")) {
            ..
        }
        return new CustomApacheHttpRequest(apacheHttpTransport.getHttpClient(), requestBase);
    }
}

This custom request is like ApacheHttpRequest except for when it is executed it creates a custom entity, CustomContentEntity, which will be repeatable depending on whether the request content supports retries.

final class CustomApacheHttpRequest extends LowLevelHttpRequest {
    
    private final HttpClient httpClient;
    private final HttpRequestBase request;
    private RequestConfig.Builder requestConfig;
    
    CustomApacheHttpRequest (HttpClient httpClient, HttpRequestBase request) {
        this.httpClient = httpClient;
        this.request = request;
        this.requestConfig = RequestConfig.custom().setRedirectsEnabled(false).setNormalizeUri(false).setStaleConnectionCheckEnabled(false);
    }
    
    ...
        
    @Override
    public LowLevelHttpResponse execute () throws IOException {
        if (this.getStreamingContent() != null) {
            Preconditions.checkState(request instanceof HttpEntityEnclosingRequest, "Apache HTTP client does not support %s requests with content.", request.getRequestLine().getMethod());
            
            CustomContentEntity entity = new CustomContentEntity(this.getContentLength(), this.getStreamingContent());
            entity.setContentEncoding(this.getContentEncoding());
            entity.setContentType(this.getContentType());
            if (this.getContentLength() == -1L) {
                entity.setChunked(true);
            }
            ((HttpEntityEnclosingRequest) request).setEntity(entity);
        }
        
        request.setConfig(requestConfig.build());
        return new CustomApacheHttpResponse(request, httpClient.execute(request));
    }
}

The key in CustomContentEntity is isRepeatable method wich do not returns always false as ContentEntity does.

final class CustomContentEntity extends AbstractHttpEntity {
    
    private final long contentLength;
    private final StreamingContent streamingContent;
    
    CustomContentEntity (long contentLength, StreamingContent streamingContent) {
        this.contentLength = contentLength;
        this.streamingContent = streamingContent;
    }
    
    @Override
    public boolean isRepeatable () {
        return ((HttpContent) streamingContent).retrySupported();
    }
    ...
}

Also I have to create CustomApacheHttpResponse as response for CustomApacheHttpRequest because ApacheHttpResponse is package-private (CustomApacheHttpResponse is exactly like ApacheHttpResponse).

Answer 2

It is correct that the library returns with the error saying "your request is not retryable." It is working as intended.

POST requests are fundamentally considered non-retryable, as they are most likely to have a server store data. For example, a server is recommended to return 201 (Created) as a response when the server successfully created one or more resources. Retrying a POST request may end up inserting, uploading, or posting data multiple times. This is why sometimes web browsers show the following prompt to avoid "a duplicate credit card transaction":

A potential retry logic for POST should be implemented at the user application level, not at the library level.

In your case, the cause of the error is that you are not authorized to use the proxy. Therefore, you need to authenticate with the proxy first before attempting to use it, and then send (or re-send) a POST request.

---

UPDATES for the questions asked later in the comment as well as in the GitHub issue.

Why is the library who tries to repeat the request? (failling on a POST request).

The question reads weird, so I'm not sure what you're asking. Anyways, the library is designed to intentionally not repeat a request for POST. For GET, it's a different story.

Why the library have the same behaviour (retrying the request) with a GET request? (but in this case sucessfully because GET request do not have entity and do not matters if it is repeatable or not).

GET is by its nature considered a repeatable request. See this doc for example to understand the nature of the difference of GET and POST.

GET requests are only used to request data (not modify)

POST is used to send data to a server to create/update a resource.

.	GET	POST
BACK button/Reload	Harmless	Data will be re-submitted (the browser should alert the user that the data are about to be re-submitted)

Why if I change the entity, as show in workaround, to make it repeatable, the POST request works successfully through the proxy for which you say I'm not authorized to use?

You programmed your app to repeat the request when it fails at the application level through the use of the Apache API. Nothing prevents you from whatever you want with the Apache library. And of course, if we change the Google library to do what you are trying to do, it is technically possible to make it work that way. However, what I am saying is that it is wrong for the library to do so. And lastly, auth is not really relevant; it's just one kind of many failures you may encounter. For POST, in almost all cases, it doesn't make sense to automatically re-send the request regardless of which kind of error you encounter.

If as you say I'm not authorized to use the proxy:

You are not authorized to use the server for the initial request. That's why you get 407 Proxy Authentication Required from the proxy server. A client most likely needs to check the returned Proxy-Authenticate value and take an appropriate action to figure out the credentials. What action it needs to take depends on the value of the header, as explained in the doc:

This status is sent with a Proxy-Authenticate header that contains information on how to authorize correctly.

The form of the credentials you provide may not be the final form the proxy may expect. Often, your initial credentials are used to obtain the final form of the credentials that the server wants. Later once you have obtained them, the client will have to provide these credentials in subsequent requests. In any case, the truth is that, the server did return 407, saying "I'm denying your request, because authentication is required."

---

UPDATE2

Apache HttpClient is retrying the request

Yes, of course. And you manually programmed your app to allow Apache HttpClient to re-send a request for POST (which may be a viable workaround for you but this shouldn't be generalized for other cases).

Now I see what you are missing and where you have a wrong idea. When interacting with a proxy (or a non-proxy) that requires auth, generally you (whether it is you or the Apache library) will have to make at least two requests. First, you try without sending any sensitive information (why would you disclose your information upfront to someone who cannot be trusted? Even if you trust them, you don't really know if they are going to need your info at all. Moreover, even so, you don't know how correctly you should present your sensitive info). That first request may (or may not) fail with an error like "407 Proxy Authentication Required" (people call this that the server is "challenging" you), and based on what kinds of challenges the server gives you, you will need to take the right action to prepare an auth header for the second request. And the Apache library does that for you.

despite I provide the credentials

What did you expect that calling .setDefaultCredentialsProvider() would do? It doesn't do what you are currently thinking. The Apache library does nothing about your password in the first request. As I said earlier, in the end, you need to provide the right form of credentials that the server wants after checking the value of Proxy-Authenticate, which tells you how you should correctly auth with the server. That is why generally you have to repeat a request. If all these sound alien to you, please take a moment to read this introductory doc to understand how this challenge-based HTTP auth framework works. (The doc makes a note that it will explain only with the "Basic" scheme for educational purposes, but note that there are other non-basic schemes.)