I have a java (java-11) gRPC client streamer trying to talk with a golang gRPC server, with tls and envoy proxy in the middle.
And I'm getting
UNAVAILABLE: Failed ALPN negotiation: Unable to find compatible protocol
error from the ServerResponseObserver.
Full stacktrace:
12/05/22 18:29:29 (798) grpc-default-executor-1 SEVERE *** ERROR *** server respond with onError:
io.grpc.StatusRuntimeException: UNAVAILABLE: Failed ALPN negotiation: Unable to find compatible protocol
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
at io.grpc.Status.asRuntimeException(Status.java:535)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:487)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:468)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:432)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:465)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
The calls do not reach the gRPC server and seem like they won't pass the envoy proxy.
envoy trace logs show a few ssl errors such as "ssl error occurred while read: WANT_READ"
configuration of the java gRPC client streamer:
ChannelCredentials credentials;
credentials = TlsChannelCredentials.newBuilder()
.trustManager(new File("<path-to-file>/cacert.pem"))
.build();
originChannel = Grpc.newChannelBuilderForAddress(host, port, credentials).build();
ClientInterceptor interceptor = new HeaderClientInterceptor();
Channel channel = ClientInterceptors.intercept(originChannel, interceptor);
stub = LogStreamerGrpc.newStub(channel);
Apparently, when using a proxy one need to ensure the proxy is configured correctly with h2 enabled for ALPN
from envoy documantation alpn_protocols:
For me, supporting h2 was enough.
Here is the configuration I added to envoy.yaml running on go under in order to make it work: