I have a very basic C# application that uses no fancy dependency (only Microsoft.*) and connects to an HTTPS endpoint (namely a French public information: https://registre-national-entreprises.inpi.fr/api). This is packaged in a docker image:
FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-env
WORKDIR /app
# prepare rocksdbsharp
RUN apt-get update && apt-get install -y --no-install-recommends \
libc6-dev \
libsnappy1v5 \
zlib1g \
libbz2-1.0 \
libgflags2.2 \
liblz4-1 \
libzstd1 \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y bzip2 lz4 librocksdb-dev
COPY . .
WORKDIR /app/*redacted*
RUN dotnet restore ./*redacted*.csproj
RUN dotnet publish ./*redacted*.csproj -c Release -o /out
# runtime image
FROM mcr.microsoft.com/dotnet/aspnet:8.0
COPY --from=build-env /out .
COPY --from=build-env /usr/lib /usr/lib
COPY --from=build-env /usr/local/lib/ /usr/local/lib/
COPY --from=build-env /usr/local/bin/ /usr/local/bin/
ENV LD_LIBRARY_PATH="/usr/lib:/usr/local/bin:/usr/local/lib:${LD_LIBRARY_PATH}"
ENTRYPOINT ["dotnet", "*redacted*.dll"]
When I run this locally it works as a charm, both by running directly the .NET application and the docker. When I deploy it on Google Cloud Run Job though I get this error:
2024-03-21 14:31:10.716 CET
---> System.IO.IOException: The decryption operation failed, see inner exception.
2024-03-21 14:31:10.716 CET
---> Interop+OpenSsl+SslException: Decrypt failed with OpenSSL error - SSL_ERROR_SSL.
2024-03-21 14:31:10.716 CET
---> Interop+Crypto+OpenSslCryptographicException: error:0A000119:SSL routines::decryption failed or bad record mac
2024-03-21 14:31:10.716 CET
--- End of inner exception stack trace ---
2024-03-21 14:31:10.716 CET
at Interop.OpenSsl.Decrypt(SafeSslHandle context, Span`1 buffer, SslErrorCode& errorCode)
2024-03-21 14:31:10.716 CET
at System.Net.Security.SslStreamPal.DecryptMessage(SafeDeleteSslContext securityContext, Span`1 buffer, Int32& offset, Int32& count)
2024-03-21 14:31:10.716 CET
--- End of inner exception stack trace ---
2024-03-21 14:31:10.716 CET
at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
2024-03-21 14:31:10.716 CET
at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
2024-03-21 14:31:10.716 CET
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
2024-03-21 14:31:10.716 CET
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.CopyToAsyncCore(Stream destination, CancellationToken cancellationToken)
2024-03-21 14:31:10.716 CET
at System.Net.Http.HttpConnectionResponseContent.<SerializeToStreamAsync>g__Impl|6_0(Stream stream, CancellationToken cancellationToken)
2024-03-21 14:31:10.716 CET
at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
I have to admit I'm no expert in SSL at all... I've read several recommendations like:
ethtool -K eth0 tso off gro off gso off ufo off
but this is not accessible in a dockerfile I guess.
sed -i 's/MinProtocol = TLSv1.3/MinProtocol = TLSv1/' /etc/ssl/openssl.cnf
&& sed -i 's/CipherString = DEFAULT@SECLEVEL=2/CipherString = DEFAULT@SECLEVEL=1/' /etc/ssl/openssl.cnf
did not solve my issue either (and anyway I guess this would be for downgrading TLS version mainly but here the endpoint server supports TLS1.3).
EDIT
I suspect this happens for queries that have large response bodies (>200KB). This happens regarless of whether I use a VPC or not, but I read somewhere this could happen with MTU parameters... Though I don't know if I can configure MTU when there's no network / subnet involved... (don't event know if that's relevant at all).
I also wonder if it could be related to some sort of quotas of Google Cloud Run (I've read the section about limit size in https://cloud.google.com/run/quotas but it seems to be about the container itself and not about the requests/responses the container makes).