HttpClient.GetAsync Throws 413 Request Entity Too Large

Question

I have a .NET 8 C# Azure Functions app. It calls a third party vendor API, which returns a very large JSON. Most of the times the following code works, but only sometimes it throws 413 Request Entity Too Large error. When it throws an error in my code, the same request works perfectly fine in Postman.

public async Task<MyLargeObject> GetMyLargeObjectAsync(string unique_id)
{
    MyLargeObject myLargeObject = new MyLargeObject();

    try
    {
        HttpClient client = new HttpClient()
        client.DefaultRequestHeaders.Clear();
        client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("*/*"));
        client.DefaultRequestHeaders.UserAgent.ParseAdd("MyApp/1.0.0");
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _token);

        var response = await client.GetAsync(_vendorEndPointURL + unique_id);

        response.EnsureSuccessStatusCode(); // This line throws exception when response JSON is very large
        if (response.Content != null)
        {
            HttpContent content = response.Content;
            string result = content.ReadAsStringAsync().Result;
            myLargeObject = JsonConvert.DeserializeObject<MyLargeObject>(result);

        }
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, $"Error getting MyLargeObject from Vendor API. Unique ID: {unique_id}");
        throw;

    }
    return myLargeObject;
}

I even added the following in Program.cs, but still no joy:

services.Configure<KestrelServerOptions>(options =>
{
    options.Limits.MaxRequestBodySize = long.MaxValue;
    options.Limits.MaxResponseBufferSize = long.MaxValue;
}); 
services.Configure<IISServerOptions>(options =>
{
    options.MaxRequestBodySize = long.MaxValue; 
});

Answer 1

Instantiating HTTPClient like this solved the problem:

                HttpClientHandler handler = new HttpClientHandler() { AutomaticDecompression = System.Net.DecompressionMethods.GZip | System.Net.DecompressionMethods.Deflate | System.Net.DecompressionMethods.Brotli };
                HttpClient client = new HttpClient(handler);