System.Net.WebException: The remote server returned an error: (429) Too Many Requests

Question

I am working on API development project using ASP.NET Core 2.2, GraphQL.NET , CosmosDB, EntityFrameworkCore (Microsoft.EntityFrameworkCore.Cosmos(2.2.4).

On running the solution, I see an error in the output window of the VS2019:

System.Net.WebException: The remote server returned an error: (429) Too Many Requests.
   at System.Net.HttpWebRequest.GetResponse()
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClient.CreateDocumentCollectionIfNotExistsOnce(DbContext _, String collectionId)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request finished in 119926.3431ms 200 application/json
Microsoft.AspNetCore.Hosting.Internal.WebHost:Information: Request starting HTTP/1.1 POST http://localhost:53116/graphql application/json 1468
Microsoft.EntityFrameworkCore.Infrastructure:Information: Entity Framework Core 2.2.4-servicing-10062 initialized 'TaxathandDbContext' using provider 'Microsoft.EntityFrameworkCore.Cosmos' with options: ServiceEndPoint=https://taxathanddb.documents.azure.com/ Database=TaxathandDb 
Microsoft.EntityFrameworkCore.Infrastructure:Information: A transient exception has been encountered during execution and the operation will be retried after 9031ms.
System.Net.WebException: The remote server returned an error: (429) Too Many Requests.
   at System.Net.HttpWebRequest.GetResponse()
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClient.CreateDocumentCollectionIfNotExistsOnce(DbContext _, String collectionId)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)

Startup.cs

public void ConfigureServices(IServiceCollection services)
{
    string serviceEndPoint = Configuration.GetValue<string>("CosmosDBEndpoint");
    string authKeyOrResourceToken = Configuration.GetValue<string>("CosmosDBAccessKey");
    string databaseName = Configuration.GetValue<string>("CosmosDBName");
    services.AddEntityFrameworkCosmos();
    services.AddScoped<DbContext, SampleDbContext>();
    services.AddDbContext<TaxathandDbContext>(options => options.UseCosmos(serviceEndPoint, authKeyOrResourceToken, databaseName, contextOptions =>
    {
        contextOptions.ExecutionStrategy(d => new CosmosExecutionStrategy(d));
    }

    ));
    services.Configure<AppSettings>(Configuration.GetSection("AppSettings"));
    services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();
    services.AddSingleton<IDocumentExecuter, DocumentExecuter>();
    services.AddSingleton<IDataLoaderContextAccessor, DataLoaderContextAccessor>();
    services.AddSingleton<DataLoaderDocumentListener>();
    services.AddSingleton<IDocumentWriter, DocumentWriter>();
    services.AddScoped<IUtilityService, UtilityService>();
    services.AddScoped<ICommonService, CommonService>();
    services.AddScoped<ICountryService, CountryService>();
    services.AddScoped<CountryResultType>();
    services.AddScoped<GraphQLQuery>();
    services.AddScoped<ICountriesResolver, CountriesResolver>();
    services.AddScoped<CountryType>();
    services.AddScoped<Response>();
    services.AddScoped(typeof(ResponseGraphType<>));
    services.AddScoped(typeof(ResponseListGraphType<>));
    services.AddTransient<IAddressRepository, AddressRepository>();
    services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Latest);
    services.AddGraphQL(o =>
    {
        o.ExposeExceptions = true;
    }

    ).AddGraphTypes(ServiceLifetime.Scoped);

    services.AddScoped<IDependencyResolver>(s => new FuncDependencyResolver(s.GetRequiredService));
    services.AddScoped<SampleSchema>();
}

Can anyone help me to fix this issue?

Answer 1

I would like to know more information about context file as the error says Too Many Requests. So Considering you have already deployed database in cosmosdb it is recommended to remove Database.EnsureCreated() as it will try to check the database and create models on every request which is not recommended and will create performance issues.

Refer to this documentation for more information https://learn.microsoft.com/en-us/ef/core/providers/cosmos/?tabs=dotnet-core-cli

Answer 2

Considering you have no middleware in place to dynamically restrict traffic, and considering the stack trace refers to Cosmos quite a bit, I wager it is Cosmos that is causing the 429 error code.

Cosmos DB actually has a setting where you can specify the throughput, and it defaults to 1000 RU/second (Request Units per second). Chances are you are exceeding this default value and Azure is telling you to 'stop' by returning a 429 error code.

Refer to this documentation for more information about this setting and how you can configure it https://learn.microsoft.com/en-us/azure/cosmos-db/request-units