Linq Count() timing out -Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding

Question

I have a linq query which tries to fetch approximately 500K records from DB. I have a Count() which eventually timing out.

I want to know if my linq query contains 5000 or more records or not. I don't count of all records, just need to check if linq contains 5000 records.

Is there any effective way to check if there 5000 or more records in linq without calling Count()? I am using EF core 3.1.

Linq Query :

  var results = (from a in RepoContext.Employee
                          join b in RepoContext.Program on a.ProgramId equals b.ProgramId 
                          where a.ActiveFlag == true
                                && b.ClientId == 2
                          select new RAManufacturerDto
                          {

                              BusinessName = a.BusinessName,
                              ClientId = a.ClientId.Value,
                              ClientName = b.ClientName
                              DCode = b.DCode,
                              StoreId = b.StoreId,
                              ProgramId = a.ProgramId
                          });

bool isRecordsLimitReached = results.Count() > 5000;

I am getting an error when trying to do Count() on result. I just want to get if it contains more than 5000 records.

Answer 1

Use linq directly instead, suppose you have your dbcontext properly configured and such relation between the two defined in onmodelcreating, move the code into the RepoContext. ( if in doubt to define relations check this out: https://learn.microsoft.com/en-us/ef/core/modeling/relationships?tabs=fluent-api%2Cfluent-api-simple-key%2Csimple-key )

    DbSet<Employee> Employees {get;set;}
    DbSet<Program> Programs {get;set;}

    public bool HasMoreThan(int number) {
      var count = RepoContext.Employee.Include(x => x.Program).Count(y => 
      y.ActiveFlag == true && y.Program.ClientId == 2);
      return count >= number;
    }

Answer 2

Just try to skip 5000 records and if there are records - we have reached our goal:

bool isRecordsLimitReached = results.Skip(5000).Any();

Answer 3

If you only need to get the count, you don't need to select all of those properties. You can optimize this query by selecting only one of the non nullable properties.

var results = (from a in RepoContext.Employee
               join b in RepoContext.Program on a.ProgramId equals b.ProgramId 
               where a.ActiveFlag == true && 
               b.ClientId == 2
               select a.Id); //Id = Primary Key of the Employee database

bool isRecordsLimitReached = results.Count() > 5000;

If you still get a timeout with this then you may need to add an index to the foreign key on the Program table if it is not already there. An extra index for ActiveFlag and ClientId would not hurt either.