Azure.Data.Tables QueryAsync filtering erring with multiple filter conditions

1.1k Views Asked by At

In the old SDK Azure Table you could create FilterConditions like the following

var partitionFilter = TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, symbolRoot);
    var startDateFilter =
        TableQuery.GenerateFilterConditionForDate("TradeDate", QueryComparisons.GreaterThanOrEqual, startDate);
    var endDateFilter =
        TableQuery.GenerateFilterConditionForDate("TradeDate", QueryComparisons.LessThanOrEqual, endDate);

    var filter = TableQuery.CombineFilters(
        TableQuery.CombineFilters(
            partitionFilter,
            TableOperators.And,
            startDateFilter),
        TableOperators.And, endDateFilter);

I'd love to be able to continue with that pattern but....

In the new Azure.Data.Tables SDK a lot has changed I see no methods to create filters with multiple parameters, even though the QueryAsync methods accepts a filter as a parameter. The reference material for it, found here shows how to create a single parameter filter, but not a multiple parameter filter.

This method below fails because the DateTime conversion apparently is not supported within the filtering

public async Task<List<EquityDataEntity>> GetEquityPriceDataXDaysBackAsync(string symbol, int daysBackFromToday)
{
    if (daysBackFromToday > 0)
    {
        daysBackFromToday *= -1; // put it proper form for Add... method
    }
    var data = await TableClient!
        .QueryAsync<EquityDataEntity>(u => u.PartitionKey == symbol.ToUpper()
                                  && Convert.ToDateTime(u.RowKey) >= DateTime.Now.AddDays(daysBackFromToday))  
        //Yes, unfortunately the RowKey is the date
        .ToListAsync();

    return data;
}

"Method ToDateTime not supported."

My two questions are:

  1. Is there a way to generate filters outside the method to use within the method as was done in prior SDK?

  2. How can I accomplish the date comparison in the Query method if the above is not doable and which requires a filter with multiple parameters (PartitionKey and date comparison)?

Update:

To note, the table structure is as follows The PartitionKey is a symbol like GLD, TSLA etc The RowKey is a Date like 2023-03-03

I have tried generating the filter like so

var filter = TableClient.CreateQueryFilter($"PartitionKey eq {symbol.ToUpper()} and RowKey gt {dateDataStarts}");

which produces this

PartitionKey eq 'GLD' and RowKey gt datetime'2021-10-20T17:42:01.3095704Z'

which according to the docs is what the OData filter should look like but it fails.

doing it with LINQ like so

var data = await TableClient!
        .QueryAsync<EquityDataEntity>(u => u.PartitionKey == symbol.ToUpper() && u.RowKey >= DateTime.Now.AddDays(daysBackFromToday).ToString())
        .ToListAsync();
    return data;

creates a VS2022 error

Operator 'operator' cannot be applied to operands of type 'string' and 'string'

replacing >= with gt doesn't work either

Update 2

The filter works now. The issue was the date was in the wrong format. This code now works as expected.

 var dateDataStarts = DateTime.Now.AddDays(daysBackFromToday).ToString("yyyy-MM-dd");
   
 var filter = TableClient.CreateQueryFilter($"PartitionKey eq {symbol.ToUpper()} and RowKey gt {dateDataStarts}");

which generates the OData filter as "PartitionKey eq 'GLD' and RowKey gt '2021-10-20'"

1

There are 1 best solutions below

4
Gaurav Mantri On BEST ANSWER

For the 1st one where you are specifying OData filter, you would need to convert your dateDataStarts variable to string. So your code would be something like:

var filter =
    TableClient.CreateQueryFilter($"PartitionKey eq {symbol.ToUpper()} and RowKey gt {dateDataStarts.ToString("yyyy-MM-dd")}");

For the 2nd one where you are using LINQ, you are correct. You cannot use >= for string comparison. You will need to use String.CompareTo. So your code would be something like:

TableClient.QueryAsync<TableEntity>(u => u.PartitionKey == symbol && u.RowKey.CompareTo(dateDataStarts.ToString("yyyy-MM-dd")) >= 0)