Deleting older azure table storage log entries created by semantic logging

Question

I have implemented semantic logging to log entries into the azure table storage. Now I want to remove old log entries(say older than 30 days) from the azure table. The problem is that the partitionKeys and RowKeys in the azure table are dynamically generated. Something like this

PartitionKey         RowKey

2519569987199999999  MyProjectName_2519569788418593594_FFFFFFF8 //entry date 20 october 2015
2519573205599999999  MyProjectName_2519573006928838932_FFFFFFE1 // entry date 16 october 2015

I researched and found out that the PartitionKey is of the format DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks

How can I fetch data from the table for deletion? Would I need to do a full table scan since the partitionKey is dynamic?

Answer 1

How can I fetch data from the table for deletion? Would I need to do a full table scan since the partitionKey is dynamic?

No, you don't need to do a full table scan. Here's how you can find out the entities that are created before 30 days.

        var today = DateTime.UtcNow.Date;
        var thirtyDaysFromToday = today.AddDays(-30);
        var pk = (DateTime.MaxValue.Ticks - thirtyDaysFromToday.Ticks).ToString();
        var account = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
        var tableClient = account.CreateCloudTableClient();
        var table = tableClient.GetTableReference("YourTableName");
        TableQuery query = new TableQuery().Where("PartitionKey ge '" + pk + "'").Select(new List<string>(){"PartitionKey", "RowKey"});//Since you only need PartitionKey/RowKey for entities deletion thus just fetch that only
        TableContinuationToken token = null;
        do
        {
            var queryResult = table.ExecuteQuerySegmented(query, token);
            token = queryResult.ContinuationToken;
            var entities = queryResult.Results;
            //Now you can delete those entities
        }
        while (token != null);

Note: I haven't tested the code so please try with a smaller set of data first before going all out :)