EWS ExtendedPropertyDefinition search filter no longer filtering

Question

For literally over a decade we have had C# code that talks to Exchange via EWS. Virtually all our clients have migrated to Office 365, and the our code was operating perfectly fine on O365 via EWS, until recently.

Our software processes emails that are received into an Exchange mailbox. This mailbox is used by humans as well as our software, so we've been using an ExtendedProperty to keep track of which emails we have processed. The code is thus:

// Build our extended property definition
Guid readByEmailComponentGUID = new Guid("{FCF74ABF-684E-4cc7-8A83-B36D3A209219}"); //GUID for our property
ExtendedPropertyDefinition X_READBYCOMPONENT = new ExtendedPropertyDefinition(readByEmailComponentGUID, "X_READBYCOMPONENT", MapiPropertyType.Boolean);   //Settings and name for our property

// Create a search filter for querying EWS
SearchFilter.SearchFilterCollection srchFilter = new SearchFilter.SearchFilterCollection(LogicalOperator.Or);
srchFilter.Add(new SearchFilter.Not(new SearchFilter.Exists(X_READBYCOMPONENT)));  //Show messages that do not have our new property (thus have never been received)
srchFilter.Add(new SearchFilter.IsEqualTo(X_READBYCOMPONENT, false));              //Show messages that have our new property, but it's set to false (to re-receive)

ItemView view = new ItemView(250, 0, OffsetBasePoint.Beginning);

// Query EWS with our search filter and our view
FindItemsResults<Item> findResults = EWSService.FindItems("Inbox", srchFilter, view);

// Process the results of our search
foreach (Item item in findResults){
  // Check to see if this email has our internal read flag on it, and if it does, do not process it again.
  ExtendedProperty ReadByComponentFlag = item.ExtendedProperties.FirstOrDefault(itm => itm.PropertyDefinition == X_READBYCOMPONENT);
  if (!(ReadByComponentFlag is null) && (bool)ReadByComponentFlag.Value == true)
  {
    continue;
  }

  if (item is EmailMessage){
    EmailMessage em = item as EmailMessage;     //Load the email message

    // process the mail message and when that's done mark it as read

    em.SetExtendedProperty(X_READBYCOMPONENT, true);   //Sets our new property to TRUE to exclude this email from future receive actions
    em.Update(ConflictResolutionMode.NeverOverwrite);

  }
}

Since 2011 that code has been working brilliantly, but starting around a month ago our customers on Office 365 have had their search filters failing, and EWS has been returning every single email message. Inspecting the email messages as they come through and manually looking at their extended properties I can see that our custom property IS present, and that it IS still set to true.

Has anything changed in how EWS on O365 processes these filters? Has anyone else run into similar issues?

Right now we've added a workaround (which you can see in the code above) to manually inspect the extended properties and skip the email but this is not a viable long-term solution as some of these mailboxes have 14,000 emails in them that we are pulling and receiving every run.

Answer 1

I tried to reproduce the issue with the same property and logic and can't eg

        var InboxFolder = Folder.Bind(exchangeService, WellKnownFolderName.Inbox);
        // Create a search filter for querying EWS
        // Build our extended property definition
        Guid readByEmailComponentGUID = new Guid("{FCF74ABF-684E-4cc7-8A83-B36D3A209219}"); //GUID for our property
        ExtendedPropertyDefinition X_READBYCOMPONENT = new ExtendedPropertyDefinition(readByEmailComponentGUID, "X_READBYCOMPONENT", MapiPropertyType.Boolean);   //Settings and name for our property

        Item it = InboxFolder.FindItems(new ItemView(1)).Items[0];
        it.SetExtendedProperty(X_READBYCOMPONENT, true);
        it.Update(ConflictResolutionMode.AutoResolve);
        SearchFilter.SearchFilterCollection srchFilter = new SearchFilter.SearchFilterCollection(LogicalOperator.Or);
        srchFilter.Add(new SearchFilter.Not(new SearchFilter.Exists(X_READBYCOMPONENT)));  //Show messages that do not have our new property (thus have never been received)
        srchFilter.Add(new SearchFilter.IsEqualTo(X_READBYCOMPONENT, false));
        FindItemsResults<Item> findResults = InboxFolder.FindItems(srchFilter, new ItemView(100));
        if(findResults.Items.Any(x => x.Id.UniqueId == it.Id.UniqueId))
        {
            Console.WriteLine("Item Found");
        }
        else
        {
            Console.WriteLine("Item not found");
        }

        Console.WriteLine("Done");

If it's happening on multiple mailboxes in multiple environment and you can build a test rig and reproduce the issue then you should log it as a bug with Microsoft as that is the only way you will get some type of traction/repair. Search Filter shouldn't fail open, you may also want to try it out on the Graph to see if the same problem exists eg on the Graph that query would be something like

https://graph.microsoft.com/v1.0/me/mailFolders('Inbox')/messages? $filter=singleValueExtendedProperties/any(ep: ep/id eq 'Boolean {FCF74ABF-684E-4cc7-8A83-B36D3A209219} Name X_READBYCOMPONENT' and cast(ep/value, Edm.Boolean) eq null) OR singleValueExtendedProperties/any(ep: ep/id eq 'Boolean {FCF74ABF-684E-4cc7-8A83-B36D3A209219} Name X_READBYCOMPONENT' and cast(ep/value, Edm.Boolean) eq false)

You should be able to test it with the Graph Explorer, if you find the same issue in the Graph then you can create a bug against that as well.