Azure.Search.Documents - Update or insert object with collections

Question

I need to either add new document with children or add child to already existing parent. The only way I know how to do it is ugly:

public async Task AddOrUpdateCase(params)
{
    try
    {
        await UpdateCase(params);
    }
    catch (RequestFailedException ex)
    {
        if (ex.Status != (int)HttpStatusCode.NotFound)
           throw;

       await AddCase(params);
    }
}

private async Task UpdateCase(params)
{
    // this line throws when document is not found
    var caseResponse = await searchClient.GetDocumentAsync<Case>(params.CaseId) 
    // need to add to existing collection
    caseResponse.Value.Children.Add(params.child);
}

I think there wouldn't be any problem if this document didn't contain collection. You cannot use MergeOrUpload if there are child collections. You need to load them from index and add element. Is there better way to do it?

Answer 1

Azure Cognitive Search doesn't support partial updates to collection fields, so retrieving the entire document, modifying the relevant collection field, and sending the document back to the index is the only way to accomplish this.

The only improvement I would suggest to the code you've shown is to search for the documents you want to update instead of retrieving them one-by-one. That way, you can update them in batches. Index updates are much more expensive than queries, so to reduce overhead you should batch updates together wherever possible.

Note that if you have all the data needed to re-construct the entire document at indexing time, you can skip the step of retrieving the document first, which would be a big improvement. Azure Cognitive Search doesn't yet support concurrency control for updating documents in the index, so you're better off having a single process writing to the index anyway. This should hopefully eliminate the need to read the documents before updating and writing them back. This is assuming you're not using the search index as your primary store, which you really should avoid.

If you need to add or update items in complex collections often, it's probably a sign that you need a different data model for your index. Complex collections have limitations (see "Maximum elements across all complex collections per document") that make them impractical for scenarios where the cardinality of the parent-to-child relationship is high. For situations like this, it's better to have a secondary index that includes the "child" entities as top-level documents instead of elements of a complex collection. That has benefits for incremental updates, but also for storage utilization and some types of queries.