How to implement complex conditional bulk partial updates via JSON API?

Question

There are POST and PATCH methods in HTTP protocol for partial updates. On the PATCH method for JSON API there are RFC 5789, RFC 6902, RFC 7396.

But I have a question specific to large resources that need to be partially updated with rather complex conditions.

Let us say we have an API endpont "/members" which resembles a large collection with thousands of entries where each entry has dozens of fields. So clients partially read those values via the "GET /members?fields=f1,f2,f3" method with pagination and each client reads only a subset of fields essential to their current task at hand.

To simplify the question, let's assume any entry has one required "email" field for identification and several optional fields.

{
"email": "[email protected]",
"optional1": "x",
"optional2": "x",
"optional3": "x"
}

And we have two clients working in parallel to create and update new records in bulk. But each client wishes to overwrite only their subset of optional fields.

So, client A would send a request with A values for a bulk patch that would be allowed to overwrite the "optional1" field value only because that field is essential to their task. And that client would also specify a value for the "optional2" field for new entries. And that client did not specify the "optional3" field at all.

{
"email": "[email protected]",
"optional1": "A",
"optional2": "A"
}

While that record is new, it is written into the storage completely. Then, client B would send a request with B values for a bulk patch that would be allowed overwrite the "optional2" field value only because that field is essential to their task.

{
"email": "[email protected]",
"optional1": "B",
"optional2": "B",
"optional3": "B"
}

Since the record already exists, the "optional1" field of that record should remain with the same old value A because that field is not essential, the "optional2" field value should be overwritten with the new value B because that field is essential, and the "optional3" field value should receive value B because that field was just empty. The merged result of the actions should be as follows.

{
"email": "[email protected]",
"optional1": "A",
"optional2": "B",
"optional3": "B"
}

So, the behavior depends on the existence of the record. How would you guys implement that in JSON API reusable endpoint method for concurrent parallel partial updates of such collections with complex conditions that would allow clients of that API to overwrite only a subset of fields in a record if that record already existed and contained some values?

The RFC 6902 solution does not match, because their add and replace operations both actually replace a value.

add - If the target location specifies an object member that does exist, that member's value is replaced.

replace - The "replace" operation replaces the value at the target location with a new value. The operation object MUST contain a "value" member whose content specifies the replacement value.

You can see how that matches the Java Map.put() method.

If the map previously contained a mapping for the key, the old value is replaced by the specified value.

But the missing feature matches rather the Java Set.add() method.

Adds the specified element to this set if it is not already present (optional operation). More formally, adds the specified element e to this set if the set contains no element e2 such that (e==null ? e2==null : e.equals(e2)). If this set already contains the element, the call leaves the set unchanged and returns false.

How would you make a JSON API that would allow both types of operations for a bulk patch of a very large collection with elements that contain dozens of optional fields on them?

---

Update

Thanks to @chad-jensen for the idea of attribute prioritization rules. The mentioned document offers quite flexible approach, so I have attempted to tailor it to my needs. I think I will have two operations, the "initialize" with lower priority, and "add" or maybe rather "replace" with higher priority.

Then, in my example, the JSON data sent by client A and client B would be as follows.

Data sent by Client A, which updates the "optional2" field with lower priority and updates the "optional1" field with highest priority.

{
    "email": "[email protected]",
    "initialize":
    {
        "optional2": "A"
    },
    "replace":
    {
        "optional1": "A"
    }
}

Data sent by Client B, which updates the "optional1" and "optional3" fields with lower priority and updates the "optional2" field with highest priority.

{
    "email": "[email protected]",
    "initialize":
    {
        "optional1": "B",
        "optional3": "B"
    },
    "replace":
    {
        "optional2": "B"
    }
}

Answer 1

You'll need some sort of prioritization mapping that decides which fields are prioritized for each client.

It seems that you already have that, but I'm going to assume that it's stored somewhere / the server and or client are aware of which field they have the highest priority(precedence) of. You also might need to know which client last updated each field.

With that said, you should be able to do a a null/empty/check on each attribute and only replace values when they are default value (null/empty string). In the scenario that the value is already populated, then you will need to check if the incoming client has a higher/lesser priority than that of who last updated the value.

I found a document that might clarify what I mean (it gets a little specific about certain 3rd party tools, but the concept is there): https://www.ibm.com/support/knowledgecenter/en/SSPLFC_7.2.0/UserGuide/c_cmdb_reconcilation_overview.html