Inverse operation for a yaml merge such as yq multiply?

Question

Many application read several input files and merge them logically such that inputs with higher priority override previously read fields.

For yaml files, we can achieve this logic using a merge command such as (unsing Go yq):

yq '. *= load("special.yaml")' base.yaml > merged.yaml

For a base.yaml:

a:
  - foo
  - bar
b:
  foo: bar

and special.yaml:

b:
  foo: SPECIAL
  bar: SPECIAL
c:
  - SPECIAL

This yields as merged.yaml:

a:
  - foo
  - bar
b:
  foo: SPECIAL
  bar: SPECIAL
c:
  - SPECIAL

The task at hand is to reverse this operation:

Given merged.yaml and base.yaml, derive a file that only contains the subset of merged.yaml not in base.yaml. The result then can be plugged into the command above as special.yaml to recreate merged.yaml

Answer 1

You didn't specify which of the two implementations of yq you are using (see the Tag Info to yq).

With kislyuk/yq, you can break up the scalars of both documents into their stream representation (paths and values) using tostream, then subtract one from the other, and iteratively re-assemble the remaining difference using setpath:

yq -sy '
  reduce (map([tostream | select(has(1))]) | .[0] - .[1])[] as $i
    (null; setpath($i[0]; $i[1]))
' merged.yaml base.yaml

b:
  foo: SPECIAL
  bar: SPECIAL
c:
  - SPECIAL

This solution also works with itchyny/gojq, using the flags gojq -s --yaml-input --yaml-output.

---

Edit:

Adds linkt to Go yq

unsing Go yq

You could convert the approach from above into a mikefarah/yq filter by manually breaking up the documents' scalars into their paths and values using .. and path, and then just perform the same subtraction and the same iterative re-assembly:

yq ea '
  [[.. | select(kind == "scalar") | [path, .]]] | .[0] - .[1]
  | .[] as $i ireduce(null; setpath($i[0]; $i[1]))
' merged.yaml base.yaml

b:
  foo: SPECIAL
  bar: SPECIAL
c:
  - SPECIAL