Merging array values of arbitrary keys with jsonnet

Question

I have a list of configuration files in a jsonnet document main.jsonnet, like this:

local configFiles = [
  import "file1.jsonnet",
  import "file2.jsonnet",
  import "file3.jsonnet",
];

{
# ...something goes here...
}

The imported files contain arbitrary sets of keys (that is, we do not know the complete list of keys in advance). For this example, file1.jsonnet looks like:

{
  names: ["alice", "bob"],
  colors: ["red"],
}

file2.jsonnet looks like:

{
  names: ['mallory'],
  animals: ['cow'],
}

And file3.jsonnet looks like:

{
  names: ['angus'],
  colors: ['brown'],
  animals: ['horse'],
  size: ['medium'],
}

I want to replace # ...something goes here... with appropriate code to generate:

{
  names: ["alice", "bob", "mallory", "angus"],
  colors: ["red", "brown"],
  animals: ["cow", "horse"],
  size: ["medium"],
}

This would be easy with a simple iterative solution; in Python I might write something like:

import _jsonnet as jsonnet
import json

merged = {}
fileNames = ["file1.jsonnet", "file2.jsonnet", "file3.jsonnet"]
configFiles = [json.loads(jsonnet.evaluate_file(fn)) for fn in fileNames]

for configFile in configFiles:
    for k, v in configFile.items():
        merged[k] = merged.get(k, []) + v

print(json.dumps(merged, indent=2))

The logic seems simple, but the only iterators available in jsonnet are array and object comprehensions which seems trickier. I tried this:

{
    [k]: if self[k] then self[k] + configFile[k] else configFile[k]
    for configFile in configFiles
    for k in std.objectFields(configFile)
}

But that results in:

RUNTIME ERROR: duplicate field name: "names"
        main.jsonnet:(7:1)-(11:2)

Answer 1

A very useful std function to overcome the [keyName] duplicate issue is std.foldl.

Below example implements merging the arrays with it, for the sake of simplicity I've embedded the configFiles in the same src (i.e. no import which would achieve the same in this case).

source: foo.jsonnet

local f1 = {
  names: ['alice', 'bob'],
  colors: ['red'],
};
local f2 = {
  names: ['mallory'],
  animals: ['cow'],
};
local f3 = {
  names: ['angus'],
  colors: ['brown'],
  animals: ['horse'],
  size: ['medium'],
};
local configFiles = [f1, f2, f3];

// Use std.foldl() which will call the function() one for each element in the array,
// thus somehow "avoiding" the `[keyName]` duplicate issue
std.foldl(
  function(acc, x) acc {
    [kv.key]+: kv.value
    for kv in std.objectKeysValues(x)
  },
  configFiles,
  {}
)

output

$ jsonnet foo.jsonnet
{
   "animals": [
      "cow",
      "horse"
   ],
   "colors": [
      "red",
      "brown"
   ],
   "names": [
      "alice",
      "bob",
      "mallory",
      "angus"
   ],
   "size": [
      "medium"
   ]
}