Filter JSON with JOLT Transform

323 Views Asked by At

I am writing Jolt Specification in Apache NiFi and got stuck with filtering JSON array.

I have an input like:

{
  "blacklisted": ["A", "B", "F"],
  "items": [
    {"name": "A", "data": 1},
    {"name": "B", "data": 3},
    {"name": "C", "data": 4},
    {"name": "D", "data": 7}
  ]
}

I need a result to be:

{
  "items":[
    {"name": "C", "data": 4},
    {"name": "D", "data": 7}
  ]
}

Is it possible to filter data with JOLT Transform for this case? Can you please help me with the jolt spec?

Another option is to get output like:

{
  "blacklisted": ["A", "B", "F"],
  "items": [
    {"name": "A", "data": 1, "blacklisted": true},
    {"name": "B", "data": 3, "blacklisted": true},
    {"name": "C", "data": 4, "blacklisted": false},
    {"name": "D", "data": 7, "blacklisted": false}
  ]
}
1

There are 1 best solutions below

0
On

You can use successive shift transformation specs such as

[
  {
    "operation": "shift",
    "spec": {// make "name" values objects labels which are under "items" array 
      "*": "&",
      "items": {
        "*": {
          "@": "@(1,name)"
        }
      }
    }
  },
  {// combine the values from "blacklisted" and "items" in order to get arrays vs. non-arrays
    "operation": "shift",
    "spec": {
      "blacklisted": {
        "*": {
          "*": {
            "@(3,&)": "&"//traverse 3 levels up the tree
          }
        }
      },
      "*": {
        "@(1,&)": "&"
      }
    }
  },
  {// eliminate the arrays which are generated by duplicated object labels by exchanging keys vs. values of objects
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "$": "&2.@(0)"
        }
      }
    }
  },
  {// exhange back keys vs. values
    "operation": "shift",
    "spec": {
      "*": {
        "*": {
          "$": "items.&2.@(0)"
        }
      }
    }
  },
  {// get rid of extra object labels
    "operation": "shift",
    "spec": {
      "*": {
        "*": "&1"
      }
    }
  }
]

in order to get

{
  "items":[
    {"name": "C", "data": 4},
    {"name": "D", "data": 7}
  ]
}