Jolt transform from a Map<Person, list<Property>, List<Feature>> to Map<Property, Map<Feature, List<Person>>>

Question

I was wondering if it is possible to do a jolt transform for below use case

Person {
 String id;
}

Property {
 Id
 name
 area
 address
}

Feature{
 String featureId;
}

Basically we have Person having multiple properties and features. We want to reverse map in such a way using jolt transform that we want per Property list of all Person grouped by feature.

Input is:

[
 {
  personId: "personId1",
  properties: [{id: 1, name: "property1", area: "abc"}, {id: 2, name: "property2", area: "def"}],
  features: [{featureId: "feature1"}, {featureId: "feature2"}, {featureId: "feature3"}}]
 },
 {
  personId: "personId2",
  properties: [{id: 1, name: "property1", area: "abc"}, {id: 3, name: "property3"}],
  features: [{featureId: "feature1"}, {featureId: "feature3"}, {featureId: "feature5"}}]
 }
]

Output is expected as:

[
{
 id: 1,
 name: "property1",
 area: "abc",
 featureMap: [
 {
 featureId: "feature1",
 persons: [personId1, personId2]
 },
 {
 featureId: "feature2",
 persons: [personId1]
 },
 {
 featureId: "feature3",
 persons: [personId1, personId2]
 },
 {
 featureId: "feature5",
 persons: [personId2]
 }]
},
{
 id: 2,
 name: "property2",
 area: "def",
 featureMap: [
 {
 featureId: "feature1",
 persons: [personId1]
 },
 {
 featureId: "feature2",
 persons: [personId1]
 },
 {
 featureId: "feature3",
 persons: [personId1]
 }]
},
{
 id: 3,
 name: "property3",
 featureMap: [
 {
 featureId: "feature1",
 persons: [personId2]
 },
 {
 featureId: "feature3",
 persons: [personId2]
 },
 {
 featureId: "feature5",
 persons: [personId2]
 }]
}
]

Above data input will contains hundreds of Person in input with multiple features and properties.

I am not able to understand what should be used as jolt transform here. even this complex transformation is possible. It kind of aggregating all the data per properties, so doing so need some hack in jolt transform to convert the propertyId as key in all the input json and then aggregate using that key to get all the common merged. Might need addition logic to remove duplicates.

Answer 1

First, please make sure when you need help with something like this is to provide valid json data. When I copied your data to work on I noticed double quotes were missing from the keys and in one case there was an extra curly brackets.

Regarding the spec, I think the below should work. Basically you need to create a way to group persons by properties & features (2ed shift transformation). To group easily I have concatenated all possible properties values into a string (1st modify spec). Finally I restructured the grouping output to get the desired result based on possible combinations of the properties concatenation (since some values might not exist which would be easier if not).

[

  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "*": {
        "properties": {
          "*": {
            "allfields": "=toList(@(1,id),@(1,name),@(1,area),@(1,address))",
            "allfieldsConcat": "=join('-',@(1,allfields))"
          }
        }
      }
    }
    }

  ,
  {
    "operation": "shift",
    "spec": {
      "*": {
        "personid": {
          "*": {
            "@(2,properties)": {
              "*": {
                "allfieldsConcat": {
                  "@(5,features)": {
                    "*": {
                      "featureid": {
                        "*": {
                          "@(8)": "[#1].@(5).&1"
                          
                        }
                      } //feature id
                    } //each feature element
                  } //each feature
                } //allfieldsConcat
              } //each property element 
            } //eah property
          } //each personid value
        }
      } // each person
    }
  }
 ,
  {
    "operation": "shift",
    "spec": {
      "*": {
        "*-*-*-*": {
          "$(0,1)": "[#2].id",
          "$(0,2)": "[#2].name",
          "$(0,3)": "[#2].area",
          "$(0,4)": "[#2].address",
          "feature*": {
            "$": "&2.featureMap[].featureId",
            "@": "&2.featureMap[].persons"
          }
        },
        "*-*-*-": {
          "$(0,1)": "[#2].id",
          "$(0,2)": "[#2].name",
          "$(0,3)": "[#2].area",
          "feature*": {
            "$": "[#3].featureMap[#2].featureId",
            "@": "[#3].featureMap[#2].persons"
          }
        },
        "*-*-": {
          "$(0,1)": "[#2].id",
          "$(0,2)": "[#2].name",
          "feature*": {
            "$": "[#3].featureMap[#2].featureId",
            "@": "[#3].featureMap[#2].persons"
          }
        }
      }
    }
  }
  /**/
]

@Barbaros Özhan, I wonder if spec can be written in cleaner more concise way.let me know what you think.

Thanks