How to use jq for firehose metadata extraction in aggregated events

95 Views Asked by At

I have this aggregated single event in firehose. When using jq to create keys:values for I get the below error. My jq is like this

id:.id
users:users[].uuid

Error: metadata extraction failed because JQ query failed. invalid jq query causing a single record to split into 2 or more.

{   "id":20
    "users": [
        {   "uuid":2330
            "first": "Stevie",
            "last": "Wonder"
        },
        {   "uuid":9033131
            "first": "Michael",
            "last": "Jackson"
        }
    ]
}
{   "id":30
    "users": [
        {   "uuid":984209302
            "first": "lower",
            "last": "harvest"
        },
        {   "uuid":8980323932
            "first": "Duncan",
            "last": "mighty"
        }
    ]
}

I want to be able to create dynamicc partitioning using uuid.

{   "id":20
        "users": [
            {   "uuid":2330
                "first": "Stevie",
                "last": "Wonder"
            }
]
},
{   "id":20
        "users": [
            {   "uuid":9033131
                "first": "Michael",
                "last": "Jackson"
            }
        ]
    }
},
    {   "id":30
        "users": [
            {   "uuid":984209302
                "first": "lower",
                "last": "harvest"
            }
        ]
    }
1

There are 1 best solutions below

2
user197693 On

I believe that the part you need is explained in the manual's section on object construction.

This will put the objects in an array, which is close to your desired output.

jq -n '[ inputs | { id, "users": .users[] } ]'

But perhaps this is what you want.

jq '{ id, "users": .users[] }'