How can I re-name duplicate keys in a large JSON file?

246 Views Asked by At

I have a large file following this format. Originally I tried parsing this in javascript with JSONStream and ran into some issues. I've been trying to figure out a way to change the duplicate keys so that I can parse this more easily. For example: Instead of content each would have a counter appended - content-1, content-2. This is a very large file so I cannot do it manually, any suggestions on how I can do this or restructure with js would be greatly appreciated!

{
    "Test": {
        "id": 3454534344334554345434,
        "details": {
            "text": "78679786787"
        },
        "content": {
            "text": 567566767656776
        },
        "content": {
            "text": 567566767656776
        },
        "content": {
            "text": 567566767656776
        }
    }
}
1

There are 1 best solutions below

5
Alejandro Teixeira Muñoz On

I know this is a Javascript question, but I understand you need to load the file in the system and that is a big file.

For this issue, you can use AWK if you are able to use another language outside javascript to parse the file. AWK can be executed under linux, windows bash, etc.

Here is the code:

awk 'BEGIN{a=0}/"content"/{a++;gsub("content","content-"a,$0); print $0}!/"content"/{print $0}' file.json
    {
        "Test": {
            "id": 3454534344334554345434,
            "details": {
                "text": "78679786787"
            },
            "content-1": {
            "content-1": {
                "text": 567566767656776
            },
            "content-2": {
            "content-2": {
                "text": 567566767656776
            },
            "content-3": {
            "content-3": {
                "text": 567566767656776
            }
        }
    }