Loading json data into Cassandra using dsbulk

612 Views Asked by At

I feel like the documentation on loading json files into cassandra is really lacking in dsbulk docs.

Here is part of the json file that im trying to load:

[
  {
    "tags": [
      "r"
    ],
    "owner": {
      "reputation": 23,
      "user_id": 12235281,
      "user_type": "registered",
      "profile_image": "https://www.gravatar.com/avatar/60e28f52215bff12adb9758fc2cf86dd?s=128&d=identicon&r=PG&f=1",
      "display_name": "Me28",
      "link": "https://stackoverflow.com/users/12235281/me28"
    },
    "is_answered": false,
    "view_count": 3,
    "answer_count": 0,
    "score": 0,
    "last_activity_date": 1589053659,
    "creation_date": 1589053659,
    "question_id": 61702762,
    "link": "https://stackoverflow.com/questions/61702762/merge-dataframes-in-r-with-different-size-and-condition",
    "title": "Merge dataframes in R with different size and condition"
  },
  {
    "tags": [
      "python",
      "location",
      "pyautogui"
    ],
    "owner": {
      "reputation": 1,
      "user_id": 13507535,
      "user_type": "registered",
      "profile_image": "https://lh3.googleusercontent.com/a-/AOh14GgtdM9KrbH3X5Z33RCtz6xm_TJUSQS_S31deNYUcA=k-s128",
      "display_name": "lowhatex",
      "link": "https://stackoverflow.com/users/13507535/lowhatex"
    },
    "is_answered": false,
    "view_count": 2,
    "answer_count": 0,
    "score": 0,
    "last_activity_date": 1589053657,
    "creation_date": 1589053657,
    "question_id": 61702761,
    "link": "https://stackoverflow.com/questions/61702761/want-to-get-a-grip-of-this-pyautogui-command",
    "title": "Want to get a grip of this pyautogui command"
  }
]

The way I have been trying to load this is following: dsbulk load -url ./data_so1.json -k stackoverflow_t -t staging_t -h '182.14.0.1' -header false -u username -p password

This is the closest i get and it pushes the values into Cassandra row by row like this:

 data
-------------------------------------------------------------------------------------------------------------------------------
      "title": "'Microsoft.ACE.OLEDB.12.0' provider is not registered on the local machine giving exception on client"
               "profile_image": "https://www.gravatar.com/avatar/05085ede54486bdaebefcf8363e081e2?s=128&d=identicon&r=PG&f=1",
                                                                                                            "view_count": 422,
                                                                                                      "question_id": 61702768,
                                                                                                          "user_id": 12235281,

This just takes the rows as they are (including the commas). I've tried the -m key for mapping but didnt really get anywhere with it.

What would be the right way to get these values to their own respective columns?

0

There are 0 best solutions below