How can I fix python 2.7 keyerror? (Trying to srcape facebook pages)

Question

I have to scrape facebook pages for my studies and I have a probleme with this. I am beginner in python. I found a python code on github: https://github.com/precognox-admin/FBscraper/blob/master/src/scrapeFB.py

Here is a piece of the code:

def write_data(self, d):

    date_inserted = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    messages = d['data']
    self.no_messages += len(messages)

    for message in messages:

        person_name = message['from']['name']
        person_id = message['from']['id']
        person_hash_id = hashlib.md5(person_id.encode('utf-8')).hexdigest()
        published_date = message['created_time']
        published_date = datetime.strptime(published_date, "%Y-%m-%dT%H:%M:%S+%f").strftime("%Y-%m-%d %H:%M:%S")
        post_type = message['type']
        post_id = message['id']
        org_id = post_id.split('_')[0]
        status_id = post_id.split('_')[1]
        post_link = 'https://www.facebook.com/%s/posts/%s' % (org_id, status_id)

When I want to run the .py file I get this error message:

Traceback (most recent call last):
  File "FBscraper/src/scrapeFB.py", line 325, in <module>
    s.scrape()
  File "FBscraper/src/scrapeFB.py", line 264, in scrape
    self.write_data(d)
  File "FBscraper/src/scrapeFB.py", line 91, in write_data
    person_name = message['from']['name']
KeyError: 'from'

I know that it means that the key 'from' doesn't exist but it should be. Here is a JSON for example:

{
  "posts": {
    "data": [
      {
        "comments": {
          "data": [
            {
              "created_time": "2017-09-17T08:42:41+0000",
              "from": {
                "name": "Rafiq Ahmed",
                "id": "10207278423025677"
              },
              "message": "Admirable and applaudable under the 
circumstances!",
              "id": "10155950241431323_10155950367031323"
            }

I've read documentations and whatched and read tutotrials on youtube and on other sides. I've tried .get() method too but it does not works. :/

Answer 1

You should try: messages = d['data'][0]['data'][0]