Freshdesk Contact API to PostgreSQL Table

116 Views Asked by At

I have been trying to run this query to fetch contact data from the Freshdesk API and pipe it into a Postgres table using sqlalchemy in Python. I keep getting the following error: psycopg2.errors.UndefinedColumn: column "other_companies" of relation "freshdesk_contacts" does not exist

I'm not sure how that can be, since my understanding is that my python script creates the freshdesk_contacts table based off of the columns in the pandas dataframe that has all the Freshdesk API data in it.

Anyone have any ideas of what could be going wrong?

Also, I've included a picture as an example of the json output for one company. this is an example of the json output for 1 contact

import requests
import json
import pandas as pd
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from time import sleep


# Load sensitive information
url = 'https://<my_domain>.freshdesk.com/api/v2/contacts'
token = '<my_token>'
db_url = '<my_db_url'


# Define a function to handle API requests and insert data into the database
def fetch_contacts(conn, page=1, per_page=100):
    headers = {'Authorization': 'Basic ' + token}
    params = {'page': page, 'per_page': per_page}
    response = requests.get(url, headers=headers, params=params)
    if response.status_code == 429:
        print('Rate limit exceeded. Waiting 60 seconds...')
        sleep(61)
    if response.status_code == 200:
        data = json.loads(response.content)
        flattened_data = pd.json_normalize(data)
        flattened_data.to_sql('freshdesk_contacts', conn, if_exists='append', index=True)
        if len(flattened_data) == per_page:
            return True, page+1 # return the next page number
    return False, page # return the current page number

engine = create_engine(db_url)

# Fetch tickets and insert data into the database
with engine.connect() as conn:
    more_contacts_remaining = True
    contact_page = 1
    while more_contacts_remaining:
        more_contacts_remaining, contact_page = fetch_contacts(conn, page=contact_page)
        print('Loading...')
    conn.commit()
0

There are 0 best solutions below