Extract composite unique key from GoHighLevel API with Python {{ contact.utm_source }}

Question

I need to separate the parts of a unique key {{ contact.utm_source }} from the GoHighLevel API, with Python.

— "Contact" is a dataset from the df_Contacts dataframe and unique key "id".

— "utm_source" appears as a value in the [fieldKey] column of the df_Custom dataframe, and also a column with values ​​that I cannot access through the API, but can check through the CRM.

I need to transform "utm_source" into a column and iterate with the contacts. Note: "id" is a column in the df_Contacts dataframe but does not appear in the df_Custom dataframe, it is implicit in "contacts" in the key {{ contact.utm_source }} .

I managed to separate this single key conta.utm_source, but I couldn't get the values ​​from the [utm_source] column... the column is empty, and I'm sure of the data because I can check it through the CRM.

import requests
import pandas as pd

# URL and headers for the first request
url = 'https://rest.gohighlevel.com/v1/custom-fields/'
headers = {
'Authorization': 'Bearer'
}

# First request to obtain customFields data
response = requests.get(url, headers=headers)
dataCustom = response.json()

# Assuming the JSON has a "customFields" key that contains a list of dictionaries
customFields = dataCustom['customFields']

# Create the DataFrame directly from the dictionary list
df_Custom = pd.DataFrame(customFields)



# URL and headers for the second request
url = 'https://rest.gohighlevel.com/v1/contacts/?limit=100'
headers = {
'Authorization': 'Bearer'
}

# Request to obtain contact data
response = requests.get(url, headers=headers)
dataContacts = response.json()

# Assuming the JSON has a "contacts" key that contains a list of dictionaries
contacts = dataContacts['contacts']

# Create the DataFrame directly from the dictionary list
df_Contacts = pd.DataFrame(contacts)



# Create an empty dataframe to store the merged data
df_merged = pd.DataFrame()

# Create a dictionary to map the column names:
column_map = {
"{{ contact.utm_source }}": "utm_source",
}

# Iterate DataFrame & Extract ID
for index, row in df_Contacts.iterrows():
contact_id = row["id"]  # Extract ID from df_Contacts

# Filter df_Custom based on the actual ID from df_Contacts
filtered_custom = df_Custom[df_Custom["fieldKey"].str.startswith(f"contact:{contact_id}")]

# Check if any matching rows are found
if not filtered_custom.empty:
custom_value = filtered_custom[column_map["{{ contact.utm_source }}"]].values[0]
df_merged.loc[index, "utm_source"] = custom_value
else:
# Handle cases where no matching row is found in df_Custom
df_merged.loc[index, "utm_source"] = None  # Set a default value for missing data