Accessing filing data from SEC EDGAR API in python

Question

I want to pull reports from the SEC EDGAR API and conduct analysis within python. From what I can tell, it looks like the main issue is that Im using the wrong file format, but methods I have found to convert to HTML did not work.

I have limited experience in python and even less with RESTful API use. I found some resources for the API on the SEC website but I couldnt make much sense out of it.

I also tried printing the response into a dataframe but I got a 403 response

This is my code:

import requests
import pandas as pd
import numpy as np

ticker = "AAPL"
start_date = "2022-01-01"
end_date = "2020-12-31"



sec_url = "https://www.sec.gov/cgi-bin/browse-edgar"
sec_params = {
    "action": "getcompany",
    "CIK": ticker,
    "type": "10-k",
    "dateb": start_date,
    "owner": "exclude",
    "count": 100
}

sec_response = requests.get(sec_url, params=sec_params)
sec_data = sec_response.json()

The error I received was a JSONDecodeError

Answer 1

The issue is that response returned is not JSON, it is an HTML page containing the error. (You can check the response returned by using sec_response.text).

In your case, SEC requires you to include your company/email in the USER-AGENT of the request. (See here)

USER-AGENT should be included in the header of the request. To see how you can check this StackOverflow question:

...

headers = {
    'User-Agent': 'Sample Company Name AdminContact@<sample company domain>.com'
}

...

sec_response = requests.get(sec_url, params=sec_params, headers=headers)