Decode text/event-stream content type to JSON

404 Views Asked by At

I'm accessing an endpoint that is responding with two objects almost at the same time with a content-type of 'text/event-stream'.

I'm struggling to access this data in Python.

I tried turning it into JSON with the .json() function, however that seems to not be working.

Does anyone have any ideas on how I can access the data from this response?

Here is the response in the network tab:

enter image description here

My Code:

import requests

request_session = requests.session()

initialResponse = request_session.post('https://transport.tamu.edu/busroutes.web/mapHub/negotiate?negotiateVersion=1').json()

connectionToken = initialResponse['connectionToken']

finalResponse = request_session.get(f'https://transport.tamu.edu/busroutes.web/mapHub?id={connectionToken}') # .json() not working

print(finalResponse) # 200 OK Response

I tried using the SSEClient library as such, but can only see an empty string:

import requests
from sseclient import SSEClient
import json

request_session = requests.session()

initialResponse = request_session.post('https://transport.tamu.edu/busroutes.web/mapHub/negotiate?negotiateVersion=1').json()

connectionToken = initialResponse['connectionToken']

sse = SSEClient(f'https://transport.tamu.edu/busroutes.web/mapHub?id={connectionToken}')

for event in sse:
    if event.event == 'message':
        try:
            # Parse the event data as JSON
            json_data = json.dumps(event.data)
            print(json_data)
            
            # Handle or process the JSON data as needed
            # For example, you can access json_data['key'] to access specific fields
        except json.JSONDecodeError as e:
            print(f"Error parsing JSON: {e}")
    else:
        # Handle other types of events if needed
        pass

enter image description here

For reference, I'm trying to scrape the network requests from this site: 'https://transport.tamu.edu/busroutes.web/Routes?r=01'

Thanks!

0

There are 0 best solutions below