It appears that my script has started facing a decoding error during requests.
After some investigation and testing, I found out that the issue was related to the header {"Accept-Encoding": "gzip, deflate, br"}. Removing the br solved the problem.
I even tested it on a simpler script and the same issue occured:
import requests
URL = "https://pokeapi.co/api/v2/pokemon/10"
response = requests.request("GET", URL, headers={"Accept-Encoding": "gzip, deflate"})
response_br = requests.request("GET", URL, headers={"Accept-Encoding": "gzip, deflate, br"})
print(response.content.decode(response.encoding))
print(response_br.content.decode(response_br.encoding))
The first print works fine but the second one raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcd in position 1: invalid continuation byte
So, to fix the issue, I have to force the accept-encoding without the br but I feel like it's more a workaround than a proper fix.
Are there more effective/elegant solutions to address this problem?
Python 3.11
requests 2.31.0
Thanks @mark-tolonen for the prompt comment
As stated in the linked answer and
in these requests issue and PR I was just missing the Brotli package
After a pip install Brotli both prints worked fine