Python - Can't decode content when `br` is in `accept-encoding` header

33 Views Asked by At

It appears that my script has started facing a decoding error during requests.
After some investigation and testing, I found out that the issue was related to the header {"Accept-Encoding": "gzip, deflate, br"}. Removing the br solved the problem.

I even tested it on a simpler script and the same issue occured:

import requests


URL = "https://pokeapi.co/api/v2/pokemon/10"

response = requests.request("GET", URL, headers={"Accept-Encoding": "gzip, deflate"})
response_br = requests.request("GET", URL, headers={"Accept-Encoding": "gzip, deflate, br"})

print(response.content.decode(response.encoding))
print(response_br.content.decode(response_br.encoding))

The first print works fine but the second one raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcd in position 1: invalid continuation byte

So, to fix the issue, I have to force the accept-encoding without the br but I feel like it's more a workaround than a proper fix.
Are there more effective/elegant solutions to address this problem?

Python 3.11
requests 2.31.0


Thanks @mark-tolonen for the prompt comment

As stated in the linked answer and in these requests issue and PR I was just missing the Brotli package
After a pip install Brotli both prints worked fine

0

There are 0 best solutions below