Python requests not looking into /etc/ssl/certs for self-signed certificates

Question

I'm trying to add a certificate into a Dockerfile, needed for Python requests package:

FROM python:3.9-slim-buster
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH="$PYTHONPATH:/app"
WORKDIR /app
COPY ./app .
COPY ./certs/*.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates
RUN pip3 install requests
CMD ["python3", "main.py"]

With the above Dockerfile, I get the following error:

[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain

Based on my tests, that is because requests is using certifi and is looking only inside /usr/local/lib/python3.9/site-packages/certifi/cacert.pem. If I add my certificates inside cacert.pem, everything works as expected and the errors are gone.

What is the pythonic way to deal with this issue? Ideally, I would prefer to insert certificates into a directory, instead of modifying a file. Is there a way to "force" Python requests look inside /etc/ssl/certs for certificates, as well into certifi cacert.pem file? If I list the /etc/ssl/certs directory contents, it contains my .pem certificates.

Running an apt-get update will not update ca-certificates, I'm already using the latest version. When I execute update-ca-certificates, the new certificates are detected:

STEP 10/11: RUN update-ca-certificates
Updating certificates in /etc/ssl/certs...
2 added, 0 removed; done.

Thank you for your help.

Answer 1

There only reasonable solution I found is:

from requests import post
from requests.exceptions import HTTPError, RequestException, SSLError

try:
    result = post(url=url, data=dumps(data), headers=headers, verify='/etc/ssl/certs')
except (HTTPError, RequestException, SSLError) as e:
    raise

Setting verify=/etc/ssl/certs will see the self-signed certificates.