Self signed CA verification error with Python and Requests

Question

I'm experiencing some issues with Python and Requests due my Kubernetes cluster not having it's own FQDN.

In short it's a cluster with many different namespaces and ingresses; internal routing is managed as a function of hosts from the request header and whitelisted IPs. When I try shoot of a get-request (with client certificates) to the IP I catch the following exception:

HTTPSConnectionPool(host='*MyExternalIpAddr*', port=443): Max retries exceeded with url: /test(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)')))

The issue I'm having is due to the following code:

url = "https://*MyExternalIpAddr*/test"
headers = {"Host":"*TheHostnameInTheCluster*"}
cert = (cert_file_path, key_file_path)
r = requests.get(url, headers=headers, cert=cert, auth=auth, verify=ca_file)

I'm 99% sure that the IR-url is the problem because if I map the external IP up to the hostname in drivers/etc/host (and change the url to "https://hostname/test") the request goes through without any problems.

Now I'm assuming it has to do with the way Requests checks the client certificates against the rootCA (CN or SAN?). Unfortunately I'm not clever enough to google my way through this last snag.

It's worth mentioning that all the certificates are self generated with openSSL so I can change them if that's needed. I though of maybe putting the IP in the SAN, but because there will be many different microservices with different certificate packages in the cluster I think this could cause conflicts down the road(?).

I'm hoping for a clever way to solve this in Python and therefore I turn to you, o' great collective wisdom of the internet.

Looking forward to hearing back from you (and sorry if I've overlooked a previous answer to a similar question; I've searched far and wide to no avail)!

Edit

So, I've come to realize that the issues has nothing to do with the client certificates, but with the way the root CA validates the server certificate.

I've tried to add the IP to the SAN in the server cert, and it sort of works. With sort of I mean that Postman is no longer throwing a hissy fit when I "Get" from the IP (had to use the FQDM-mapped name from the host-file), but unfortunately Requests is not playing along. Any thoughts? :)

Answer 1

So I solved it with ForcedIPHTTPSAdapter.

import requests
from requests.auth import HTTPBasicAuth
from forcediphttpsadapter.adapters import ForcedIPHTTPSAdapter
### You'll have to "pip install requests[security] forcediphttpsadapter" ###


if __name__ == "__main__":
    cert_file_path = r"*path_to*\client.crt"
    key_file_path = r"*path_to*\client.key"
    ca_file = r"*path_to*\rootCA.crt"
    auth = HTTPBasicAuth("*UID*", "*PWD*")
    ### Cluster IP ###
    cip = "*127.0.0.1*"
    base_url = "*https://hostname.com*"
    endpoint = "/test"
    cert = (cert_file_path, key_file_path)
    s = requests.Session()
    s.mount(base_url, ForcedIPHTTPSAdapter(dest_ip=cip))
    r = s.get(base_url + endpoint, cert=cert, auth=auth, verify=ca_file)
    if r.status_code == 200:
        print(r.text)

And it has the added benefit of not having to specify the host in the headers!