How to setup an aiohttp server using mTLS self signed SSL CA Certificates?

Question

I'm trying to setup a python server on my Ubuntu 22 server.

A web/api server. It's only for private usage, and only ever needs to be accessed by a few certain devices. Therefore buying a domain or getting a certificate signed by a CA isn't an option. The goal is to have a .pfx file that I can hand clients. Only those with the certificate installed should have access the my server. The certificate is for security, both the server and client. Clients without the certificate shouldn't be able to access the server, even when ignoring the security warning.

So I created a ca.key and self signed ca.crt. I also created a server.key and signed a server.crt using the ca.crt. I then created a .pfx file using server.key, server.crt and ca.crt.

——- EDIT: Here‘s how I created the certificates, following a guide at https://deliciousbrains.com/ssl-certificate-authority-for-local-https-development/

openssl genrsa -des3 -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -sha256 -days 824 -out ca.crt

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 824 -sha256 -extfile server.ext

openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 824 -sha256 -extfile client.ext

openssl pkcs12 -export -out client.p12 -inkey client.key -in client.crt -certfile ca.crt

——-

(Important note: Due to some bug when creating the .pfx file and entering the password in the most recent version of openssl 3.2.1, one cannot use the entered password when installing the certificate on iOS, therefore I'm using openssl (1.1.1) currently See: https://developer.apple.com/forums/thread/723242 https://github.com/strongswan/strongswan/discussions/1554 )

Now, when installing the .pfx file on my machine tried windows and iOS, I cannot access the server, in fact, My browser displays:

ERR_CONNECTION_REFUSED

My server log shows:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 213, in _accept_connect
    await waiter
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 534, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/lib/python3.10/asyncio/sslproto.py", line 188, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/lib/python3.10/ssl.py", line 975, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: PEER_DID_NOT_RETURN_A_CERTIFICATE] peer did not return a certific

Here's my server code:

import asyncio
from aiohttp import web
import json
import ssl
import logging



async def handler(req):
    packed = await req.read()
    data = json.loads(packed)
    return web.Response(text=f"Thank you for your {data}")

async def create_server():
    app = web.Application()

    ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    ssl_context.load_cert_chain(certfile='server.crt', keyfile='server.key')
    ssl_context.load_verify_locations('ca.crt')
    ssl_context.verify_mode = ssl.CERT_REQUIRED

    app.add_routes([
        web.get('/', handler),
    ])

    runner = web.AppRunner(app)
    await runner.setup()
    site = web.TCPSite(runner, '0.0.0.0', 2343, ssl_context=ssl_context)
    await site.start()

    return app

async def main():
    app = await create_server()
    await asyncio.sleep(10_000)

if __name__ == '__main__':
    asyncio.run(main(), debug=True)

I'm almost certain that the error is in the way I've created and used the certificates, as I can access my server without problem when not using a SSL context & http:// prefix. Therefore I'd be glad if someone could provide a detailed step-by-step guide on how to create the certificates and which certificate to use where and hand to whom. Best regards!

Answer 1

For the sake of being a good Stackoverflow-communitymember, I figured it out myself (kind of), and here's the solution.

First of all I discovered this incredible tutorial: https://learn.microsoft.com/en-us/azure/application-gateway/self-signed-certificates And it also features this link: https://appgwbackendcertgenerator.azurewebsites.net/ where you can have the certificates needed generated in one click. For the 'domain name' I used my servers IP-address and port: xx.xxx.xxx.xx:abcd/ I'm mostly certain that it doesn't matter what one puts into that field. Also creating the certificates as described on the Microsoft page also works fine.

Now the complete guide:

1. Open the website https://appgwbackendcertgenerator.azurewebsites.net/, put in your servers IP-address and port: xx.xxx.xxx.xx:abcd/, click 'Generate Certificate' and download the file: certificates.zip.

2. Head to your venv or create a dir, call it whatever you want. Inside, unpack the .zip file into certificates dir.

3. Create the server.py (main.py, whatever.py) inside as follows:

import asyncio
from aiohttp import web
import json
import ssl
import logging


async def handler(req):
    return web.Response(text=f"Everthing worked. Hello secure web.")

async def create_server():
    app = web.Application()

    ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    ssl_context.load_cert_chain(certfile='certificates/server.cer', keyfile='certificates/server.key')
    ssl_context.load_verify_locations('certificates/root.cer')
    ssl_context.verify_mode = ssl.CERT_REQUIRED

    app.add_routes([
        web.get('/', handler),
    ])

    runner = web.AppRunner(app)
    await runner.setup()
    site = web.TCPSite(runner, IPADDRESS, PORT, ssl_context=ssl_context)
    await site.start()

    return app

async def main():
    app = await create_server()
    await asyncio.sleep(10_000)

if __name__ == '__main__':
    asyncio.run(main(), debug=True)

-> Fill in IPADDRESS and PORT as needed, for testing, you can use IPADDRESS = 'localhost' and PORT = 8080.

4. Run the programm. If you now try to open https://localhost:8080/, you'll get ERR_CONNECTION_REFUSED.

5. Open your browsers settings. Find the certificates section in the browser. (In my Opera Browser it was under security). You should find some button, when clicked, a system window opens, where certificates can be installed. On my windows 10 machine it looked like this: Hit import, select the server.pfx file. You'll need to prompt a password, which is stored in the password.txt. You'll get a confirmation message. You may need to restart your browser or even device afterwards.

6. If you now reload the website, you'll get promted to select a certificate. Select the one just imported. You should now see a confirmation message.

-> Note, I've not yet tested the import of the .pfx on my iPhone. I'll give an update about that in the comments. Also, if anybody could give a guide on how to sign different client certificates and create the according .pfx, that would still be great.