Internal nginx to serve files to an external nginx: SSL handshake problem

71 Views Asked by At

Here's my problem: I have a django app (served by gunicorn or runserver_plus for dev) in a docker container and we deploy it with kubernetes with a common nginx for all of the apps which manages SSL. I need to serve files from my app (staticfiles and mediafiles) and I want to use another nginx server internally because I can't access to the common nginx (and thus cannot serve files through the external nginx server). Let's say my app runs at web:8080 and it is the upstream for my internal nginx, then my internal nginx listens to web:8000 which is the endpoint for the external nginx.

Now, my app checks whether the header HTTP_X_FORWARDED_PROTO has "https" for value to determine if the site is secure.

So here is my setup to test if this is functioning:

I simulate the external nginx with the following configuration:

server {
    ssl_certificate /app/certificates/cert.crt;  
    ssl_certificate_key /app/certificates/cert.key;

    # if no Host match, close the connection to prevent host spoofing
    listen 443 default_server ssl;
    return 444;
}

server {
    ssl_certificate /app/certificates/cert.crt;
    ssl_certificate_key /app/certificates/cert.key;
    
    listen 443 ssl;
    server_name server.com; 

    location / {
        proxy_pass https://web:8000;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;  # does not contains port ($http_host does)
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;
    }
}

with the corresponding Dockerfile

FROM nginx

RUN rm /etc/nginx/conf.d/default.conf
COPY external-nginx-dev.conf /etc/nginx/conf.d/default.conf

I use it in my docker-compose.yml:

# Docker compose used for integration with docker and nginx
version: '3'
services:
  web:
    image: dev-img
    build:
      context: .
      dockerfile: Dockerfile
      target: dev
    volumes: 
      - .:/app  
    env_file:
      - env/.env.dev
      - env/.env.mssql
    extra_hosts:
      - host.docker.internal:host-gateway
    ports:
      - 8000:8000
    command: supervisord -c /app/supervisord-dev.conf
  
  nginx:
    depends_on:
      - web
    image: external-nginx
    build:
      context: .
      dockerfile: Dockerfile.external-nginx
    volumes: 
      - .:/app  
    extra_hosts:
      - host.docker.internal:host-gateway
    ports:
      - 443:443
    command: nginx -g 'daemon off;'

which sets up my web container with the following nginx.conf:

worker_processes  auto;  
worker_rlimit_nofile 8192;
include /etc/nginx/modules-enabled/*.conf;
events {
  worker_connections  512;  ## Default: 1024
}

http {
    include    /etc/nginx/mime.types;
    include    /etc/nginx/fastcgi.conf;

    default_type application/octet-stream;

    sendfile     on;
    tcp_nopush   on;
    types_hash_max_size 2048;
    server_names_hash_bucket_size 128;

    gzip on;

    server {
        # if no Host match, close the connection to prevent host spoofing
        listen 8000 default_server ssl;
        return 444;
    }

    server {
        listen 8000 ssl;
        server_name server.com;

        location / {
            proxy_pass http://127.0.0.1:8080;

            proxy_pass_request_headers on;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $host;  # does not contains port ($http_host does)
            proxy_set_header X-Forwarded-Proto https;  # maybe $scheme can be used
            
            proxy_ssl_server_name on;

            proxy_redirect off;
        }

        location /static/ {
            alias /app/staticfiles/;
        }

        location /media/ {
            alias /app/data/;
        }
    }
}

started in supervisord:

[supervisord]
nodaemon=true

[program:runserver]
command=/bin/bash -c "python3 manage.py runserver_plus --cert /app/certificates/cert 127.0.0.1:8080"
directory=/app
autostart=true
autorestart=true

[program:nginx]
command=nginx -g 'daemon off;'
autostart=true
autorestart=true

For now, I receive the following error in external nginx when I access to server.com in my browser:

2023/11/21 14:48:38 [error] 29#29: *3 SSL_do_handshake() failed (SSL: error:0A00010B:SSL routines::wrong version number) while SSL handshaking to upstream, client: 172.18.0.1, server: server.com, request: "GET / HTTP/1.1", upstream: "https://172.18.0.2:8000/", host: "server.com"
0

There are 0 best solutions below