Implementing rate limiting with Nginx for dynamic HTTPS server routing without SSL certificates/keys on disk

Question

I'm currently working to block access to an upstream Node.js server executing on the same machine's port 443 via nginx. The hitch is, I can't access any of the SSL certificates or keys from nginx because they aren't on disk for configuration in the nginx file.

To give a little context, I've developed a Node.js HTTPS web server which implements the SNI Callback feature. This script dynamically retrieves the SSL certificate and key from a database according to the hostname specified in incoming requests. It then allows the server to present varying SSL/TLS certificates corresponding to the client's requested hostname. Ideally, all of this will happen at the application level after nginx has forwarded the request to my upstream Node.js server.

I aim to use nginx to implement a rate limit based on known IP addresses and only forward requests to my application if they're not from known blacklisted IPs. The problem is finding the correct nginx directive to achieve this, as most examples seem to read SSL certificate and key data from a disk-based local folder. So, would it be possible to have nginx conditionally block or allow https traffic as I've proposed, without first reading the SSL cert or key from disk?

I've managed a working rate limit for an HTTP server but haven't been able to get it working with my HTTPS server. I suspect the issue might be that my HTTPS Node.js server is only reachable when accessed via a hostname like https://example.com. This is because my HTTPS server relies on the incoming request's hostname to fetch the necessary SSL certificate and key from a database. In contrast, my HTTP server can be reached using an IP address, such as http://8.8.8.8 or http://8.8.8.8:3000, without any issues.

Here is the relevant HTTPS Node.js server code:

const http = require('http');
const https = require('https');
const tls = require('tls');
const { PrismaClient } = require('@prisma/client');
const prisma = new PrismaClient();
require('dotenv').config();

let host;

const httpServer = http.createServer((req, res) => {
    host = req.headers.host;
    console.log('Client IP Address:', req.headers['x-real-ip']);
    console.log('Headers:', req.headers);

    const redirectURL = `https://${req.headers.host}${req.url}`;
    res.writeHead(301, { Location: redirectURL });
    res.end();
});

let httpsServer;

httpServer.listen(3000, () => {
    console.log('HTTP server is running on Port 80');
    startHttpsServer();
});

async function startHttpsServer() {
    try {
        const sniCallback = (hostname, callback) => {
            getSSLCertificates(hostname)
                .then((options) => {
                    const tlsContext = tls.createSecureContext(options);
                    callback(null, tlsContext);
                })
                .catch((error) => {
                    console.error('Error in SNI callback:', error);
                    callback(error);
                });
        };
        httpsServer = https.createServer({ SNICallback: sniCallback }, (req, res) => {
            console.log('Client IP Address:', req.headers['x-real-ip']);
            console.log('Headers:', req.headers);
            res.writeHead(200, { 'Content-Type': 'text/plain' });
            res.end('Hello, HTTPS World!\n');
        });

        httpsServer.listen(443, () => {
            console.log('HTTPS server is running on Port 443');
        });
        
    } catch (error) {
        console.error('Error starting HTTPS server:', error);
        console.error('No SSL certificates found. HTTPS server not started.');
    }
}

…

I've tried countless configurations such as https://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html but am still struggling to get this working.

My attempts include using map and setting the proxy_pass with the domain as the server name, among other things, none of which seem to work.

map $remote_addr $is_limited {
    7.7.7.7 1;
    default 0;
}

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;

server {
    listen 80;
    server_name example.com;

    location / {
        limit_req zone=mylimit burst=1 nodelay;

        proxy_pass https://example.com;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

I have also tried TCP proxying

geo $blacklist {
    default 1; # Block all traffic
    # There is no need to add any blacklisted IP here, because all IPs are already blocked
}

# Main server block for TCP proxying

server {
    listen 8843;
    # listen 443;

    # Allow connections only from non-blacklisted IPs
    if ($blacklist) {
        return 403; # Forbidden
    }

    # SSL Passthrough for SNI-based routing
    proxy_ssl_server_name on;
    proxy_set_header X-Forwarded-Host $ssl_server_name;

    location / {
        proxy_pass https://example.com/;
    }
} 

None of which have lead to the results I am looking for.

As i mentioned above I have, though, been able to implement a basic rate limit when paired with an HTTP server at an IP address, such as http://8.8.8.8.

For instance, when I alter my nginx config file to:

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;

server {
    listen 80;
    server_name 8.8.8.8;

    location / {
        limit_req zone=mylimit burst=1 nodelay;

        proxy_pass http://8.8.8.8:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

...and modify my Node.js code

const http = require('http');

const server = http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/plain' });
  res.end('Hello, World!\n');
});

const port = 3000;
server.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}/`);
});

I am effectively rate-limited.

But I really need help adapting this to HTTPS; your suggestions would be greatly appreciated!