NGINX proxy_pass directive with upstream not working, failing on SSL handshake, but works if host is typed on the directly on the directive

Question

I am new to NGINX and am trying to setup a reverse proxy to access a 3rd party service. It has multiple fallback endpoints so I am adding them to an upstream, then use the upstream on the proxy_pass directive. The problem is, if I type the hostname directly it works fine, but it wont work if I set it to use the upstream.

Here is an example of my configuration:

    upstream service_endpoints {
        server ep1.services.com:443;
        server ep2.services.com:443;
        server ep3.services.com:443;
    }

    #This does not work.
    location / {
        proxy_pass https://service_endpoints;
    }

    #This works good.
    location / {
        proxy_pass https://ep1.services.com;
    }

The error I see on the logs is this: [error] 3616#9336: *5 SSL_do_handshake() failed (SSL: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:SSL alert number 40) while SSL handshaking to upstream

I have searched a lot and have tried many things, for example:

    proxy_ssl_server_name on;
    proxy_ssl_verify off;
    proxy_ssl_session_reuse off;
    proxy_set_header Host $host;
    proxy_ssl_name $host;

Nothing works when I try to use the upstream... Its almost like it uses different settings as I had this error before and adding "proxy_ssl_server_name on;" solved it but only if I use the static hostname on the proxy_pass directive.

Any idea what the problem might be?

Thank you