Cannot connect to AWS RDS Postgres with pg8000 but good with psycopg2

Question

friends,

I'm trying to connect to a Postgres instance in AWS RDS from python script running in an EC2 machine. It's working well with psycopg2/sqlalchemy as

from sqlalchemy import create_engine 
import psycopg2

db_string = 'postgresql+psycopg2://{user}:{password}@{host}:5432/{database}'.format(
        user=DB_USER,
        password=DB_PASS,
        host=DB_HOST,
        database=DB_NAME)

engine = create_engine(db_string, echo=True)    
conn = engine.connect()

But it failed when I use pg8000/sqlalchemy as

from sqlalchemy import create_engine 
import pg8000

db_string = 'postgresql+pg8000://{user}:{password}@{host}:5432/{database}'.format(
        user=DB_USER,
        password=DB_PASS,
        host=DB_HOST,
        database=DB_NAME)

 engine = create_engine(db_string, echo=True)    
 conn = engine.connect()

The error info is

InterfaceError: (pg8000.exceptions.InterfaceError) {'S': 'FATAL', 'V': 'FATAL', 'C': '28000', 'M': 'no pg_hba.conf entry for host "*.*.*.*", user "*", database "*", SSL off', 'F': 'auth.c', 'L': '513', 'R': 'ClientAuthentication'}
(Background on this error at: http://sqlalche.me/e/14/rvf5)

I also tried to set the connect_args={'ssl_context': True} but still failed on the connection. Anyone has any thoughts? I'm using python v3.6, sqlalchemy v1.4.15, and pg8000 v1.19.4. Thank you.

Answer 1

The value for the ssl_context key is typically created with ssl.create_default_context().

By default that uses the trust the system’s default CA certificates. One can also download AWS's certificate bundle (a global one, or one of the region specific variants. See https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/UsingWithRDS.SSL.html#UsingWithRDS.SSL.CertificatesAllRegions). And then set the parameter cafile. E.g. Something like this:

import ssl
from os import path

# This snippet uses sqlmodel's create_engine which uses sqlalchemy under the hood.
from sqlmodel import Session, create_engine

# assume that the PEM file is in the same directory as current python script file.
ssl_server_certs_path = path.join(path.dirname(__file__), "global-bundle.pem")

engine = create_engine(
        config_uri,
        connect_args={"ssl_context": ssl.create_default_context(cafile=ssl_server_certs_path)})

Answer 2

As of pg8000 1.31.0, by default an SSL connection will be attempted using lenient settings, so it should work in the same way as psycopg.