Configuring Python redshift_connector (or psycopg2) to connect with Redshift Serverless

Question

I just want to connect to the data stored in my Redshift serverless using Python.

I see an error when I run the code below.

import redshift_connector
conn = redshift_connector.connect(
    host='Endpoint in the screenshot',
    database='dev',
    user='my aws account id',
    password='my aws pw',
 )

Since it is serverless, the 'Endpoint in the screenshot' looks like '111111111111.us-east-1.redshift-serverless.amazonaws.com' (the red box in the image below).

The error says "redshift_connector.error.InterfaceError: ('communication error', TimeoutError(60, 'Operation timed out'))" When I searched about it, some mentioned 'Edit inbound rules' in Security Group. So I added one rule there with a protocol TCP. Also, some people mentioned 'making the cluster public'. I am not sure if I can set this for the Redshift Serverless. I cannot find the webpage. Plus, some instructions with Redshift require 'cluster_id' as an argument, and I cannot find this info.

What else can I try?

FYI. I am not familiar with AWS. I installed AWS CLI, and I could get the data using CLI by running "aws redshift-data execute-statement ...". So I think it is also possible to get the data using Python libraries. Right?

Answer 1

You can make your serverless cluster publicly accessible: https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-connecting.html

There is a setting when you create the workgroup to make publicly accessible this means it will be able to be queried from the internet.

Otherwise you have to query it from a lambda function or EC2 instance in the same VPC that your cluster is in, which is a better security setting as all the traffic stays in the VPC.