AWS Glue Python Shell script timing out

Question

I am trying to connect to and read an on-premise data source using an AWS Glue Python Shell job. I am using Pygresql (which comes bundled on Glue) and Pandas. Everything works locally.

But when I push this job up to Glue, the database connections all timeout. Why is this happening? Do I need to do something magic with VPCs?

Answer 1

Note that pygresql SQL query doesn't work in python shell. Recommended is postgresql

Answer 2

I guess you need to create a Glue connection with your VPC settings and attach it to the Glue job.