EC2 instance will not connect to DocumentDB in different AZ. Why?

Question

I have set up a documentdb cluster in us-east-1. I am attempting to connect via an EC2 instance in us-west-1. I have set up connection peering with the VPC in us-west-1 having a CIDR of 172.31.0.0/16 and the VPC in us-east-1 having a CIDR of 172.32.0.0/16. Connection peering is established and active. When I attempt to connect to the documentdb from mongo shell from the EC2 instance, I get the exception:

connecting to: mongodb://cluster-name.cluster-uniquecode.us-east-1.docdb.amazonaws.com:27017/?gssapiServiceName=mongodb
2020-07-15T00:50:16.004+0000 W NETWORK https://forums.aws.amazon.com/ Failed to connect to 172.32.83.229:27017 after 5000ms milliseconds, giving up.
2020-07-15T00:50:16.004+0000 E QUERY https://forums.aws.amazon.com/ Error: couldn't connect to server cluster-name.cluster-uniquecode.us-east-1.docdb.amazonaws.com:27017, connection attempt failed :
connect@src/mongo/shell/mongo.js:263:13
@(connect):1:6
exception: connect failed

The security group attached to the us-east-1 VPC is set to allow all IP addresses and all ports, so that doesn't seem to be the issue.

So.... why the the failure to connect? Anything I missed?

Answer 1

VPC peering does not implictly handle reverse-path routes for return traffic, so tou need to add routes to both VPCs.

You need routes in the tables of VPC A sending b.b.b.b/x over the peering connection and you need routes in VPC B to send a.a.a.a/y traffic over the peering connection, regardless of which end originates the traffic.

The owner of the peer VPC must also complete these steps to add a route to direct traffic back to your VPC through the VPC peering connection.

https://docs.aws.amazon.com/vpc/latest/peering/vpc-peering-routing.html

Answer 2

I would take a look to the route tables in VPC for us-west-1. Make sure there is a record that sends 172.32.0.0/16 through the vpc peering.