I have:
- 4 servers running a single RethinkDB instance in cluster (4 shards / 3 replicas tables)
- 2 application servers (tornado + RethinkDB proxy)
The clients connect only to their local proxy.
How to specify both the local + the other proxy so that the clients could fail over to the other proxies when their local proxy crashes or experiences issues?
Below is my opinion on how I setup thing.
When the local proxy crashes, they should restart by using a process monitor like systemd.
I don't use RethinkDB local proxy. I used HAProxy runs in TCP mode locally on every app server, to forward to RethinkDB. I used Consul Template so that when a RethinkDB node join cluster, HAProxy configuration will be updated and add the node and restart on its own.
HAProxy is very lighweight and rock solid for me. Not just RethinkDB, HAProxy runs locally and do all kind of proxying request, even MySQL/Redis...
HAProxy has all kind of routing/failover scenrario, like backup backend,...