How to test redis failover with redis sentinel and docker compose?

208 Views Asked by At

I'm starting using redis. I wrote a docker compose with 1 master, 1 replica and 1 sentinel.

version: '3.8'

  redis-master:
    image: redis:7.2.3
    command: redis-server /etc/redis/master.conf
    volumes:
      - ./conf:/usr/local/etc/redis
    ports:
    - "6379:6379"

  redis-slave:
    image: redis:7.2.3
    command: redis-server /etc/redis/slave.conf
    volumes:
      - ./conf:/etc/redis
    links:
      - redis-master
    ports:
      - "6380:6379"
      
  sentinel:
    image: redis:7.2.3
    command: bash -c "
      chown redis:redis /etc/redis/sentinel.conf &&
      redis-server /etc/redis/sentinel.conf --sentinel
      "
    volumes:
      - ./conf:/usr/local/etc/redis
    depends_on:
      - redis-master
      - redis-slave
    ports:
      - "26381:26379"

Here is my sentinel.conf :

sentinel announce-hostnames yes
sentinel resolve-hostnames yes
sentinel announce-ip "sentinel"
sentinel monitor mymaster redis-master 6379 1
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 500
port 26379
dir "/data"
sentinel known-replica mymaster redis-slave 6379

master.conf :

bind 0.0.0.0
protected-mode no
port 6379
dir "/data"
replica-announce-ip "redis-master"

slave.conf :

bind 0.0.0.0
protected-mode no
port 6379
dir "/data"
slave-read-only no
slaveof redis-master 6379
replica-announce-ip "redis-slave"

Everything works fine using hostname instead of IP in my configurations. When I use SENTINEL failover mymaster or docker compose pause redis-master, my replica is granted to master.

However, when I do : docker compose stop redis-master here are my logs in my sentinel :

+tilt #tilt mode entered
Failed to resolve hostname 'redis-master'
+tilt #tilt mode entered
Failed to resolve hostname 'redis-master'

If my sentinel cannot contact my master, isn't it supposed to grant my replica ?

I tried to add : sentinel known-replica mymaster redis-slave 6379 but my sentinel is still trying to resolve hostname 'redis-master' and my replica is not granted.

I am expecting to have a replica granted to master after a docker compose stop redis-master. Or my test is wrong and my sentinel's logs are normal and we just can't test failover using docker compose stop.

Thanks for the answers

0

There are 0 best solutions below