Getting "Duplicate master name" in sentinel container during redis failover testing

61 Views Asked by At

Getting "Duplicate master name" in sentinel container during redis failover testing.

Tried to test failover on my redis sentinel setup where initially master was redis0 and after rebooting node1 master changed to redis1 but the sentinel (sentinel0) present in the node is not coming up with error: Duplicate master name.

We are deploying using helm chart. Can any redis/sentinel conf change help to solve this?

Pods:

NAME         READY   STATUS             RESTARTS     AGE     IP             NODE             NOMINATED NODE   READINESS GATES
redis-0      2/2     Running            2            4m3s    10.244.6.5     192.x.x.167 (node1)  <none>           <none>
redis-1      2/2     Running            0            3m56s   10.244.25.13   192.x.x.150   <none>           <none>
redis-2      2/2     Running            0            3m50s   10.244.6.144   192.x.x.244   <none>           <none>
sentinel-0   1/2     CrashLoopBackOff   3 (8s ago)   5m10s   10.244.6.2     192.x.x.167 (node1)  <none>           <none>
sentinel-1   2/2     Running            0            5m8s    10.244.25.12   192.x.x.150   <none>           <none>
sentinel-2   2/2     Running            0            5m6s    10.244.6.143   192.x.x.244   <none>           <none>

Sentinel 1 log:

1:X 19 Dec 2023 07:44:37.643 # -sdown slave redis-2.redis.redis-partner32-ns.svc.cluster.local:6379 redis-2.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:58.127 # +sdown master mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:58.313 * Sentinel new configuration saved on disk
1:X 19 Dec 2023 07:46:58.313 # +new-epoch 3
1:X 19 Dec 2023 07:46:58.318 * Sentinel new configuration saved on disk
1:X 19 Dec 2023 07:46:58.318 # +vote-for-leader 9e4aadf4982ea00c4fad84cf929fef1b596cd873 3
1:X 19 Dec 2023 07:46:58.617 # +sdown sentinel 7ca36ce7f713f6ec0ff80b61c44ed0f459d40e66 sentinel-0.sentinel 26379 @ mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.238 # +odown master mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 #quorum 2/2
1:X 19 Dec 2023 07:46:59.238 # Next failover delay: I will not start a failover before Tue Dec 19 07:47:18 2023
1:X 19 Dec 2023 07:46:59.381 # +config-update-from sentinel 9e4aadf4982ea00c4fad84cf929fef1b596cd873 sentinel-2.sentinel 26379 @ mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.381 # +switch-master mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.384 * +slave slave redis-2.redis.redis-partner32-ns.svc.cluster.local:6379 redis-2.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.385 * +slave slave redis-0.redis.redis-partner32-ns.svc.cluster.local:6379 redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.390 * Sentinel new configuration saved on disk
1:X 19 Dec 2023 07:47:00.428 # +sdown slave redis-0.redis.redis-partner32-ns.svc.cluster.local:6379 redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:47:43.512 # Failed to resolve hostname 'sentinel-0.sentinel'
1:X 19 Dec 2023 07:47:44.826 # Failed to resolve hostname 'redis-0.redis.redis-partner32-ns.svc.cluster.local'

Sentinel0 log:

*** FATAL CONFIG FILE ERROR (Redis 7.0.12) ***
Reading the configuration file, at line 36
 6379 2'tinel monitor mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local
Duplicate master name.
0

There are 0 best solutions below