Getting "Duplicate master name" in sentinel container during redis failover testing.
Tried to test failover on my redis sentinel setup where initially master was redis0 and after rebooting node1 master changed to redis1 but the sentinel (sentinel0) present in the node is not coming up with error: Duplicate master name.
We are deploying using helm chart. Can any redis/sentinel conf change help to solve this?
Pods:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
redis-0 2/2 Running 2 4m3s 10.244.6.5 192.x.x.167 (node1) <none> <none>
redis-1 2/2 Running 0 3m56s 10.244.25.13 192.x.x.150 <none> <none>
redis-2 2/2 Running 0 3m50s 10.244.6.144 192.x.x.244 <none> <none>
sentinel-0 1/2 CrashLoopBackOff 3 (8s ago) 5m10s 10.244.6.2 192.x.x.167 (node1) <none> <none>
sentinel-1 2/2 Running 0 5m8s 10.244.25.12 192.x.x.150 <none> <none>
sentinel-2 2/2 Running 0 5m6s 10.244.6.143 192.x.x.244 <none> <none>
Sentinel 1 log:
1:X 19 Dec 2023 07:44:37.643 # -sdown slave redis-2.redis.redis-partner32-ns.svc.cluster.local:6379 redis-2.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:58.127 # +sdown master mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:58.313 * Sentinel new configuration saved on disk
1:X 19 Dec 2023 07:46:58.313 # +new-epoch 3
1:X 19 Dec 2023 07:46:58.318 * Sentinel new configuration saved on disk
1:X 19 Dec 2023 07:46:58.318 # +vote-for-leader 9e4aadf4982ea00c4fad84cf929fef1b596cd873 3
1:X 19 Dec 2023 07:46:58.617 # +sdown sentinel 7ca36ce7f713f6ec0ff80b61c44ed0f459d40e66 sentinel-0.sentinel 26379 @ mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.238 # +odown master mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 #quorum 2/2
1:X 19 Dec 2023 07:46:59.238 # Next failover delay: I will not start a failover before Tue Dec 19 07:47:18 2023
1:X 19 Dec 2023 07:46:59.381 # +config-update-from sentinel 9e4aadf4982ea00c4fad84cf929fef1b596cd873 sentinel-2.sentinel 26379 @ mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.381 # +switch-master mymaster redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.384 * +slave slave redis-2.redis.redis-partner32-ns.svc.cluster.local:6379 redis-2.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.385 * +slave slave redis-0.redis.redis-partner32-ns.svc.cluster.local:6379 redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:46:59.390 * Sentinel new configuration saved on disk
1:X 19 Dec 2023 07:47:00.428 # +sdown slave redis-0.redis.redis-partner32-ns.svc.cluster.local:6379 redis-0.redis.redis-partner32-ns.svc.cluster.local 6379 @ mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local 6379
1:X 19 Dec 2023 07:47:43.512 # Failed to resolve hostname 'sentinel-0.sentinel'
1:X 19 Dec 2023 07:47:44.826 # Failed to resolve hostname 'redis-0.redis.redis-partner32-ns.svc.cluster.local'
Sentinel0 log:
*** FATAL CONFIG FILE ERROR (Redis 7.0.12) ***
Reading the configuration file, at line 36
6379 2'tinel monitor mymaster redis-1.redis.redis-partner32-ns.svc.cluster.local
Duplicate master name.