How to Handle Multiple servers trying to clear the redis Database

Question

I have four Severs connected to redis. When the redis server goes down and Comes up. I want to clear the redis database. Issue is that All the servers are firing the clear action. How to handle such that only one servers fires the clear and notifies to all other servers. I am using the StackExchange.Redis and CacheManager.Net Packages as redis client libraries

Answer 1

To disable any save, set appendonly no

and comment or delete any save

#save 900 1

If you don't want to disable Redis from saving data, and more in general, you'd have to sync communication somehow, which is not easy as this is a distributed system.

You can try Add a fake key with CacheManager before clearing the cache. Only one client will be able to write the key, and the Add method returns False in case the key already exists. Meaning, only the first client will write the "lock" and that client then executes the Clear.

However, this will work only if no FLUSH is performed wile the clients try to Add the item (because the FLUSH would remove that key, too...)

To synchronize that action, you could either wait a few seconds before you clear the cache. Or, you use another Redis database for that "lock" mechanism.

Another option could be using CacheManager's backplane. A clear would fire an event which gets propagated to all connected clients. You could use that to test if another client just cleared the cache. However, that's also not 100% reliable because of the delay and network latency.