Distributed Redis Search using Redis Index

Question

We are currently utlizing redis search as vector database. We have deployed redis stack cluster with 6 nodes 3 Master and 3 Slaves.Our data is divided into multiple groups for each group we have created index to do fast search, and there can be mulitiple indexes for each group of data.

Now issue is that index is being created on one node and its respective data is being moved to all the nodes. And only data that is available on index's node can be searched.

We have used {} syntax to force data to move towards single nodes as suggested by redis as following

HSET key_{abc}_doc <value> 

here all the data with abc in the key will go to same node as they will share same hash slot.

Now we want that index and its respective data should go to same node so it can be searched again. Kinldy suggest a viable solution.

Redis Search Version: 2.0+

Answer 1

When you run a cluster, the keys are smeared across the nodes. In order to search all the nodes, you need something that will:

1. Create the indices on all of the nodes.
2. Search all the nodes and aggregate the results.

Redis Cloud and Redis Enterprise take care of this for you but if you're rolling deployments, RSCoordinator is the tool you'll want to look at.

Mind you, I've not used it myself, but it was created to solve this very problem.