Frequency of Full reindex on SolrCloud

Question

How often do I need to run full reindex on SolrCloud? It takes more than 12 hours for full reindex to run and we run it every night but is it really necessary to do it as delta runs correctly. New data comes in at the rate of 2000 documents on every delta per 30 seconds. Total index size : 20GB Solr: 6.5.2

Answer 1

If delta runs correctly, there should be no need to run a reindex at all. The exception might be if you do not have disabled any merging while the index is operative; in that case you might end up with a very fragmented index file wise, and the reindex ends up building a complete set as a single index file instead, but isn't usually how Solr is configured, and if it is - it's done for a reason.

So - if your delta is working correctly and you run Solr with fairly standard settings, you can safely skip reindexing unless you're starting over with an empty index (or have a situation where the schema has changed). But be sure that this also includes deletions - a reindex would probably not include deleted elements, so the question then becomes whether your delta import handles deletions as well.

None of our Solr based services reindex at all - everything is done with live updates and a decent merge factor.