What does optimize method do? Alternatives for optimize method in latest versions of lucene

Question

I am pretty new to lucene I am trying to understand the segment merging process. I came across the method optimize(which will merge all the available Lucene index segment at that instance). My exact question is, Does Optimize merges all the levels of segments & creates one complex segment? Alternatives in the latest version of Lucene( say Lucene 6.5)? Is it good to always call the optimize method after the indexing process, so that my index will always have a single segment and searches will be fast?

Answer 1

First of all, it's not needed to always merge segments to just one segment. It could be configured. In principle, idea of merging segments/optimizing index is coming from the implementation of deletes in the Lucene. Lucene do not deleting documents, but rather marking them for deletion, second, new documents are coming into new segments.

Lucene have a lot of per-segment files - like term dictionary and many others, so merging them together will reduce the heap and makes searches faster. However, usually the process of merging isn't that fast.

Overall, you need to have a balance between calling merging/optimizing every time you index new docs and not doing it all. One thing to look at is MergePolicy, which defines different types of merging, with different strategies. If you will not find any suitable for you (which I doubt), you could implement one for your needs.

As in Lucene 6.5 you could use

public void forceMerge(int maxNumSegments) of IndexWriter class