ElasticSearch ReIndex Request not working in synchronous manner using Java Rest High Level Client

Question

I have 2 indexes with names - source_index, destination_index

Both indexes have same mappings.

I have written following code for making reindex and search requests -

@Autowired
RestHighLevelClient client;

public BulkByScrollResponse reIndexElasticResponse(ReindexRequest reindexRequest){
    BulkByScrollResponse reindexResponse = null;
    try{
      reindexResponse = client.reindex(reindexRequest,RequestOptions.DEFAULT);
    } catch (IOException e){
      throw new RuntimeException(e);
    }
    return reindexResponse;
}

public SearchResponse searchElasticResponseByScrollAPI(SearchScrollRequest searchScrollRequest) {
    SearchResponse searchResponse = null;
    try {
      searchResponse =  client.scroll(searchScrollRequest,RequestOptions.DEFAULT);
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
    return searchResponse;
}

Now my application is reindexing only selected records from source_index to destination_index. And just after reindexing request, I am running one search query on destination_index.

The problem is that my search query is not giving me correct response as I think that reindexing request is not working in synchronous manner. If I use Thread.sleep(5000); after my reindexing request then my search query gives me correct result.

reIndexElasticResponse(reindexRequest);
Thread.sleep(5000);
searchElasticResponseByScrollAPI(searchScrollRequest);

The above code will work fine. But if I comment Thread.sleep(5000) then search query doesn't give expected output.

I want my program to wait until all the records are reindexed and then run the search query on destination_index

Answer 1

This is expected behaviour because it should be take some time to refresh index after commiting index (default is 1 second). You can read answer provided by Val here which will give you more clarity why this is happening.

So when you are setting Thread.sleep(5000); in your code, elastic is able to complete index refresh so you are able to get search response.

Below code changes you can done in your code to refresh index manully.

public BulkByScrollResponse reIndexElasticResponse(ReindexRequest reindexRequest){
    BulkByScrollResponse reindexResponse = null;
    try{
      reindexRequest.setRefresh(true); <--- check this one
      reindexResponse = client.reindex(reindexRequest,RequestOptions.DEFAULT);
    } catch (IOException e){
      throw new RuntimeException(e);
    }
    return reindexResponse;
}