Best way to save users search queries in ElasticSearch?

Question

I am developing application that uses ElasticSearch and SQL database. So, what is the best way to save users search queries and show statistics like what are the most popular requests? The simply way is to save in SQL database and count. But maybe there are some technique built in ElasticSearch?

Answer 1

You could do this by creating a second index in your ES cluster. When a user submits a search through your application you perform two steps.

1. Submit the search as a query to Elasticsearch for normal search behavior.
2. Submit an index request to the cluster with the search terms the user supplied.

With a second index of all search terms that have been submitted you can do a number of neat things. For your case, you can have a 'count' field just like in SQL that you increment as more people search for that term. Another great use case is a google style recommended terms. Your UI can submit a search request with the entered text on each key press and populate a drop down with hits from the previously searched terms. You can even personalize this by adding a user field and filtering out results not from that particular user.

The thing to keep in mind is that ElasticSearch can be used as both a primary and secondary data store. I always suggest that you only keep data you are willing to lose (like search history) as primary data though. Keep your system critical data in a more traditional data store like SQL, that way it is easy to back up and restore if anything ever goes wrong!