We're using SphinxSearch (not really relevant likely as we're returning the resulting objects from MySQL) to return user's search results. This part is working fine. We're displaying 30 items per page, but there may be up to 20k results that match.
What we're trying to do is add the ability to filter search results based on the total search results attributes and options. Take this amazon search for instance:
https://www.amazon.ca/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=tablet
If you look at the left side, you can filter by brand, category, keywords, discount percentage, memory capacity, screen size, et al. Obviously this doesn't just apply to the currently displayed search results, but the entire result set (which in this Amazon maxes out at 400 pages).
If we were to do that, how can we avoid loading and looping through all 400*30 results to display relevant attribute/category filters? We've tried looping just to see how long that would take, and it's easily above 15 seconds. We've also tried caching common search terms (such as tablet in this case) but obviously, most user searches won't fall neatly into easily cacheable result sets.
Also, is there a name for this post search entire result set type of filtering?
Often called faceted search. Ie can filter results by facets.
Good overview... http://sphinxsearch.com/blog/2013/06/21/faceted-search-with-sphinx/
In short let sphinx calculate the list and counts, rather than doing it in post