We are using ES 8.3.3 with local ssd based VMs in Oracle Kubernetes engine. stack infra is as following: master: 3 nodes, 5cpu, 16gb data: : 6 nodes, 10 cpu, 60gb, heap - 30gb, 5TB ssd each coordinating: 3 nodes, 10 cpu, 60gb
We have a particular index with 200-300 gb, 6 shards, 1 replica, 10M docs. this index has a field over which unique count aggregation is being done, and its taking time around 22-25 seconds for last 24hrs and returning 1M unique results. but while we do the same query for more than 24hrs it times out.
Please suggest next steps to be taken.
we have followed everything said here:
https://www.elastic.co/blog/improving-the-performance-of-high-cardinality-terms-aggregations-in-elasticsearch
https://www.elastic.co/guide/en/elasticsearch/reference/current/system-config.html