Recently, encountered increases in the heap memory usage in the master nodes (heap memory overflow master nodes continuous garbage collection ). I try to debug the root cause using the heap dump saved in the storage ( sample file name for reference: java_pid1.hprof ) but those files are encrypted unable find anything.
Is this the correct way to debug the heap memory issue, If yes, how to get the decrypted heap dump to get a proper info Else how to debug the heap memory issue in the master node
Elastic Search Info:
Running in Kubernetes Dedicated 3 master nodes 3 data nodes (which are also the ingest nodes)
3 data nodes - each node spec(ram 64GB memory limit 32GB) - heap size - 28GB disk size - 1TB 3 master nodes - each node spec(ram 16GB memory limit 4GB) - heap size - 4GB disk size - 10GB
Hprof files can be opened inside Eclipse. Eclipse has a special plugin to open hprof files. Its called the memory analyzer tool.
I have done these excercises in the past, but usually you find nothing much there.
Thanks.