The following excerpt from The Definitive Guide provides high level details as shown below but
- what exactly is virtual memory is referring to in this task counter?
- How to interpret it? How is it related to PHYSICAL_MEMORY_BYTES?
Following is an example extract from one of the jobs. Physical is 214 GB approx. and virtual is 611 GB approx.
1.What exactly is virtual memory is referring to in this task counter?
So,while setting up hadoop cluster one is advised to have the value of
vm.swappiness =1
to achieve better performance. On linux systems, vm.swappiness is set to 60 by default. Higher the value more aggresive swapping of memory pages.https://community.hortonworks.com/articles/33522/swappiness-setting-recommendation.html
2. How to interpret it? How is it related to PHYSICAL_MEMORY_BYTES?
This is the relation between PHYSICAL_MEMORY_BYTES and VIRTUAL_MEMORY_BYTES.