I am getting out of memory issues while running docker container after a certain amount of time. I tried to investigate using jcmd 305 VM.native_memory
to see what part of memory is causing this. It gives the following result,
Native Memory tracking:
Total: reserved=5888920KB, committed=1545948KB
Java Heap (reserved=4091904KB, committed=988160KB)
(mmap: reserved=4091904KB, committed=988160KB)
Class (reserved=1138672KB, committed=101896KB)
(classes #16457)
(malloc=6128KB #40571)
(mmap: reserved=1132544KB, committed=95768KB)
Thread (reserved=131925KB, committed=131925KB)
(thread #94)
(stack: reserved=131444KB, committed=131444KB)
(malloc=307KB #469)
(arena=174KB #185)
Code (reserved=261762KB, committed=70338KB)
(malloc=12162KB #16615)
(mmap: reserved=249600KB, committed=58176KB)
GC (reserved=152978KB, committed=141950KB)
(malloc=3474KB #385)
(mmap: reserved=149504KB, committed=138476KB)
Compiler (reserved=474KB, committed=474KB)
(malloc=344KB #1431)
(arena=131KB #5)
Internal (reserved=83136KB, committed=83136KB)
(malloc=83104KB #24954)
(mmap: reserved=32KB, committed=32KB)
Symbol (reserved=21183KB, committed=21183KB)
(malloc=18426KB #177513)
(arena=2757KB #1)
Native Memory Tracking (reserved=4118KB, committed=4118KB)
(malloc=16KB #192)
(tracking overhead=4101KB)
Arena Chunk (reserved=2767KB, committed=2767KB)
(malloc=2767KB)
The above results are fine as I have allocated 4GB RAM to each container and committed memory is committed=1545948KB or 1.5GB
But when I run ps -o pid,user,vsz,rss,comm,args
it gives me below result,
PID USER VSZ RSS COMMAND COMMAND
305 root 13g 3.8g java java -XX:+UseContainerSupport -XX:+PrintFlagsFinal -XX:+PrintCommandLineFlags -Djava.security.egd=file:/dev/./urandom -XX:+ExitOnOutOfMemoryError -XX:OnOutOfMemoryError=/var/tmp/java/cleanup_old_
Here we can see the process (PID 305) is taking 3.8g of RSS memory. Why this difference is there? And what should I investigate/fix to see where this memory wastage/difference is coming from?