Cassandra -Out of memory: Killed process 20707 (java) total-vm:4393466140kB

592 Views Asked by At

We have Cassandra only in standalone (single Node) database which is serving writing approx 7800 data-points per sec.

However, from from last few days we are facing issue that Cassandra is getting stop frequently.

Even though we have enough resources available.

We have observe the same error coming again and again.

Our server configuration is 16 core CPU and 62 RAM (However there are other service also running).

Dec  5 16:31:50 localhost kernel: [875863.395340] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/cassandra.service,task=java,pid=20707,uid=114
Dec  5 16:31:50 localhost kernel: [875863.395843] Out of memory: Killed process 20707 (java) total-vm:4393466140kB, anon-rss:13708728kB, file-rss:130260kB, shmem-rss:0kB, UID:114 pgtables:311304kB oom_score_adj:0

Also node our database is working on default configuration and Cassandra Version is 4.0.3.

find attached logs for your reference

#Cassandra#optimization#OOM

As per error its seem like that there is memory issue so we have remove all our extra service from that server and put all in another server of 4 core CPU and 16 GB RAM.

However, after removing external services we are still facing same issue.

1

There are 1 best solutions below

0
On

Without a heap dump it's hard to say for sure what the issue is, but I've personally hit several memory leaks in Cassandra 4, I'd recommend upgrading to the latest 4.0.7 release, it seems to be far more stable and there are multiple leaks fixed between your version and the latest. The most recently fixed that has helped is: https://issues.apache.org/jira/browse/CASSANDRA-16681

You can grab the latest version here: https://cassandra.apache.org/_/download.html