RocksDB with jemalloc or tcmalloc in KafkaStreams

76 Views Asked by At

My KafkaStreams apps run as Docker images in Kubernetes cluster and thus had to configure the pods to contain jemalloc and tcmalloc. By adding

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
# or 
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc.so.4

the standard glibc memory allocator will be overridden, but it doesn't work as expected:

  • with jemalloc the streams stall in rebalancing state and
  • with tcmalloc the override just doesn't happen.

Did anyone successfully use either of these with Kafka Streams?

0

There are 0 best solutions below