Spring Cloud Stream Kinesis binder error in child process

Question

I would like to use Spring Cloud Stream kinesis binder with KPL/KCL enabled. However, when I enabled that by using kpl-kcl-enabled: true the following error keeps coming up:

com.amazonaws.services.kinesis.producer.IrrecoverableError: Error starting child process at 

com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:537) at 
com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:468) at 
com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63) at 
com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at 
java.lang.Thread.run(Thread.java:748) 
Caused by: java.io.IOException: Cannot run program "/tmp/amazon-kinesis-producer-native-

binaries/kinesis_producer_685427917724EC847D7D65F261E7040F3FCCB039": error=2, No such file or directory 
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at
com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:466) ... 5 common frames omitted Caused by: java.io.IOException: error=2, No such file or directory at
java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at
java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 6 common frames omitted

After quite a few attempts to restart it it throws out of memory exception:

Exception in thread "kpl-daemon-0000" java.lang.OutOfMemoryError: Java heap space
    at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
    at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
    at com.amazonaws.services.kinesis.producer.Daemon.<init>(Daemon.java:95)
    at com.amazonaws.services.kinesis.producer.KinesisProducer$MessageHandler.onError(KinesisProducer.java:168)
    at com.amazonaws.services.kinesis.producer.Daemon.fatalError(Daemon.java:537)
    at com.amazonaws.services.kinesis.producer.Daemon.startChildProcess(Daemon.java:468)
    at com.amazonaws.services.kinesis.producer.Daemon.access$100(Daemon.java:63)
    at com.amazonaws.services.kinesis.producer.Daemon$1.run(Daemon.java:133)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Answer 1

KPL expects glibc in version 2.5 or higher to be available in your Linux version. openjdk:8-jdk-alpine docker image does not provide that.

You need to use a different docker image, for example: openjdk:8-jdk-slim to have jdk and glibc already installed, or frolvlad/alpine-glibc for alpine image with glibc.