Can't use TDLight (TDLib fork) in Java

Question

please help. 2 days wasted for this problem :(

I'm creating Telegram bot in Java using https://github.com/tdlight-team/tdlight.
I created Gradle project, built jar, and now trying to run it in Docker image FROM openjdk:17-alpine

Some important things from build.gradle:

buildscript {
    apply from: 'versions.gradle'
    repositories {
        mavenCentral()
        maven { url "https://plugins.gradle.org/m2/" }
        maven { url "https://mvn.mchv.eu/repository/mchv/" }
        jcenter()
    }
    dependencies {
        classpath("org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version")
        classpath("org.jetbrains.kotlin:kotlin-stdlib-common:$kotlin_version")
    }
}
plugins {
    id 'org.jetbrains.kotlin.jvm' version "$kotlin_version"
    id 'org.jetbrains.kotlin.plugin.serialization' version "$kotlin_version"
    id "com.github.johnrengelman.shadow" version '7.1.2'
}
dependencies {
//    implementation "org.telegram:telegrambots:${versions.telegram}"
    implementation platform("it.tdlight:tdlight-java-bom:${versions.tdLight}")
    implementation group: 'it.tdlight', name: 'tdlight-java', classifier: 'jdk8' // Java 8 is supported if you use the following dependency classifier: `jdk8`
//    implementation group: 'it.tdlight', name: 'tdlight-natives', classifier: 'linux_amd64_gcc_ssl1'
//    implementation group: 'it.tdlight', name: 'tdlight-natives', classifier: 'linux_amd64_gcc_ssl3'
    implementation group: 'it.tdlight', name: 'tdlight-natives', classifier: 'windows_amd64'
    implementation group: 'it.tdlight', name: 'tdlight-natives', classifier: 'linux_amd64_ssl1'
    implementation group: 'it.tdlight', name: 'tdlight-natives', classifier: 'linux_amd64_ssl3'

    implementation "com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:${versions.jackson}"
    implementation "com.fasterxml.jackson.core:jackson-databind:${versions.jackson}"
    implementation "com.sksamuel.hoplite:hoplite-core:${versions.hoplite}" //to read properties
    implementation "com.sksamuel.hoplite:hoplite-yaml:${versions.hoplite}" //to read yaml-properties

//...

kotlin {
    jvmToolchain(11)
}

compileKotlin {
    kotlinOptions.jvmTarget = '11'
}

compileTestKotlin {
    kotlinOptions.jvmTarget = '11'
}

jar {
    manifest {
        attributes 'Main-Class': 'name.nepavel.otus.watchbot.MainKt'
    }
}

shadowJar {
    archiveVersion.set(project.version)
}

Almost everything is exactly as in TDLight readme.md. Then I connected to Docker container's shell and run:
docker run -v D:\:/d -it --entrypoint /bin/sh --name open17 norg/openjdk-17

/ # java -version
openjdk version "17-ea" 2021-09-14
OpenJDK Runtime Environment (build 17-ea+14)
OpenJDK 64-Bit Server VM (build 17-ea+14, mixed mode, sharing)

java -jar /d/work/otus/repos/watchbot/build/libs/watchbot-1.0-SNAPSHOT-all.jar
And got a long stacktrace:

Exception in thread "main" it.tdlight.util.UnsupportedNativeLibraryException: Failed to load TDLight native libraries
        at it.tdlight.util.Native.loadLibrary(Native.java:65)
        at it.tdlight.util.Native.loadNativesInternal(Native.java:37)
        at it.tdlight.Init.init(Init.java:53)
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt:43)
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt)
Caused by: java.lang.IllegalArgumentException: Failed to load any of the given libraries: [tdjni.linux_amd64_ssl1, tdjni.linux_amd64_ssl3, tdjni.linux_amd64_gcc_ssl1, tdjni.linux_amd64_gcc_ssl3]
        at it.tdlight.util.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:132)
        at it.tdlight.util.Native.loadLibrary(Native.java:55)
        ... 4 more
        Suppressed: java.lang.UnsatisfiedLinkError: /tmp/tdlight-java-natives15557113517622973995/libtdjni.linux_amd64_ssl16069453053884461735.so: Error loading shared library libc++.so.1: No such file or directory (needed by /tmp/tdlight-java-natives15557113517622973995/libtdjni.linux_amd64_ssl16069453053884461735.so)
                at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
                at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:383)
                at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:227)
                at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:169)
                at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2383)
                at java.base/java.lang.Runtime.load0(Runtime.java:746)
                at java.base/java.lang.System.load(System.java:1857)
                at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:21)
                at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:439)
                at it.tdlight.util.NativeLibraryLoader.load(NativeLibraryLoader.java:236)
                at it.tdlight.util.NativeLibraryLoader.loadFirstAvailable(NativeLibraryLoader.java:123)
                ... 5 more
                Suppressed: java.lang.UnsatisfiedLinkError: /tmp/tdlight-java-natives15557113517622973995/libtdjni.linux_amd64_ssl16069453053884461735.so: Error loading shared library libc++.so.1: No such file or directory (needed by /tmp/tdlight-java-natives15557113517622973995/libtdjni.linux_amd64_ssl16069453053884461735.so)
                        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
                        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:383)
                        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:227)
                        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:169)
                        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2383)
                        at java.base/java.lang.Runtime.load0(Runtime.java:746)
                        at java.base/java.lang.System.load(System.java:1857)
                        at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:21)
                        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
                        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
                        at it.tdlight.util.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:468)
                        at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:431)
                        ... 7 more
                Suppressed: java.lang.UnsatisfiedLinkError: no tdjni.linux_amd64_ssl1 in java.library.path: /usr/lib:/usr/lib/bin
                        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2423)
                        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:808)
                        at java.base/java.lang.System.loadLibrary(System.java:1893)
                        at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:23)
                        at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:439)
                        at it.tdlight.util.NativeLibraryLoader.load(NativeLibraryLoader.java:179)
                        ... 6 more
                        Suppressed: java.lang.UnsatisfiedLinkError: no tdjni.linux_amd64_ssl1 in java.library.path: /usr/lib:/usr/lib/bin
                                at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2423)
                                at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:808)
                                at java.base/java.lang.System.loadLibrary(System.java:1893)
                                at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:23)
                                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
                                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                                at java.base/java.lang.reflect.Method.invoke(Method.java:568)
                                at it.tdlight.util.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:468)
                                at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:431)
                                ... 7 more

... ... ...

Suppressed: java.lang.UnsatisfiedLinkError: could not load a native library: tdjni
                at it.tdlight.util.NativeLibraryLoader.load(NativeLibraryLoader.java:257)
                at it.tdlight.util.Native.loadLibrary(Native.java:58)
                ... 4 more
        Caused by: java.io.FileNotFoundException: META-INF/tdlightjni/libtdjni.so
                at it.tdlight.util.NativeLibraryLoader.load(NativeLibraryLoader.java:204)
                ... 5 more
                Suppressed: java.lang.UnsatisfiedLinkError: no tdjni in java.library.path: /usr/lib:/usr/lib/bin
                        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2423)
                        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:808)
                        at java.base/java.lang.System.loadLibrary(System.java:1893)
                        at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:23)
                        at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:439)
                        at it.tdlight.util.NativeLibraryLoader.load(NativeLibraryLoader.java:179)
                        ... 5 more
                        Suppressed: java.lang.UnsatisfiedLinkError: no tdjni in java.library.path: /usr/lib:/usr/lib/bin
                                at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2423)
                                at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:808)
                                at java.base/java.lang.System.loadLibrary(System.java:1893)
                                at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:23)
                                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
                                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                                at java.base/java.lang.reflect.Method.invoke(Method.java:568)
                                at it.tdlight.util.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:468)
                                at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:431)
                                ... 6 more

This means that precompiled natives are not accessible in my jar file. Then I decided to build tdjni native inside my Docker image. I took the build commands from here: https://tdlight-team.github.io/tdlight/build.html?language=Kotlin
and created the following Dockerfile:

FROM openjdk:17-alpine
ARG VER=$VER
ENV VER=$VER
ENV JAVA_OPTS=-Xmx1g
RUN echo "Version = $VER"
RUN echo "JAVA_HOME = $JAVA_HOME"

RUN apk update && \
    apk upgrade && \
    apk add --update alpine-sdk linux-headers git zlib-dev openssl-dev gperf php cmake openjdk8 && \
    git clone https://github.com/tdlight-team/tdlight.git && \
    cd tdlight && \
    rm -rf build && \
    mkdir build && \
    cd build && \
    cmake -DCMAKE_BUILD_TYPE=Release -DJAVA_HOME=/usr/lib/jvm/java-1.8-openjdk/ -DCMAKE_INSTALL_PREFIX:PATH=../example/java/td -DTD_ENABLE_JNI=ON .. && \
    cmake --build . --target install && \
    cd .. && \
    cd example/java && \
    rm -rf build && \
    mkdir build && \
    cd build && \
    cmake -DCMAKE_BUILD_TYPE=Release -DJAVA_HOME=/usr/lib/jvm/java-1.8-openjdk/ -DCMAKE_INSTALL_PREFIX:PATH=/usr/local -DTd_DIR:PATH=$(readlink -f ../td/lib/cmake/Td) .. && \
    cmake --build . --target install && \
    cd ../../.. && \
    cd .. && \
    ls -l /usr/local

CMD ["/bin/sh"]

But result remained the same.
Then I specified the path to my built tdjni when starting jar inside new container:
java -Djava.library.path=/usr/local:/usr/local/bin -jar /d/work/otus/repos/watchbot/build/libs/watchbot-1.0-SNAPSHOT-all.jar
Library was found this time:

[ 1][t 0][1688555466.113252401][tl_jni_object.cpp:39]   Can't find class [org/drinkless/tdlib/Client]
FATAL ERROR in native method: Can't find class [org/drinkless/tdlib/Client]
        at jdk.internal.loader.NativeLibraries.load(java.base@17-ea/Native Method)
        at jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(java.base@17-ea/NativeLibraries.java:383)
        at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17-ea/NativeLibraries.java:227)
        - locked <0x0000000083d6d970> (a java.util.HashSet)
        at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17-ea/NativeLibraries.java:169)
        at jdk.internal.loader.NativeLibraries.findFromPaths(java.base@17-ea/NativeLibraries.java:310)
        at jdk.internal.loader.NativeLibraries.loadLibrary(java.base@17-ea/NativeLibraries.java:282)
        at java.lang.ClassLoader.loadLibrary(java.base@17-ea/ClassLoader.java:2416)
        at java.lang.Runtime.loadLibrary0(java.base@17-ea/Runtime.java:808)
        at java.lang.System.loadLibrary(java.base@17-ea/System.java:1893)
        at it.tdlight.util.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:23)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@17-ea/Native Method)
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@17-ea/NativeMethodAccessorImpl.java:78)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17-ea/DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(java.base@17-ea/Method.java:568)
        at it.tdlight.util.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:468)
        at it.tdlight.util.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:431)
        at it.tdlight.util.NativeLibraryLoader.load(NativeLibraryLoader.java:179)
        at it.tdlight.util.Native.loadLibrary(Native.java:58)
        at it.tdlight.util.Native.loadNativesInternal(Native.java:37)
        at it.tdlight.Init.init(Init.java:53)
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt:43)
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt)
Aborted

Why it requires org/drinkless/tdlib/Client while I'm using TDLight, which should contain TDLib API? I decided to "show him" where the TDLib compiled Java classes are located (they were created by build commands):
java -Djava.library.path=/usr/local:/usr/local/bin -cp /d/work/otus/repos/watchbot/build/libs/watchbot-1.0-SNAPSHOT-all.jar:/usr/local/bin name.nepavel.otus.watchbot.MainKt
and got:

11:12:21.898 [main] ERROR it.tdlight.TDLight - Can't set verbosity level on startup
java.lang.UnsatisfiedLinkError: 'it.tdlight.jni.TdApi$Object it.tdlight.tdnative.NativeClient.nativeClientExecute(it.tdlight.jni.TdApi$Function)'
        at it.tdlight.tdnative.NativeClient.nativeClientExecute(Native Method) ~[watchbot-1.0-SNAPSHOT-all.jar:?]
        at it.tdlight.NativeClientAccess.execute(NativeClientAccess.java:14) ~[watchbot-1.0-SNAPSHOT-all.jar:?]
        at it.tdlight.Init.init(Init.java:56) [watchbot-1.0-SNAPSHOT-all.jar:?]
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt:43) [watchbot-1.0-SNAPSHOT-all.jar:?]
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt) [watchbot-1.0-SNAPSHOT-all.jar:?]
Exception in thread "main" java.lang.UnsatisfiedLinkError: 'int it.tdlight.tdnative.NativeClient.createNativeClient()'
        at it.tdlight.tdnative.NativeClient.createNativeClient(Native Method)
        at it.tdlight.NativeClientAccess.create(NativeClientAccess.java:10)
        at it.tdlight.InternalClient.createAndRegisterClient(InternalClient.java:160)
        at it.tdlight.InternalClient.initialize(InternalClient.java:149)
        at it.tdlight.AutoCleaningTelegramClient.initialize(AutoCleaningTelegramClient.java:46)
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt:48)
        at name.nepavel.otus.watchbot.MainKt.main(Main.kt)

Now it looks like TDLight expects another version of TDLib native (as far as I can see). What am I doing wrong? How to make it work?

I have tried different base images, different JAVA_HOME, different JDK from 11 to 17, different TDLight dependencies in build.gradle, but nothing changed. I suppose that the mistake is small and stupid, but I can't find it by myself.

Answer 1

The first stacktrace says that you don't have libc++ installed: libc++.so.1: No such file or directory

You are using the following classifiers that require libc++: linux_amd64_ssl1 and linux_amd64_ssl3.

If you don't have libc++ and you don't want to install it, you should choose the following classifiers instead: linux_amd64_gcc_ssl1 and linux_amd64_gcc_ssl3

Answer 2

Faced the same problem. The reason may be that your openjdk:17-alpine image does not have the required openssl version. Try updating openssl like here. It helped me and this guy.

FROM  openjdk:17-jdk-slim-bullseye
#Perl is required to install openssl
RUN apt-get update \
    && apt-get install -y ca-certificates wget bash \
    && apt-get -qy install perl

#Remove current openssl
RUN apt-get -y remove openssl

#This is required to run “tar” command
RUN apt-get -qy install gcc

RUN apt-get -q update && apt-get -qy install wget make && \
    wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz &&  \
    tar -zxf openssl-1.1.1g.tar.gz && cd openssl-1.1.1g && \
    ./config && \
    make install

WORKDIR /app

ADD build/libs/*.jar /app/libs/
ADD build/resources/* /app/resources/

ENTRYPOINT ["sh", "-c", "java ${JAVA_OPTS} -jar /app/libs/example.jar"]