Suppress Log4j Output from org.apache.zookeeper.ZooKeeper

Question

I am building a test driver that will utilize org.apache.zookeeper.ZooKeeper. The library is dumping noisy output to the console that I would like to control or suppress.

I have attempted to create a custom logj2.properties file and a custom log4j2.xml file. I have tried passing several runtime properties to override the log4j configuration, but I have not been able suppress this output.

I think the logger is created here.

Run Zookeeper locally: docker-compose.yml

version: '3.9'
services:
  zoo:
    container_name: zoo
    image: zookeeper:3.8.3
    ports:
    - published: 8084
      target: 2181
    restart: unless-stopped
    stdin_open: true
    tty: true

Pom File to pull in ZooKeeper: pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.my</groupId>
    <artifactId>MyApp</artifactId>
    <version>1.0</version>
    <packaging>jar</packaging>

    <name>Test</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
    <dependency>
      <groupId>org.apache.zookeeper</groupId>
      <artifactId>zookeeper</artifactId>
      <version>3.8.3</version>
    </dependency>
  </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.2</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>2.4</version>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <mainClass>org.my.Test</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-dependency-plugin</artifactId>
                <executions>
                    <execution>
                        <phase>install</phase>
                        <goals>
                            <goal>copy-dependencies</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.2</version>
        <configuration>
            <!-- Includes Test* and *Test by default -->
            <!-- skips surefire tests without skipping failsafe tests.
             Property
            value seems to magically default to false -->
            <skipTests>${skip.surefire.tests}</skipTests>
        </configuration>
      </plugin>
    </plugins>
        <directory>target</directory>
        <outputDirectory>target/classes</outputDirectory>
        <finalName>${project.artifactId}-${project.version}</finalName>
        <sourceDirectory>src/main/java</sourceDirectory>
    </build>
</project>

Test Driver: src/main/java/org/my/Test.java

package org.my;

import java.io.IOException;

import org.apache.zookeeper.ZooKeeper;

public class Test {
  public static void main(String[] argv) throws IOException {
    System.out.println("hello");
    ZooKeeper zk = new ZooKeeper("localhost:8084", 100, null);
  }
}

Build Command

mvn clean install

Run Command

java -jar target/MyApp-1.0.jar 

Output (partially redacted)

hello
17:26:50.713 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.8.3-6ad6d364c7c0bcf0de452d54ebefa3058098ab56, built on 2023-10-05 10:34 UTC
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=...
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=11.0.22
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Red Hat, Inc.
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-11-openjdk-11.0.22.0.7-1.amzn2.0.1.x86_64
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=target/MyApp-1.0.jar
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.14.336-257.562.amzn2.x86_64
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=...
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=...
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=...
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=244MB
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=3962MB
17:26:50.716 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=250MB
17:26:50.721 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:8084 sessionTimeout=100 watcher=null
17:26:50.728 [main] INFO org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
17:26:50.734 [main] INFO org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 1048575 Bytes
17:26:50.745 [main] INFO org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false

Answer 1

Looking more closely at the ZooKeeper.java class, I see that the code is using SLF4J.

import org.slf4j.LoggerFactory;

Looking at the jar files created in the build, I see that logback is included.

bash-4.2$ ls -1 target
audience-annotations-0.12.0.jar
classes
commons-io-2.11.0.jar
generated-sources
logback-classic-1.2.10.jar
logback-core-1.2.10.jar
maven-archiver
MyApp-1.0.jar
netty-buffer-4.1.94.Final.jar
netty-codec-4.1.94.Final.jar
netty-common-4.1.94.Final.jar
netty-handler-4.1.94.Final.jar
netty-resolver-4.1.94.Final.jar
netty-transport-4.1.94.Final.jar
netty-transport-classes-epoll-4.1.94.Final.jar
netty-transport-native-epoll-4.1.94.Final.jar
netty-transport-native-unix-common-4.1.94.Final.jar
slf4j-api-1.7.30.jar
zookeeper-3.8.3.jar
zookeeper-jute-3.8.3.jar

This led me to create src/main/resources/logback.xml to override the logging level.

<configuration>
  <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>

  <root level="error">
    <appender-ref ref="STDOUT" />
  </root>
</configuration>

Once this file was put into place, I see the output that I hoped to see.

bash-4.2$ java -jar target/MyApp-1.0.jar 
hello

Answer 2

Zookeeper is both a complete Java application and a library:

• as an application it is distributed with Logback as logging backend,
• as a library it should leave the choice of logging backend to application developer.

Due to ZOOKEEPER-4763 however logback-classic is declared as compile-time dependency of the org.apache.zookeeper:zookeeper artifact.

To workaround this bug you should:

1. Exclude logback-classic and logback-core from the zookeeper dependency:

   <dependency>
     <groupId>org.apache.zookeeper</groupId>
     <artifactId>zookeeper</artifactId>
     <version>3.8.3</version>
     <exclusions>
       <exclusion>
         <groupId>ch.qos.logback</groupId>
         <artifactId>logback-classic</artifactId>
       </exclusion>
       <exclusion>
         <groupId>ch.qos.logback</groupId>
         <artifactId>logback-core</artifactId>
       </exclusion>
     </exclusions>
   </dependency>
   

   to give the choice of logging backend back to your application.

2. Add explicitly a logging backend to your application. You have a choice between:

   • Using Logback and a logback.xml file as in your answer. For this you'll need these dependecies:

     <dependency>
       <groupId>ch.qos.logback</groupId>
       <artifactId>logback-classic</artifactId>
       <version>1.3.14</version>
       <scope>runtime</scope>
     </dependency>
     <!-- Optionally if other libraries use the Log4j API -->
     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
       <artifactId>log4j-to-slf4j</artifactId>
       <version>2.23.1</version>
       <scope>runtime</scope>
     </dependency>
     

   • Using Log4j Core and a log4j2.xml file as in your question. For this you'll need these dependecies:

     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
       <artifactId>log4j-slf4j2-impl</artifactId>
       <version>2.23.1</version>
       <scope>runtime</scope>
     </dependency>
     <dependency>
       <groupId>org.apache.logging.log4j</groupId>
       <artifactId>log4j-core</artifactId>
       <version>2.23.1</version>
       <scope>runtime</scope>
     </dependency>