Producer with name is already connected to topic when using Pulsar appender with Ignite logger

Question

I have a complex project that uses Ignite and has the Ignite version of log4j2 installed.

I'd also like to install a Pulsar appender to log4j2, however, when I start up the application, I get the following error:

log4j:WARN No appenders could be found for logger (org.apache.pulsar.shade.io.netty.util.internal.logging.InternalLoggerFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2020-07-28 15:58:47,732 main ERROR Failed to start pulsar manager org.apache.pulsar.client.api.PulsarClientException$ProducerBusyException: Producer with name 'pulsar-log4j2-appender-json_persistor4' is already connected to topic
    at org.apache.pulsar.client.api.PulsarClientException.unwrap(PulsarClientException.java:849)
    at org.apache.pulsar.client.impl.ProducerBuilderImpl.create(ProducerBuilderImpl.java:93)
    at org.apache.pulsar.log4j2.appender.PulsarManager.startup(PulsarManager.java:127)
    at org.apache.pulsar.log4j2.appender.PulsarAppender.start(PulsarAppender.java:187)
    at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:266)
    at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:548)
    at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:620)
    at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:637)
    at org.apache.ignite.logger.log4j2.Log4J2Logger$5.apply(Log4J2Logger.java:405)
    at org.apache.ignite.logger.log4j2.Log4J2Logger$5.apply(Log4J2Logger.java:402)
    at org.apache.ignite.logger.log4j2.Log4J2Logger.addConsoleAppenderIfNeeded(Log4J2Logger.java:302)
    at org.apache.ignite.logger.log4j2.Log4J2Logger.setNodeId(Log4J2Logger.java:402)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.initLogger(IgnitionEx.java:2568)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.initializeConfiguration(IgnitionEx.java:2206)
    at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1697)
    at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1117)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:637)
    at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:563)
    at org.apache.ignite.Ignition.start(Ignition.java:321)
    at com.coinflex.service.release.ReleaseApplication.main(ReleaseApplication.java:39)

Clearly, there is a clash between the two dependencies. I have excluded most when I imported Pulsar appender to my pom.xml however:

 <dependencies>
    <dependency>
        <groupId>org.apache.ignite</groupId>
        <artifactId>ignite-spring</artifactId>
        <version>${ignite.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.ignite</groupId>
        <artifactId>ignite-zookeeper</artifactId>
        <version>${ignite.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.ignite</groupId>
        <artifactId>ignite-urideploy</artifactId>
        <version>${ignite.version}</version>
    </dependency>
    <dependency>
        <groupId>org.apache.ignite</groupId>
        <artifactId>ignite-log4j2</artifactId>
        <version>${ignite.version}</version>
    </dependency>
    <dependency>
        <groupId>com.google.code.gson</groupId>
        <artifactId>gson</artifactId>
        <version>2.8.6</version>
    </dependency>
    <dependency>
        <groupId>org.apache.pulsar</groupId>
        <artifactId>pulsar-client</artifactId>
        <version>${pulsar.version}</version>
    </dependency>
    <dependency>
        <groupId>org.java-websocket</groupId>
        <artifactId>Java-WebSocket</artifactId>
        <version>1.3.8</version>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>spring-web</artifactId>
    </dependency>

    <!-- Pulsar logging -->
    <dependency>
        <groupId>org.apache.pulsar</groupId>
        <artifactId>pulsar-log4j2-appender</artifactId>
        <version>2.6.0</version>
        <exclusions>
            <exclusion>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-slf4j-impl</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.apache.pulsar</groupId>
                <artifactId>pulsar-client</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-api</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.apache.logging.log4j</groupId>
                <artifactId>log4j-core</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
</dependencies>

And this is what my log4j2.xml looks like:

<Configuration monitorInterval="60">
    <Appenders>
        <Console name="CONSOLE" target="SYSTEM_OUT">
            <PatternLayout pattern="%d{hh:mm:ss,SSS} [%t] %-5p %c %x - %m%n"/>
            <ThresholdFilter level="ERROR" onMatch="DENY" onMismatch="ACCEPT"/>
        </Console>

        <Console name="CONSOLE_ERR" target="SYSTEM_ERR">
            <PatternLayout pattern="[%d{ISO8601}][%-5p][%t][%c{1}]%notEmpty{[%markerSimpleName]} %m%n"/>
        </Console>

        <Routing name="FILE">
            <Routes pattern="$${sys:nodeId}">
                <Route>
                    <RollingFile name="Rolling-${sys:nodeId}" fileName="ignite/work/log/ignite-${sys:nodeId}.log"
                                 filePattern="ignite/work/log/ignite-${sys:nodeId}-%i-%d{yyyy-MM-dd}.log.gz">
                        <PatternLayout pattern="[%d{ISO8601}][%-5p][%t][%c{1}]%notEmpty{[%markerSimpleName]} %m%n"/>
                        <Policies>
                            <TimeBasedTriggeringPolicy interval="6" modulate="true" />
                            <SizeBasedTriggeringPolicy size="10 MB" />
                        </Policies>
                    </RollingFile>
                </Route>
            </Routes>
        </Routing>
        <Pulsar name="PULSAR" serviceUrl="pulsar://172.21.11.82:6650" topic="json_persistor"  avoidRecursive="false">
            <PatternLayout pattern="%msg%n"/>
        </Pulsar>
    </Appenders>

    <Loggers>
     
        <Logger name="org.apache.ignite" level="ERROR"/>
        <Logger name="org.springframework" level="WARN"/>
        <Logger name="org.eclipse.jetty" level="WARN"/>
        <Logger name="org.eclipse.jetty.util.log" level="ERROR"/>
        <Logger name="org.eclipse.jetty.util.component" level="ERROR"/>
        <Logger name="com.amazonaws" level="WARN"/>

        <Root level="INFO">
            <AppenderRef ref="CONSOLE" level="INFO"/>
            <AppenderRef ref="CONSOLE_ERR" level="ERROR"/>
            <AppenderRef ref="FILE" level="DEBUG"/>
        </Root>
    </Loggers>
</Configuration>

Answer 1

The issue isn't with the log4j dependencies, but rather with a name conflict with the producer that is trying to connect to the Pulsar topic. The stack trace says "Producer with name 'pulsar-log4j2-appender-json_persistor4' is already connected to topic", which I assume is the name your application uses for one of the producers.

Therefore, you simply need to stop the previously running instance of the application in order to terminate the existing producer so that the new one can connect.

An long-term solution is to avoid explicitly naming your producers and allow Pulsar to handle that for you. Then the names are guaranteed to be unique. See Is it possible to have multiple producers for the same topic on Pulsar? for more details