Pulsar client numIoThreads and numListenerThreads per consumer instance or for all consumer instances

177 Views Asked by At

Let's say I have the following pulsar config:

pulsar:
  topicX:
    client:
      config:
        numIoThreads: 3
        numListenerThreads: 15
    consumer:
      config:
        receiverQueueSize: 20
      count: 3
  topicY:
    client:
      config:
        numIoThreads: 3
        numListenerThreads: 15
    consumer:
      config:
        receiverQueueSize: 20
      count: 10

I'm a bit confused about topicX. Are there just three threads and 15 listener threads for all the consumers, or are these numbers multiplied by the consumer count? Can you clarify?

So for topicX that would mean that we have 9 io threads and 45 listener threads.

I can't seem to find this in the documentation, any reference?

2

There are 2 best solutions below

0
On BEST ANSWER

Both the ioThreads and listenerThreads are shared resources across all the producer and consumers that are created from the Pulsar Client. Thus, you would have a total of 3 io threads, and 15 listener threads regardless of how many consumers and/or producers you create from that Pulsar Client.

1
On

Just to clarify, you are not using Spring for Apache Pulsar, correct? I only mention this to be sure this is just a core Pulsar Java client configuration question (excellent question btw).

My understanding is that the io threads are the number of connections from the client to the broker and the listener threads are a pool shared across all consumers used by that client. The Javadoc is really the only place I have found that talks more in depth about it.