Im relatively new to activemq and one of the first things im trying to do is publish from a server process to 5,000 topics ( one topic per stock ). The server and broker manage to keep up fine.
However on the consumer side its very odd. If I subscribe to all 5k topics with a single wildcard consumer ( "mytopic.>" ) everything keeps up fine. However if I try to subscribe with a single consumer per topic the performance just drops out and it cant keep up.
Ive tried playing with prefetch limits and optimized ack modes, nothing seems to help.
Any idea why a single wildcard would be able to perform fine where as 5k individual topics would not?
I could just as well demultiplex the msgs myself but would expect activemq to be able to do this for me as efficiently as I can.
EDIT: Some more information and updates on this:
- I was testing this on ~ 6,000 topics publishing once per second
- I'm using the activemq-cpp c++ library, and I was creating 1 session for all topics. It turns out the activemq implementation is horribly inefficient, it does a linear scan of all topics on every messages ( twice actually ) when delivering a message to a session.
- To make matters worse, if you create a session per topic it tries to create a thread per session so that blows out pretty quick.
- But wait! There's an option on the connection, setAlwaysSessionAsync, so sessions dont create their own threads, great!
- D'OH! not so fast, sessions still create some some RW mutex in non-async mode, and they use some home-grown TLS data which had a hardcoded limit of ~300 instances per thread... ugh
- Ok, so I had to limit number of sessions i can create to ~ 150 ( I guess other objects are using TLS data as well ) and then round-robin my topics on these...
- It would be nice if I can control how many threads I can have processing the data off the wire, but alas thats not exposed either.... ugh, hardcoded in activemq-cpp code
TLDR; activemq is a streaming pile of messy poo