Last week we got an production issue on ESB Camel JMS, we have deployed multiple micro services on FUSE server. Below exception happened in one installed service caused all the other services stopped producing AMQ messages.
04:34:30,584 | INFO | 4.201:9003@36050 | PooledConnectionFactory | 219 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630187 | Expiring connection ActiveMQConnection {id=ID:jxf2pap1-38030-1541147693149-21:138801,clientId=ID:jxf2pap1-38030-1541147693149-20:138800,started=false} on IOException: Unexpected error occurred: java.lang.IllegalStateException: Timer already cancelled.
04:34:30,585 | ERROR | [ ] | DefaultErrorHandler | 232 - org.apache.camel.camel-core - 2.17.0.redhat-630187 | Failed delivery for (MessageId: ID-jxf2pap1-37890-1541147693904-3-2612818 on ExchangeId: ID-jxf2pap1-37890-1541147693904-3-2612816). Exhausted after delivery attempt: 1 caught: org.springframework.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exceptionn | n | Message Historyn | ---------------------------------------------------------------------------------------------------------------------------------------n | RouteId ProcessorId Processor Elapsed (ms)n | [start_ITV ] [start_ITV ] [activemq://T2_ESB_ITV_INQ ] [ 19]n | [start_ITV ] [process75 ] [ref:eventProcessor ] [ 0]n | [start_ITV ] [dynamicRouter36 ] [dynamicRouter[bean[ref:processorSender method:sendToEndpoint]] ] [ 20]n | n | Stacktracen | ---------------------------------------------------------------------------------------------------------------------------------------
org.springframework.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception; nested exception is javax.jms.IllegalStateException: javax.jms.JMSException: Disposed due to prior exception
Only JMS producer failed, consumers are active and consumed messages.
Please advice on this issue, thanks.
What version of activemq you have? Prior to 5.10, method executePeriodically() wasn't syncrhonized, but modifies timer tasks map. I think they have settled it after.