1 Reply Latest reply on Dec 5, 2014 9:10 AM by akhettar

PooledSession throw Exception at closing, Fabric never recovers until the the container is restarted

akhettar Dec 4, 2014 9:43 AM

I am posting this issue here and it may not be directly related to Jboss Fuse, but I am sure many of the active members in this forum are familiar with Camel, Fabric and ActiveMQ. Hence hoping I can find an answer to the problem I am facing. I have a collection of bundles which all have an activeMQ endpoint. The issue manifest when I restart activeMA server (using remote but running locally, not embedded in a Fabric container), the PooledSession throw an Exception at closing session - see logs below. Certain connections gets refreshed successfully but some not. NO matter how many times I restart AMQ server, still seeing warnings and exceptions in the log. But if I restart the bundles individually or simply restart the container or the whole fabric server, everything get back to normal.

I've come across couple of Jira tickets (https://issues.apache.org/jira/browse/AMQ-3988, https://issues.apache.org/jira/browse/AMQ-4366) depicting the same problem and stating that the issue has been fixed in version 5.9.0 of AMQ. There were some posts on StackOverflow (http://stackoverflow.com/questions/15994367/activemqs-messages-being-redelivered-pooledconnectionfactory-misconfig-suspe…) suggesting that I should set the idleTimeout to zero, but that didn't do the trick. See below details of my running environment:

DETAILS OF MY RUNNING ENVIRONMENT

OS: os x yosemite and Centos 6.

Fabric: 1.1.0.CR5

AMQ: apache-activemq-5.9.1 / apache-activemq-5.10.0

ActiveMQ blueprint set up

<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0"
   xsi:schemaLocation="
  http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
  http://www.osgi.org/xmlns/blueprint-ext/v1.1.0 https://svn.apache.org/repos/asf/aries/tags/blueprint-0.3.1/blueprint-core/src/main/resources/org/apache/aries/blueprint/ext/blueprint-ext.xsd
  http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd
  http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.1.0 http://svn.apache.org/repos/asf/aries/trunk/blueprint/blueprint-cm/src/main/resources/org/apache/aries/blueprint/compendium/cm/blueprint-cm-1.1.0.xsd">


   <!-- *********************************************************************************************************************
  ******************************************************************************************** ActiveMQ Configuration
  -->

  <!-- ActiveMQ Configuration -->

  <!-- define our activemq component -->
   <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
  <property name="connectionFactory" ref="poolConnectionFactory"/>
   <!-- define the jms consumer/producer as transacted -->
   <property name="transacted" value="true"/>
   <!-- setup the transaction manager to use -->
   <property name="transactionManager" ref="jmsTransactionManager"/>
  <property name="cacheLevelName" value="CACHE_CONSUMER" />
  </bean>

   <!-- setup spring jms TX manager -->
   <bean id="jmsTransactionManager" class="org.springframework.jms.connection.JmsTransactionManager">
  <property name="connectionFactory" ref="poolConnectionFactory"/>
  </bean>

   <!-- setup JMS connection factory -->
   <bean id="poolConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
  <property name="maxConnections" value="${activemq.maxConnections}"/>
  <property name="connectionFactory" ref="jmsConnectionFactory"/>
  <property name="idleTimeout" value="0"/>
  </bean>

  <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
  <property name="brokerURL" value="${activemq.brokerURL}"/>
  </bean>

  <bean id="required" class="org.apache.camel.spring.spi.SpringTransactionPolicy">
  <property name="transactionManager" ref="jmsTransactionManager"/>
  <property name="propagationBehaviorName" value="PROPAGATION_REQUIRED"/>
  </bean>

</blueprint>

FABRIC LOGS

javax.jms.IllegalStateException: The Session is closed

at org.apache.activemq.ActiveMQSession.checkClosed(ActiveMQSession.java:767)[199:org.apache.activemq.activemq-osgi:5.10.0]

at org.apache.activemq.ActiveMQSession.rollback(ActiveMQSession.java:588)[199:org.apache.activemq.activemq-osgi:5.10.0]

at org.apache.activemq.jms.pool.PooledSession.close(PooledSession.java:116)[199:org.apache.activemq.activemq-osgi:5.10.0]

at org.springframework.jms.connection.JmsResourceHolder.closeAll(JmsResourceHolder.java:215)[232:org.apache.servicemix.bundles.spring-jms:3.2.8.RELEASE_1]

at org.springframework.jms.connection.JmsTransactionManager.doCleanupAfterCompletion(JmsTransactionManager.java:257)[232:org.apache.servicemix.bundles.spring-jms:3.2.8.RELEASE_1]

at org.springframework.transaction.support.AbstractPlatformTransactionManager.cleanupAfterCompletion(AbstractPlatformTransactionManager.java:1009)[233:org.apache.servicemix.bundles.spring-tx:3.2.8.RELEASE_1]

at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:805)[233:org.apache.servicemix.bundles.spring-tx:3.2.8.RELEASE_1]

at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:724)[233:org.apache.servicemix.bundles.spring-tx:3.2.8.RELEASE_1]

at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)[232:org.apache.servicemix.bundles.spring-jms:3.2.8.RELEASE_1]

at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1101)[232:org.apache.servicemix.bundles.spring-jms:3.2.8.RELEASE_1]

at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1093)[232:org.apache.servicemix.bundles.spring-jms:3.2.8.RELEASE_1]

at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:990)[232:org.apache.servicemix.bundles.spring-jms:3.2.8.RELEASE_1]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_60]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_60]

at java.lang.Thread.run(Thread.java:745)[:1.7.0_60]

2014-12-04 14:16:40,948 | WARN | udit.redelivery] | faultJmsMessageListenerContainer | ? ? | 232 - org.apache.servicemix.bundles.spring-jms - 3.2.8.RELEASE_1 | Setup of JMS message listener invoker failed for destination 'urn.mb.core.audit.redelivery' - trying to recover. Cause: Could not commit JMS transaction; nested exception is javax.jms.IllegalStateException: The Session is closed