0 Replies Latest reply on Jan 4, 2016 11:46 PM by KL Chan

    Hornetq XA error on wildfly stop

    KL Chan Newbie

      Hi guys

       

      I've setup postgresql and hornetq to do 2 phrase commit with XA, so far so good. However I see this error when I stop wildfly:

      12:28:39,949 INFO  [org.jboss.as.server] (management-handler-thread - 6) WFLYSRV0211: Suspending server

      12:28:39,956 INFO  [org.jboss.as.server] (Thread-2) WFLYSRV0220: Server shutdown has been requested.

      12:28:39,970 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-2) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/CoreJMSXAConnectionFactory]

      12:28:39,978 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-7) WFLYJCA0010: Unbound data source [java:/CorePostgresXADS]

      12:28:39,979 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 83) WFLYUT0022: Unregistered web context: /platform

      12:28:39,981 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 85) WFLYUT0022: Unregistered web context: /backend

      12:28:39,983 INFO  [io.undertow.servlet] (ServerService Thread Pool -- 85) Destroying Spring FrameworkServlet 'appServlet'

      12:28:39,983 INFO  [io.undertow.servlet] (ServerService Thread Pool -- 83) Destroying Spring FrameworkServlet 'appServlet'

      12:28:39,983 INFO  [org.springframework.web.context.support.AnnotationConfigWebApplicationContext] (ServerService Thread Pool -- 85) Closing WebApplicationContext for namespace 'appServlet-servlet': startup date [Tue Jan 05 12:27:34 CST 2016]; parent: Root WebApplicationContext

      12:28:39,983 INFO  [org.springframework.web.context.support.AnnotationConfigWebApplicationContext] (ServerService Thread Pool -- 83) Closing WebApplicationContext for namespace 'appServlet-servlet': startup date [Tue Jan 05 12:27:31 CST 2016]; parent: Root WebApplicationContext

      12:28:39,988 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ServerService Thread Pool -- 83) Stopping beans in phase 2147483647

      12:28:39,988 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.PoolByCri] (ServerService Thread Pool -- 84) IJ000615: Destroying active connection in pool: HornetQConnectionDefinition (org.hornetq.ra.HornetQRAManagedConnection@219a9ae0)

      12:28:39,988 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.PoolByCri] (ServerService Thread Pool -- 84) IJ000615: Destroying active connection in pool: HornetQConnectionDefinition (org.hornetq.ra.HornetQRAManagedConnection@2f5d60a3)

      12:28:39,989 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.PoolByCri] (ServerService Thread Pool -- 84) IJ000615: Destroying active connection in pool: HornetQConnectionDefinition (org.hornetq.ra.HornetQRAManagedConnection@2b62f190)

      12:28:39,989 WARN  [org.jboss.jca.core.connectionmanager.pool.strategy.PoolByCri] (ServerService Thread Pool -- 84) IJ000615: Destroying active connection in pool: HornetQConnectionDefinition (org.hornetq.ra.HornetQRAManagedConnection@6e606a98)

      12:28:39,991 INFO  [org.springframework.context.support.DefaultLifecycleProcessor] (ServerService Thread Pool -- 85) Stopping beans in phase 2147483647

      12:28:39,994 INFO  [org.quartz.core.QuartzScheduler] (ServerService Thread Pool -- 85) Scheduler schedulerFactoryBean_$_Lenovo-PC1451968058574 paused.

      12:28:39,995 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) WFLYJCA0019: Stopped Driver service with driver-name = postgresql

      12:28:40,005 ERROR [org.hornetq.core.client] (DefaultMessageListenerContainer-1) HQ214004: XA end operation failed : HornetQNotConnectedException[errorType=NOT_CONNECTED message=HQ119010: Connection is destroyed]

        at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:301)

        at org.hornetq.core.client.impl.ClientSessionImpl.end(ClientSessionImpl.java:1549)

        at org.hornetq.ra.HornetQRAXAResource.end(HornetQRAXAResource.java:107)

        at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.end(XAResourceWrapperImpl.java:118)

        at org.jboss.jca.core.tx.jbossts.XAResourceWrapperStatImpl.end(XAResourceWrapperStatImpl.java:96)

        at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:318)

        at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2993)

        at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2972)

        at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1675)

        at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:127)

        at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:186)

        at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1282)

        at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:143)

        at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)

        at org.jboss.tm.usertx.client.ServerVMClientUserTransaction.rollback(ServerVMClientUserTransaction.java:182)

        at org.springframework.transaction.jta.JtaTransactionManager.doRollback(JtaTransactionManager.java:1048)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:853)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:830)

        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.rollbackOnException(AbstractPollingMessageListenerContainer.java:401)

        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:236)

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1150)

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1142)

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1039)

        at java.lang.Thread.run(Thread.java:745)

       

       

      12:28:40,012 WARN  [com.arjuna.ats.jta] (DefaultMessageListenerContainer-1) ARJUNA016045: attempted rollback of < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a397be0:-559607c3:568b462c:4cd, node_name=1, branch_uid=0:ffff0a397be0:-559607c3:568b462c:4cf, subordinatenodename=null, eis_name=java:/CoreJMSXAConnectionFactory > (XAResourceWrapperImpl@76a5a92[xaResource=org.hornetq.ra.HornetQRAXAResource@60771c61 pad=false overrideRmValue=null productName=HornetQ productVersion=2.0 jndiName=java:/CoreJMSXAConnectionFactory]) failed with exception code XAException.XAER_RMERR: javax.transaction.xa.XAException

        at org.hornetq.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:1838)

        at org.hornetq.ra.HornetQRAXAResource.rollback(HornetQRAXAResource.java:160)

        at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.rollback(XAResourceWrapperImpl.java:196)

        at org.jboss.jca.core.tx.jbossts.XAResourceWrapperStatImpl.rollback(XAResourceWrapperStatImpl.java:160)

        at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelAbort(XAResourceRecord.java:336)

        at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2993)

        at com.arjuna.ats.arjuna.coordinator.BasicAction.doAbort(BasicAction.java:2972)

        at com.arjuna.ats.arjuna.coordinator.BasicAction.Abort(BasicAction.java:1675)

        at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.cancel(TwoPhaseCoordinator.java:127)

        at com.arjuna.ats.arjuna.AtomicAction.abort(AtomicAction.java:186)

        at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.rollbackAndDisassociate(TransactionImple.java:1282)

        at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:143)

        at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.rollback(BaseTransactionManagerDelegate.java:114)

        at org.jboss.tm.usertx.client.ServerVMClientUserTransaction.rollback(ServerVMClientUserTransaction.java:182)

        at org.springframework.transaction.jta.JtaTransactionManager.doRollback(JtaTransactionManager.java:1048)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:853)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:830)

        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.rollbackOnException(AbstractPollingMessageListenerContainer.java:401)

        at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:236)

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1150)

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1142)

        at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1039)

        at java.lang.Thread.run(Thread.java:745)

      Caused by: HornetQNotConnectedException[errorType=NOT_CONNECTED message=HQ119010: Connection is destroyed]

        at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:301)

        at org.hornetq.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:1806)

        ... 22 more

       

      Any clues? Here attached my messaging subsystem config:

      <subsystem xmlns="urn:jboss:domain:messaging:3.0">
                  <hornetq-server>
                      <persistence-enabled>true</persistence-enabled>
                      <shared-store>false</shared-store>
                      <journal-type>ASYNCIO</journal-type>
                      <journal-file-size>102400</journal-file-size>
                      <journal-min-files>2</journal-min-files>
      
      
                      <connectors>
                          <in-vm-connector name="in-vm" server-id="0"/>
                      </connectors>
      
      
                      <acceptors>
                          <in-vm-acceptor name="in-vm" server-id="0"/>
                      </acceptors>
      
      
                      <security-settings>
                          <security-setting match="#">
                              <permission type="send" roles="guest"/>
                              <permission type="consume" roles="guest"/>
                              <permission type="createNonDurableQueue" roles="guest"/>
                              <permission type="deleteNonDurableQueue" roles="guest"/>
                          </security-setting>
                      </security-settings>
      
      
                      <address-settings>
                          <address-setting match="#">
                              <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                              <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                              <max-size-bytes>10485760</max-size-bytes>
                              <page-size-bytes>2097152</page-size-bytes>
                              <message-counter-history-day-limit>10</message-counter-history-day-limit>
                          </address-setting>
                      </address-settings>
      
      
                      <jms-connection-factories>
                          <pooled-connection-factory name="hornetq-ra">
                              <transaction mode="xa"/>
                              <connectors>
                                  <connector-ref connector-name="in-vm"/>
                              </connectors>
                              <entries>
                                  <entry name="java:jboss/CoreJMSXAConnectionFactory"/>
                              </entries>
                          </pooled-connection-factory>
                      </jms-connection-factories>
      
      
                      <jms-destinations>
                          <jms-queue name="ExpiryQueue">
                              <entry name="java:/jms/queue/ExpiryQueue"/>
                          </jms-queue>
                          <jms-queue name="DLQ">
                              <entry name="java:/jms/queue/DLQ"/>
                          </jms-queue>
                          <jms-queue name="EmailQueue">
                              <entry name="java:/jms/queue/EmailQueue"/>
                          </jms-queue>
                      </jms-destinations>
                  </hornetq-server>
              </subsystem>
      
      

       

      I'm using Wildfly 9.0.1, and using spring "@JmsListener" listeners.