On recovery Hornetq hangs and throws: XAException.XAER_RMERR: javax.transaction.xa.XAException
miha.sandor Jul 26, 2013 11:36 AMMy environment is as follows: jboss 6.1.0 and hornetq standalone server 2.3.0 configured with resource recovery
While my application is running on jboss if i bring down the hornetq server for a few minutes and then back up i can no longer connect to the messaging server.
It seems stuck while trying to recover a transaction and keeps throwing the following exception when trying to create a new session:
ARJUNA-16086 TransactionImple.enlistResource setTransactionTimeout on XAResource < formatId=131076, gtrid_length=29, bqual_length=28, tx_uid=0:ffffc0a808d7:126a:51f12cdc:1162, node_name=1, branch_uid=0:ffffc0a808d7:126a:51f12cdc:1169, eis_name=unknown eis name > threw: XAException.XAER_RMERR: javax.transaction.xa.XAException
at org.hornetq.core.client.impl.ClientSessionImpl.setTransactionTimeout(ClientSessionImpl.java:1696) [:]
at org.hornetq.ra.HornetQRAXAResource.setTransactionTimeout(HornetQRAXAResource.java:249) [:]
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:604) [:6.1.0.Final]
at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:390) [:6.1.0.Final]
at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener$TransactionSynchronization.enlist(TxConnectionManager.java:975) [:6.1.0.Final]
at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.enlist(TxConnectionManager.java:729) [:6.1.0.Final]
at org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionReconnected(TxConnectionManager.java:459) [:6.1.0.Final]
at org.jboss.resource.connectionmanager.BaseConnectionManager2.reconnectManagedConnection(BaseConnectionManager2.java:625) [:6.1.0.Final]
at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:499) [:6.1.0.Final]
at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941) [:6.1.0.Final]
at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:832) [:]
at org.hornetq.ra.HornetQRASessionFactoryImpl.createSession(HornetQRASessionFactoryImpl.java:465) [:]
at a.b.c.d.addWO(Facade.java:25) [:1.3 (1.131.1.0)]
The recovery configuration on jboss side:
<bean name="JTAEnvironmentBean" class="com.arjuna.ats.jta.common.JTAEnvironmentBean">
<annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss.jta:name=JTAEnvironmentBean", exposedInterface=com.arjuna.ats.jta.common.JTAEnvironmentBeanMBean.class, registerDirectly=true)</annotation>
<constructor factoryClass="com.arjuna.ats.jta.common.jtaPropertyManager" factoryMethod="getJTAEnvironmentBean"/>
<property name="lastResourceOptimisationInterface">org.jboss.tm.LastResource</property>
<property name="transactionManagerClassName">com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate</property>
<property name="userTransactionClassName">com.arjuna.ats.internal.jta.transaction.arjunacore.UserTransactionImple</property>
<property name="transactionSynchronizationRegistryClassName">com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionSynchronizationRegistryImple</property>
<property name="com.arjuna.ats.jta.jtaTMImplementation" value="com.arjuna.ats.internal.jta.transaction.jts.TransactionManagerImple"/>
<property name="com.arjuna.ats.arjuna.recovery.recoveryExtension5" value="com.arjuna.ats.internal.jta.recovery.jts.XARecoveryModule"/>
<property name="xaResourceRecoveryClassNames" preinstantiate="false">
<list elementClass="java.lang.String">
<value>org.hornetq.jms.server.recovery.HornetQXAResourceRecovery;org.hornetq.core.remoting.impl.netty.NettyConnectorFactory,guest,guest,host=192.168.8.215,port=5445</value>
</list>
</property>
<property name="xaRecoveryNodes" preinstantiate="false">
<list elementClass="java.lang.String">
<value>1</value>
</list>
</property>
<property name="xaResourceOrphanFilterClassNames" preinstantiate="false">
<list elementClass="java.lang.String">
<value>com.arjuna.ats.internal.jta.recovery.arjunacore.JTATransactionLogXAResourceOrphanFilter</value>
<value>com.arjuna.ats.internal.jta.recovery.arjunacore.JTANodeNameXAResourceOrphanFilter</value>
</list>
</property>
</bean>
And the netty acceptor on the standalone hornetq server:
<acceptors>
<acceptor name="netty">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
<param key="host" value="${hornetq.remoting.netty.host:localhost}"/>
<param key="port" value="${hornetq.remoting.netty.port:5445}"/>
</acceptor>
<acceptor name="netty-throughput">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
<param key="host" value="${hornetq.remoting.netty.host:localhost}"/>
<param key="port" value="${hornetq.remoting.netty.batch.port:5455}"/>
<param key="batch-delay" value="50"/>
<param key="direct-deliver" value="false"/>
</acceptor>
</acceptors>
This seems like the basic scenario for recovery and it does not work. Not only does it not recover the lost transaction but reconnecting to the queues is not possible.
I would really appreciate it if you could give me some pointers in how to debug this. Is there something else I need to configure?
I've looked around a bit and noticed that this exception is related to permissions when dealing with recovery for db sources but found nothing for messaging.
I've also tried a little debugging by myself but I got stuck on this:
From ClientSessionImpl (hornetq-core-client 2.3.0.Final)
public boolean setTransactionTimeout(final int seconds) throws XAException
{
checkXA();
try
{
SessionXASetTimeoutResponseMessage response = (SessionXASetTimeoutResponseMessage)channel.sendBlocking(new SessionXASetTimeoutMessage(seconds), PacketImpl.SESS_XA_SET_TIMEOUT_RESP);
return response.isOK();
}
catch (HornetQException e)
{
// This should never occur
throw new XAException(XAException.XAER_RMERR);
}
}
10x!