Jboss 6.1.0, HornetQ 2.3.0 Session creation hangs after fail-over
I have a pair of live-backup stand-alone 2.3.0 HornetQ servers (set up on different machines, NFS as share storage).
I use Jboss 6.1.0 and removed the embedded HornetQ server.
I've manually upgraded hornetq libraries to 2.3.0 (jboss 6.1.0 ships with an older version).
Once the fail-over occurs the thread that should connect and push data to the queues hangs at session creation.
I waited for 40 mins and stopped (killed) jboss.
This is where the thread hangs:
- waiting on <0x3eab7ac1> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:318) - locked <0x67e035d8> (a java.lang.Object) org.hornetq.core.client.impl.ClientSessionImpl.setTransactionTimeout(ClientSessionImpl.java:1689) org.hornetq.ra.HornetQRAXAResource.setTransactionTimeout(HornetQRAXAResource.java:249) com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:604) com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:390) org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener$TransactionSynchronization.enlist(TxConnectionManager.java:975) org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.enlist(TxConnectionManager.java:729) org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionReconnected(TxConnectionManager.java:459) org.jboss.resource.connectionmanager.BaseConnectionManager2.reconnectManagedConnection(BaseConnectionManager2.java:625) org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:499) org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941) org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:832) - locked <0x267f81c2> (a java.util.HashSet) org.hornetq.ra.HornetQRASessionFactoryImpl.createSession(HornetQRASessionFactoryImpl.java:465)
Can you please provide me any clues in how to debug this.
Is this an appropriate way (see the below configs ) to make the client aware of the existence of the
two servers?
This is the connection factory:
<connection-factory name="ConnectionFactory">
<xa>true</xa>
<connectors>
<connector-ref connector-name="netty-connector"/>
</connectors>
<entries>
<entry name="XAConnectionFactory"/>
</entries>
<ha>true</ha>
<!-- Pause 1 second between connect attempts -->
<retry-interval>1000</retry-interval>
<!-- Multiply subsequent reconnect pauses by this multiplier. This can be used to
implement an exponential back-off. For our purposes we just set to 1.0 so each reconnect
pause is the same length -->
<retry-interval-multiplier>1.0</retry-interval-multiplier>
<!-- Try reconnecting an unlimited number of times (-1 means "unlimited") -->
<reconnect-attempts>-1</reconnect-attempts>
</connection-factory>
I've attached the 2 servers configs.
Jboss conf files:
/deploy/jms-ds.xml
<connection-factories>
<tx-connection-factory>
<jndi-name>jms/HornetQ/XACF</jndi-name>
<xa-transaction/>
<rar-name>jms-ra.rar</rar-name>
<connection-definition>org.hornetq.ra.HornetQRAConnectionFactory</connection-definition>
<config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>
<config-property name="ConnectorClassName" type="java.lang.String">org.hornetq.core.remoting.impl.netty.NettyConnectorFactory,org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</config-property>
<config-property name="ConnectionParameters" type="java.lang.String">host=192.168.8.241;port=5445,host=192.168.8.51;port=5445</config-property>
<max-pool-size>20</max-pool-size>
</tx-connection-factory>
</connection-factories>
/deploy/jms-ra.rar/META-INF
<resourceadapter>
<resourceadapter-class>org.hornetq.ra.HornetQResourceAdapter</resourceadapter-class>
<config-property>
<description>
The transport type. Multiple connectors can be configured by using a comma separated list,
i.e. org.hornetq.core.remoting.impl.invm.InVMConnectorFactory,org.hornetq.core.remoting.impl.invm.InVMConnectorFactory.
</description>
<config-property-name>ConnectorClassName</config-property-name>
<config-property-type>java.lang.String</config-property-type>
<!-- <config-property-value>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</config-property-value>-->
<config-property-value>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory,org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</config-property-value>
</config-property>
<config-property>
<config-property-name>ConnectionParameters</config-property-name>
<config-property-type>java.lang.String</config-property-type>
<!-- <config-property-value>server-id=0</config-property-value>-->
<config-property-value>host=192.168.8.241;port=5445,host=192.168.8.51;port=5446</config-property-value>
</config-property>
.............
Connection Factory is injected:
@Resource(mappedName = "java:/jms/HornetQ/XACF")
private static QueueConnectionFactory connectionFactory;
public static QueueConnection getQueueConnection() throws JMSException
{
return connectionFactory.createQueueConnection();
}
.....
queueConnection = getQueueConnection();
queueConnection.start();
session = queueConnection.createSession(true, Session.AUTO_ACKNOWLEDGE);
I would appreciate your help/advice. Thanks.
-
hornetq-bk-jms.xml 1.7 KB
-
hornetq-bk-configuration.xml 3.0 KB
-
hornetq-live-jms.xml 1.7 KB
