3 Replies Latest reply on Mar 28, 2012 8:39 PM by craigm123

Failing-over active-active cluster with JmsTemplate

craigm123 Mar 21, 2012 1:24 AM

I have configured an active-active cluster using a discovery group. I've connected a single client to the cluster that sends and receives messages that uses a CachingConnectionFactory, JmsTemplate, and DefaultMessageListenerContainer. After I kill one of the nodes, it take about 30 seconds before the client reconnects to the other active node. I've tried setting the ClientFailureCheckPeriod, but this doesn't seem to change the behavior.

Here's the configuration for my client -

{noformat}
  <bean id="connectionFactory" class="org.hornetq.api.jms.HornetQJMSClient" factory-method="createConnectionFactoryWitho
utHA">
    <constructor-arg name="groupConfiguration">
      <bean class="org.hornetq.api.core.DiscoveryGroupConfiguration">
        <constructor-arg name="groupAddress" value="231.7.7.7"/>
        <constructor-arg name="groupPort" value="9876"/>
      </bean>
    </constructor-arg>
    <constructor-arg name="jmsFactoryType"><util:constant static-field="org.hornetq.api.jms.JMSFactoryType.CF"/></constr
uctor-arg>
  </bean>
 
  <bean id="connectionFactoryWithPropertiesSet" class="com.myco.jmstest.CFPropertySetter">
    <constructor-arg name="connectionFactory" ref="connectionFactory"/>
  </bean>
 
  <bean id="connectionFactoryAdapter" class="org.springframework.jms.connection.UserCredentialsConnectionFactoryAdapter"
>
    <property name="targetConnectionFactory" ref="connectionFactoryWithPropertiesSet" />
    <property name="username" value="guest"/>
    <property name="password" value="guest"/>
  </bean>
 
  <bean id="cachedConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
    <property name="targetConnectionFactory" ref="connectionFactoryAdapter" />
  </bean>
 
  <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
    <property name="connectionFactory" ref="cachedConnectionFactory" />
    <property name="sessionTransacted" value="false" />
    <property name="sessionAcknowledgeMode">
      <util:constant static-field="javax.jms.Session.CLIENT_ACKNOWLEDGE"/>
    </property>
    <property name="messageIdEnabled" value="false" />
    <property name="messageTimestampEnabled" value="false" />
  </bean>
 
  <bean id="testQueue" class="org.hornetq.api.jms.HornetQJMSClient" factory-method="createQueue">
    <constructor-arg value="exampleQueue" />
  </bean>
 
  <bean id="jmsSender" class="com.myco.jmstest.JmsSender">
    <constructor-arg name="template" ref="jmsTemplate"/>
    <constructor-arg name="queue" ref="testQueue"/>
  </bean>
 
  <bean id="jmsListener" class="com.myco.jmstest.JmsListener"/>
 
  <bean id="listenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
    <property name="connectionFactory" ref="cachedConnectionFactory" />
    <property name="destination" ref="testQueue" />
    <property name="messageListener" ref="jmsListener"/>
    <property name="concurrentConsumers" value="1" />
    <property name="maxConcurrentConsumers" value="1" />
  </bean>
{noformat}

The class CFPropertySetter sets a few properties on the connection factory -

{noformat}
connectionFactory.setClientFailureCheckPeriod(500);
connectionFactory.setConnectionTTL(500);
connectionFactory.setCallTimeout(5000);
{noformat}

Is there some other setting that I'm missing? The full client code is attached (using 2.2.5).

jmstest.tgz 4.8 KB

1. Re: Failing-over active-active cluster with JmsTemplate

craigm123 Mar 28, 2012 2:49 PM (in response to craigm123)

After digging in the source code and adding some additional logging statements, I found the client failure check period was working just fine. In fact, the client connection immediately switches over to the other active member of the cluster. I also added some logging to ChannelImpl to show where messages were getting sent. Here's a clip of the log. Here the client is sending to the node with acceptor on port 5446, then I kill that broker node and the client immediately starts sending to the node with acceptor on port 5445. After about 30 seconds, the client is finally able to send to the node without getting a timeout exception.

{noformat}
WARNING: Sending to remote address: localhost/127.0.0.1:5446
Mar 28, 2012 11:36:19 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5446
Mar 28, 2012 11:36:19 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5446
Mar 28, 2012 11:36:19 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5446
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Executing failoverOrReconnect
30331 WARN [Thread-1] org.springframework.jms.connection.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection
javax.jms.JMSException: HornetQException[errorCode=2 message=Channel disconnected]
          at org.hornetq.jms.client.HornetQConnection$JMSFailureListener.connectionFailed(HornetQConnection.java:643)
          at org.hornetq.core.client.impl.ClientSessionFactoryImpl.callFailureListeners(ClientSessionFactoryImpl.java:821)
          at org.hornetq.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:608)
          at org.hornetq.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:482)
          at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connectionDestroyed(ClientSessionFactoryImpl.java:363)
          at org.hornetq.core.remoting.impl.netty.NettyConnector$Listener$1.run(NettyConnector.java:687)
          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:662)
Caused by: HornetQException[errorCode=2 message=Channel disconnected]
          ... 6 more
30334 WARN [listenerContainer-1] org.springframework.jms.listener.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'HornetQQueue[exampleQueue]' - trying to recover. Cause: Consumer is closed
30335 INFO [Thread-1] com.myco.jmstest.ConnectionDecorator - Stop connection
30335 INFO [Thread-1] com.myco.jmstest.ConnectionDecorator - Closing connection
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Creating ClientSessionFactoryImpl - attempt: 0
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Trying reconnection attempt: 0 reconnect attempts: 1
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Scheduling ping with frequency - ClientFailureCheckPeriod: 500
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
30345 INFO [listenerContainer-1] com.myco.jmstest.ConnectionDecorator - Creating connection
30345 INFO [listenerContainer-1] org.springframework.jms.connection.CachingConnectionFactory - Established shared JMS Connection: com.myco.jmstest.ConnectionDecorator@2db7ae22
30345 INFO [listenerContainer-1] com.myco.jmstest.ConnectionDecorator - Start connection
30345 INFO [listenerContainer-1] org.springframework.jms.listener.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
30345 INFO [listenerContainer-2] com.myco.jmstest.ConnectionDecorator - Creating session
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
30347 INFO [listenerContainer-2] com.myco.jmstest.SessionDecorator - Session constructor
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
Mar 28, 2012 11:36:20 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
30532 INFO [listenerContainer-2] com.myco.jmstest.JmsListener - Messages received: 8889
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:44)
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)
          at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:210)
          at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:592)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:569)
          at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:536)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)
          ... 4 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          ... 14 more
35298 INFO [main] com.myco.jmstest.ConnectionDecorator - Creating session
Mar 28, 2012 11:36:24 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
Mar 28, 2012 11:36:25 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
35299 INFO [main] com.myco.jmstest.SessionDecorator - Session constructor
Mar 28, 2012 11:36:25 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
Mar 28, 2012 11:36:25 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
35303 INFO [main] com.myco.jmstest.JmsSender - Messages sent: 8273
Mar 28, 2012 11:36:25 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:44)
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)
          at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:210)
          at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:592)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:569)
          at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:536)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)
          ... 4 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          ... 14 more
Mar 28, 2012 11:36:30 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:44)
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)
          at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:210)
          at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:592)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:569)
          at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:536)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)
          ... 4 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          ... 14 more
Mar 28, 2012 11:36:35 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:44)
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)
          at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:210)
          at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:592)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:569)
          at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:536)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)
          ... 4 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          ... 14 more
Mar 28, 2012 11:36:40 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:44)
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)
          at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:210)
          at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:592)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:569)
          at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:536)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)
          ... 4 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          ... 14 more
Mar 28, 2012 11:36:45 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:44)
Caused by: javax.jms.JMSException: Timed out waiting for response when sending packet 71
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:277)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)
          at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:210)
          at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:592)
          at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:569)
          at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:536)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:466)
          ... 4 more
Caused by: HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          ... 14 more
Mar 28, 2012 11:36:50 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
65309 INFO [main] com.myco.jmstest.JmsSender - Messages sent: 8274
65309 INFO [listenerContainer-2] com.myco.jmstest.JmsListener - Messages received: 8998
Mar 28, 2012 11:36:55 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
Mar 28, 2012 11:36:55 AM org.hornetq.core.logging.impl.JULLogDelegate warn
WARNING: Sending to remote address: localhost/127.0.0.1:5445
{noformat}

I'm still not sure what causes the 30 second delay!
Actions
2. Re: Failing-over active-active cluster with JmsTemplate

craigm123 Mar 28, 2012 5:41 PM (in response to craigm123)

I found this exception in the broker's log right at the time my client starts sending traffic again -

HornetQException[errorCode=3 message=Timed out waiting for response when sending packet 71]
          at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:276)
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:287)
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)
          at org.hornetq.core.server.cluster.impl.BridgeImpl.handle(BridgeImpl.java:407)
          at org.hornetq.core.server.impl.QueueImpl.handle(QueueImpl.java:2017)
          at org.hornetq.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:1962)
          at org.hornetq.core.server.impl.QueueImpl.addTail(QueueImpl.java:392)
          at org.hornetq.core.postoffice.impl.PostOfficeImpl.addReferences(PostOfficeImpl.java:1140)
          at org.hornetq.core.postoffice.impl.PostOfficeImpl.access$200(PostOfficeImpl.java:77)
          at org.hornetq.core.postoffice.impl.PostOfficeImpl$1.done(PostOfficeImpl.java:990)
          at org.hornetq.core.persistence.impl.journal.OperationContextImpl$1.run(OperationContextImpl.java:239)
          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:662)

I didn't realize this, but the two brokers are connected via a bridge and the communication between the two also uses the same call-timeout parameter as the client. Unfortunately I don't see a way to configure this. I did change it in the source (in ClusterConnectionImpl) and immediately got the behavior I was looking for (default is 30000 milliseconds and changed it to 500 milliseconds).

Is there any way to configure this?
Actions
3. Re: Failing-over active-active cluster with JmsTemplate

craigm123 Mar 28, 2012 8:39 PM (in response to craigm123)

I think the core problem here is that the wait in the sendBlocking method. On a channel close, the returnBlocking call needs to be called so it signals the wait in the sendBlocking call.

I tried this out on 2.2.11.Final and looks like it's been fixed. I see the following exception, which notes it unblocks sendBlocking.

org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Connection failure detected. Unblocking a blocking call that will never get a response
          at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:316)
          at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
          at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:469)
          at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:534)
          at org.springframework.jms.core.JmsTemplate.convertAndSend(JmsTemplate.java:612)
          at com.myco.jmstest.JmsSender.sendMessage(JmsSender.java:26)
          at com.myco.jmstest.Main.main(Main.java:48)
Caused by: javax.jms.JMSException: Connection failure detected. Unblocking a blocking call that will never get a response65548 WARN
Actions

Go to original post