-
1. Re: HornetQ in delivery stuck.
ananthkumar1985 Jan 7, 2013 5:47 AM (in response to ananthkumar1985) -
2. Re: HornetQ in delivery stuck.
ataylor Jan 7, 2013 6:24 AM (in response to ananthkumar1985)all the screenshot really tells us is thatthe consumer has 31 messages that it hasnt acknowledged yet, maybe if you give us some more info, or a test program, we could take a look. i.e. what is your client doing, is there anything in the logs.
-
3. Re: HornetQ in delivery stuck.
ananthkumar1985 Jan 7, 2013 7:05 AM (in response to ananthkumar1985)The logs show the following.
00:00:17,804 ERROR [org.hornetq.core.remoting.impl.netty.NettyConnector] (Thread-25 (HornetQ-server-HornetQServerImpl::serverUUID=c7798d73-46ab-11e2-8612-0050569234df-1415842597)) Failed to create netty connection: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.6.0_24]
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351) [rt.jar:1.6.0_24]
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213) [rt.jar:1.6.0_24]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200) [rt.jar:1.6.0_24]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) [rt.jar:1.6.0_24]
at java.net.Socket.connect(Socket.java:529) [rt.jar:1.6.0_24]
at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.connect(OioClientSocketPipelineSink.java:114) [netty-3.2.6.Final.jar:]
at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.eventSunk(OioClientSocketPipelineSink.java:74) [netty-3.2.6.Final.jar:]
at org.jboss.netty.channel.Channels.connect(Channels.java:541) [netty-3.2.6.Final.jar:]
at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:210) [netty-3.2.6.Final.jar:]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227) [netty-3.2.6.Final.jar:]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188) [netty-3.2.6.Final.jar:]
at org.hornetq.core.remoting.impl.netty.NettyConnector.createConnection(NettyConnector.java:473) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnection(ClientSessionFactoryImpl.java:1143) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:993) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:224) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:663) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:619) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.server.cluster.impl.ClusterConnectionBridge.createSessionFactory(ClusterConnectionBridge.java:152) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.server.cluster.impl.BridgeImpl.connect(BridgeImpl.java:729) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.server.cluster.impl.BridgeImpl$ConnectRunnable.run(BridgeImpl.java:1005) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [hornetq-core-2.2.13.Final.jar:]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_24]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_24]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_24]
And we are programmatically acknowleding the message by invoking message.acknowledge().One point is that if we didnot acknowledge then the messages would just keep piling up.It works well most of the time and somewhere it between we encounter this scenario(queue being stuck) so you can be assured that we are acknowledging the message.
We are using the Spring Framework listener on the queue.
We had faced this issue a couple of times, and all we did was to restart the server.Then the message processing works as usual.But we wanted to know the cause of this problem.
-
4. Re: HornetQ in delivery stuck.
ataylor Jan 7, 2013 7:40 AM (in response to ananthkumar1985)it looks like you have had a network issue, you should configure hornetq to deal with reconnects and failures, take a look at the examples for this. Also make sure you use the spring cached connection factory
-
5. Re: HornetQ in delivery stuck.
ananthkumar1985 Jan 7, 2013 9:22 AM (in response to ataylor)Thanks for your patience and the nice suggestion about cachedconnection factory.Will definitely take a look at that.Thelistener and the hornetQ are residing in the same Jboss so there could not be a failure in terms of network but one thing is for sure that the hornetq just hung and got stuck due to which we are not able to connect to it.What we want to know is that in what situations could the hornetq hang or stop responding.
-
6. Re: HornetQ in delivery stuck.
ataylor Jan 7, 2013 10:14 AM (in response to ananthkumar1985)the stack trace you printed clearly shows you are using netty however it is on the cluster, maybe your consumer is connected to a node thathas no messages. you will have to give me more info if i can help you any further