Hi,
The hornetq is kind of stuck.I found few threads on the same but could not jump to any conclusion.
Request you to provide some leads.Please find the screenshot below.
Few observations from my side.
1)The no:of consumers is 1.
2)The address policy is by default set to BLOCK.
Please let me know if you require any further info.Any help in this regard would be deeply appreciated.
Forgot to attach the screenshot.We are using embedded hornetq version is 2.2.14.Final in the Jboss7.1.1.Final Brontes.
all the screenshot really tells us is thatthe consumer has 31 messages that it hasnt acknowledged yet, maybe if you give us some more info, or a test program, we could take a look. i.e. what is your client doing, is there anything in the logs.
The logs show the following.
00:00:17,804 ERROR [org.hornetq.core.remoting.impl.netty.NettyConnector] (Thread-25 (HornetQ-server-HornetQServerImpl::serverUUID=c7798d73-46ab-11e2-8612-0050569234df-1415842597)) Failed to create netty connection: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.6.0_24]
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351) [rt.jar:1.6.0_24]
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213) [rt.jar:1.6.0_24]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200) [rt.jar:1.6.0_24]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) [rt.jar:1.6.0_24]
at java.net.Socket.connect(Socket.java:529) [rt.jar:1.6.0_24]
at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.connect(OioClientSocketPipelineSink.java:114) [netty-3.2.6.Final.jar:]
at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.eventSunk(OioClientSocketPipelineSink.java:74) [netty-3.2.6.Final.jar:]
at org.jboss.netty.channel.Channels.connect(Channels.java:541) [netty-3.2.6.Final.jar:]
at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:210) [netty-3.2.6.Final.jar:]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:227) [netty-3.2.6.Final.jar:]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:188) [netty-3.2.6.Final.jar:]
at org.hornetq.core.remoting.impl.netty.NettyConnector.createConnection(NettyConnector.java:473) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnection(ClientSessionFactoryImpl.java:1143) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:993) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:224) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:663) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:619) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.server.cluster.impl.ClusterConnectionBridge.createSessionFactory(ClusterConnectionBridge.java:152) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.server.cluster.impl.BridgeImpl.connect(BridgeImpl.java:729) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.core.server.cluster.impl.BridgeImpl$ConnectRunnable.run(BridgeImpl.java:1005) [hornetq-core-2.2.13.Final.jar:]
at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [hornetq-core-2.2.13.Final.jar:]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_24]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_24]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_24]
And we are programmatically acknowleding the message by invoking message.acknowledge().One point is that if we didnot acknowledge then the messages would just keep piling up.It works well most of the time and somewhere it between we encounter this scenario(queue being stuck) so you can be assured that we are acknowledging the message.
We are using the Spring Framework listener on the queue.
We had faced this issue a couple of times, and all we did was to restart the server.Then the message processing works as usual.But we wanted to know the cause of this problem.
it looks like you have had a network issue, you should configure hornetq to deal with reconnects and failures, take a look at the examples for this. Also make sure you use the spring cached connection factory
Thanks for your patience and the nice suggestion about cachedconnection factory.Will definitely take a look at that.Thelistener and the hornetQ are residing in the same Jboss so there could not be a failure in terms of network but one thing is for sure that the hornetq just hung and got stuck due to which we are not able to connect to it.What we want to know is that in what situations could the hornetq hang or stop responding.
the stack trace you printed clearly shows you are using netty however it is on the cluster, maybe your consumer is connected to a node thathas no messages. you will have to give me more info if i can help you any further