NPE when releasing consumer credits in failover test
While running NettyDiscoveryClusterWithBackupFailoverTest.testFailAllNodes, I had a NPE when failing over:
30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Failed to release outstanding credits java.lang.NullPointerException at org.hornetq.core.server.impl.ServerSessionImpl$CreditManagerHolder.<init>(ServerSessionImpl.java:2028) at org.hornetq.core.server.impl.ServerSessionImpl.getCreditManagerHolder(ServerSessionImpl.java:1961) at org.hornetq.core.server.impl.ServerSessionImpl.releaseOutStanding(ServerSessionImpl.java:1946) at org.hornetq.core.server.impl.ServerSessionImpl.handleSend(ServerSessionImpl.java:1549) at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:273) at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:456)
and the client is stuck while acquiring credits:
"main" prio=5 tid=0x0000000102002000 nid=0x101301000 waiting on condition [0x00000001012ff000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x0000000108115618> (a java.util.concurrent.Semaphore$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:905) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1217) at java.util.concurrent.Semaphore.acquire(Semaphore.java:441) at org.hornetq.core.client.impl.ClientProducerCreditsImpl.acquireCredits(ClientProducerCreditsImpl.java:69) at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:276) at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:133) at org.hornetq.core.server.cluster.impl.BridgeImpl.setupNotificationConsumer(BridgeImpl.java:631) at org.hornetq.core.server.cluster.impl.BridgeImpl.fail(BridgeImpl.java:543) at org.hornetq.core.server.cluster.impl.BridgeImpl.connectionFailed(BridgeImpl.java:488) at org.hornetq.core.client.impl.FailoverManagerImpl.callFailureListeners(FailoverManagerImpl.java:734) at org.hornetq.core.client.impl.FailoverManagerImpl.failoverOrReconnect(FailoverManagerImpl.java:696) - locked <0x0000000108035c28> (a java.lang.Object) at org.hornetq.core.client.impl.FailoverManagerImpl.handleConnectionFailure(FailoverManagerImpl.java:547) at org.hornetq.core.client.impl.FailoverManagerImpl.access$2(FailoverManagerImpl.java:545)
This occurs when the bridge connects to the backup server after failing live server and sends a management message "sendQueueInfoToQueue" to the backup's management address.
30 nov. 2009 14:50:09 org.hornetq.core.logging.impl.JULLogDelegate info INFO: ###### starting test org.hornetq.tests.integration.cluster.failover.NettyDiscoveryClusterWithBackupFailoverTest.testFailAllNodes 30 nov. 2009 14:50:10 org.hornetq.core.logging.impl.JULLogDelegate info INFO: starting server 3 30 nov. 2009 14:50:10 org.hornetq.core.logging.impl.JULLogDelegate info INFO: backup server is starting.. 30 nov. 2009 14:50:10 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: It has been detected that the cluster admin user and password which are used to replicate management operation from one node to the other have not been changed from the installation default. Please see the HornetQ user guide for instructions on how to do this. 30 nov. 2009 14:50:10 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:10 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:10 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Backup server initialised 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Unexpected Netty Version was expecting @NETTY.VERSION@ using 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Started Netty Acceptor version 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: HornetQ Server version 2.0.0.BETA5 (hornet-baby, 108) started 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: started server 3 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: starting server 4 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: backup server is starting.. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: It has been detected that the cluster admin user and password which are used to replicate management operation from one node to the other have not been changed from the installation default. Please see the HornetQ user guide for instructions on how to do this. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Backup server initialised 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Unexpected Netty Version was expecting @NETTY.VERSION@ using 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Started Netty Acceptor version 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: HornetQ Server version 2.0.0.BETA5 (hornet-baby, 108) started 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: started server 4 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: starting server 5 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: backup server is starting.. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: It has been detected that the cluster admin user and password which are used to replicate management operation from one node to the other have not been changed from the installation default. Please see the HornetQ user guide for instructions on how to do this. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Backup server initialised 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Unexpected Netty Version was expecting @NETTY.VERSION@ using 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Started Netty Acceptor version 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: HornetQ Server version 2.0.0.BETA5 (hornet-baby, 108) started 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: started server 5 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: starting server 0 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: live server is starting.. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: It has been detected that the cluster admin user and password which are used to replicate management operation from one node to the other have not been changed from the installation default. Please see the HornetQ user guide for instructions on how to do this. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Unexpected Netty Version was expecting @NETTY.VERSION@ using 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Started Netty Acceptor version 3.1.5.GA-r1772 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: HornetQ Server version 2.0.0.BETA5 (hornet-baby, 108) started 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: started server 0 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: starting server 1 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: live server is starting.. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: It has been detected that the cluster admin user and password which are used to replicate management operation from one node to the other have not been changed from the installation default. Please see the HornetQ user guide for instructions on how to do this. 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:11 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Unexpected Netty Version was expecting @NETTY.VERSION@ using 3.1.5.GA-r1772 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Started Netty Acceptor version 3.1.5.GA-r1772 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: HornetQ Server version 2.0.0.BETA5 (hornet-baby, 108) started 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: started server 1 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: starting server 2 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: live server is starting.. 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: It has been detected that the cluster admin user and password which are used to replicate management operation from one node to the other have not been changed from the installation default. Please see the HornetQ user guide for instructions on how to do this. 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Unexpected Netty Version was expecting @NETTY.VERSION@ using 3.1.5.GA-r1772 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Started Netty Acceptor version 3.1.5.GA-r1772 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: HornetQ Server version 2.0.0.BETA5 (hornet-baby, 108) started 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: started server 2 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster0.47bf339f-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster2.47bf339f-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster1.47a8774d-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster2.47a8774d-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster0.47d92441-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster1.47d92441-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster0.47bf339f-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster2.47a8774d-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster1.47a8774d-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster2.47bf339f-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster1.47d92441-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:12 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster0.47d92441-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate info INFO: *** failing node 0 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Connection failure has been detected: blah [code=0] 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Activating backup server 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate info INFO: AIO journal selected 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Failed to create consumer HornetQException[errorCode=100 message=Queue notif.4806eb13-ddb7-11de-8847-001c42000009 does not exist] at org.hornetq.core.server.impl.ServerSessionImpl.handleCreateConsumer(ServerSessionImpl.java:391) at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:125) at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:456) at org.hornetq.core.remoting.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:383) at org.hornetq.core.remoting.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:337) at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:416) at org.hornetq.integration.transports.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:68) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:98) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:391) at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317) at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299) at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:345) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:332) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323) at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:275) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) 30 nov. 2009 14:50:17 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Failed to create consumer HornetQException[errorCode=100 message=Queue notif.4806c402-ddb7-11de-8847-001c42000009 does not exist] at org.hornetq.core.server.impl.ServerSessionImpl.handleCreateConsumer(ServerSessionImpl.java:391) at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:125) at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:456) at org.hornetq.core.remoting.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:383) at org.hornetq.core.remoting.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:337) at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:416) at org.hornetq.integration.transports.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:68) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:98) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:391) at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317) at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299) at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:345) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:332) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323) at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:275) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: There is no consumer with id 0 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: There is no consumer with id 0 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Cannot find consumer with id 0 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster0.47d92441-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Cannot find consumer with id 0 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Can't find packet to clear: last received command id 8 first stored command id 0 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Connecting bridge sf.cluster0.47bf339f-ddb7-11de-8847-001c42000009 to its destination 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster0.47d92441-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate info INFO: Bridge sf.cluster0.47bf339f-ddb7-11de-8847-001c42000009 is connected to its destination 30 nov. 2009 14:50:18 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Failed to release outstanding credits java.lang.NullPointerException at org.hornetq.core.server.impl.ServerSessionImpl$CreditManagerHolder.<init>(ServerSessionImpl.java:2028) at org.hornetq.core.server.impl.ServerSessionImpl.getCreditManagerHolder(ServerSessionImpl.java:1961) at org.hornetq.core.server.impl.ServerSessionImpl.releaseOutStanding(ServerSessionImpl.java:1946) at org.hornetq.core.server.impl.ServerSessionImpl.handleSend(ServerSessionImpl.java:1549) at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:273) at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:456) at org.hornetq.core.remoting.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:383) at org.hornetq.core.remoting.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:337) at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:416) at org.hornetq.integration.transports.netty.HornetQChannelHandler.messageReceived(HornetQChannelHandler.java:68) at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:98) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:796) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:391) at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:317) at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:299) at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:216) at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:560) at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:555) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:345) at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:332) at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:323) at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:275) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:196) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Connection failure has been detected: Did not receive ping from /127.0.0.1:50059. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. The connection will now be closed. [code=3] 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Client connection failed, clearing up resources for session 47fba071-ddb7-11de-8847-001c42000009 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Failed to close connection org.hornetq.core.server.impl.ServerSessionImpl@784e8edc 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Connection failure has been detected: Did not receive ping from /127.0.0.1:50058. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. The connection will now be closed. [code=3] 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Client connection failed, clearing up resources for session 47f6704f-ddb7-11de-8847-001c42000009 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate error GRAVE: Failed to close connection org.hornetq.core.server.impl.ServerSessionImpl@3cbe7667 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Connection failure has been detected: Did not receive ping from /127.0.0.1:50064. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. The connection will now be closed. [code=3] 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Client connection failed, clearing up resources for session 484294a3-ddb7-11de-8847-001c42000009 30 nov. 2009 14:52:14 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Cleared up resources for session 484294a3-ddb7-11de-8847-001c42000009 30 nov. 2009 14:52:19 org.hornetq.core.logging.impl.JULLogDelegate warn ATTENTION: Connection failure has been detected: Did not receive ping from /127.0.0.1:50070. It is likely the client has exited or crashed without closing its connection, or the network between the server and client has failed. The connection will now be closed. [code=3]
2009-11-30 15:09:34 Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.1-b02-90 mixed mode): "Thread-5 (group:HornetQ-server-threads1603313878-271544990)" prio=5 tid=0x0000000102177000 nid=0x115200000 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Thread-7" prio=5 tid=0x0000000106406800 nid=0x119f8c000 waiting for monitor entry [0x0000000119f8b000] java.lang.Thread.State: BLOCKED (on object monitor) at org.hornetq.core.client.impl.FailoverManagerImpl.failoverOrReconnect(FailoverManagerImpl.java:556) - waiting to lock <0x0000000108035c28> (a java.lang.Object) at org.hornetq.core.client.impl.FailoverManagerImpl.handleConnectionFailure(FailoverManagerImpl.java:547) at org.hornetq.core.client.impl.FailoverManagerImpl.connectionDestroyed(FailoverManagerImpl.java:285) at org.hornetq.integration.transports.netty.NettyConnector$Listener$1.run(NettyConnector.java:615) "hornetq-discovery-group-thread-dg1" daemon prio=5 tid=0x0000000102282000 nid=0x11a5eb000 runnable [0x000000011a5ea000] java.lang.Thread.State: RUNNABLE at java.net.PlainDatagramSocketImpl.receive0(Native Method) - locked <0x0000000108308300> (a java.net.PlainDatagramSocketImpl) at java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136) - locked <0x0000000108308300> (a java.net.PlainDatagramSocketImpl) at java.net.DatagramSocket.receive(DatagramSocket.java:712) - locked <0x00000001070b1e38> (a java.net.DatagramPacket) - locked <0x000000010831a200> (a java.net.MulticastSocket) at org.hornetq.core.cluster.impl.DiscoveryGroupImpl.run(DiscoveryGroupImpl.java:317) at java.lang.Thread.run(Thread.java:637) "Thread-1 (group:HornetQ-scheduled-threads-1546137693)" prio=5 tid=0x0000000102281800 nid=0x11a4e8000 runnable [0x000000011a4e7000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x0000000107e008b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Thread-0 (group:HornetQ-Asynchronous-Persistent-Writes1091981526-538260220)" daemon prio=5 tid=0x000000010227c800 nid=0x11a3e5000 waiting on condition [0x000000011a3e4000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x0000000108305498> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Thread-0 (group:HornetQ-scheduled-threads-1546137693)" prio=5 tid=0x00000001062f7000 nid=0x11a295000 runnable [0x000000011a294000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x0000000107e008b8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "hornetq-expiry-reaper-thread" prio=3 tid=0x000000010217d800 nid=0x114d77000 in Object.wait() [0x0000000114d76000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000001082f0ef0> (a org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper) at org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper.run(PostOfficeImpl.java:1065) - locked <0x00000001082f0ef0> (a org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper) at java.lang.Thread.run(Thread.java:637) "hornetq-async-buffer" prio=5 tid=0x000000010217e800 nid=0x114a6a000 waiting on condition [0x0000000114a69000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x0000000108309468> (a org.hornetq.utils.VariableLatch$CountSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:905) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1217) at org.hornetq.utils.VariableLatch.waitCompletion(VariableLatch.java:111) at org.hornetq.core.journal.impl.TimedBuffer$CheckTimer.run(TimedBuffer.java:401) at java.lang.Thread.run(Thread.java:637) "New I/O server worker #1-2" prio=5 tid=0x000000010215a800 nid=0x11a192000 runnable [0x000000011a191000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00000001082fc3d8> (a sun.nio.ch.Util$1) - locked <0x00000001082fc3c0> (a java.util.Collections$UnmodifiableSet) - locked <0x000000010810f150> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:158) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 645492252, /127.0.0.1:50066 => localhost/127.0.0.1:5447)" daemon prio=5 tid=0x0000000106371000 nid=0x119d86000 runnable [0x0000000119d85000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 328966616, /127.0.0.1:50073 => localhost/127.0.0.1:5447)" daemon prio=5 tid=0x0000000106370800 nid=0x119c83000 runnable [0x0000000119c82000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 396793761, /127.0.0.1:50074 => localhost/127.0.0.1:5446)" daemon prio=5 tid=0x00000001064f8000 nid=0x119b80000 runnable [0x0000000119b7f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 58472846, /127.0.0.1:50065 => localhost/127.0.0.1:5446)" daemon prio=5 tid=0x0000000102191000 nid=0x119a7d000 runnable [0x0000000119a7c000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "New I/O server worker #6-2" prio=5 tid=0x00000001064f7000 nid=0x119671000 runnable [0x0000000119670000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00000001080d7870> (a sun.nio.ch.Util$1) - locked <0x00000001080d7858> (a java.util.Collections$UnmodifiableSet) - locked <0x000000010802fbb8> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:158) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 2140365031, /127.0.0.1:50061 => localhost/127.0.0.1:5447)" daemon prio=5 tid=0x00000001064f5000 nid=0x11956e000 runnable [0x000000011956d000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "New I/O server worker #6-1" prio=5 tid=0x000000010218d000 nid=0x11946b000 runnable [0x000000011946a000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00000001080e1318> (a sun.nio.ch.Util$1) - locked <0x00000001080e1300> (a java.util.Collections$UnmodifiableSet) - locked <0x000000010802fca8> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:158) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 1147066451, /127.0.0.1:50060 => localhost/127.0.0.1:5447)" daemon prio=5 tid=0x00000001064f2000 nid=0x119240000 runnable [0x000000011923f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Thread-4 (group:HornetQ-client-global-scheduled-threads-1458446590)" daemon prio=5 tid=0x0000000106509800 nid=0x11903a000 runnable [0x0000000119039000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000010804a160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 732753528, /127.0.0.1:50072 => localhost/127.0.0.1:5448)" daemon prio=5 tid=0x0000000106508800 nid=0x118f37000 runnable [0x0000000118f36000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Thread-3 (group:HornetQ-client-global-scheduled-threads-1458446590)" daemon prio=5 tid=0x0000000106508000 nid=0x118d31000 runnable [0x0000000118d30000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000010804a160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 1015483951, /127.0.0.1:50069 => localhost/127.0.0.1:5448)" daemon prio=5 tid=0x0000000106507000 nid=0x118c2e000 runnable [0x0000000118c2d000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Thread-2 (group:HornetQ-client-global-scheduled-threads-1458446590)" daemon prio=5 tid=0x0000000102289800 nid=0x118a28000 runnable [0x0000000118a27000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000010804a160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 894261275, /127.0.0.1:50057 => localhost/127.0.0.1:5446)" daemon prio=5 tid=0x0000000102289000 nid=0x118925000 runnable [0x0000000118924000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "New I/O server worker #5-2" prio=5 tid=0x0000000102288000 nid=0x118822000 runnable [0x0000000118821000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x000000010805a458> (a sun.nio.ch.Util$1) - locked <0x000000010805a440> (a java.util.Collections$UnmodifiableSet) - locked <0x0000000107fd7f78> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:158) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Thread-1 (group:HornetQ-client-global-scheduled-threads-1458446590)" daemon prio=5 tid=0x0000000102287800 nid=0x11861c000 runnable [0x000000011861b000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000010804a160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 1300857109, /127.0.0.1:50056 => localhost/127.0.0.1:5446)" daemon prio=5 tid=0x0000000102286800 nid=0x118519000 runnable [0x0000000118518000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "New I/O server worker #5-1" prio=5 tid=0x0000000102284800 nid=0x118416000 runnable [0x0000000118415000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x000000010805fd78> (a sun.nio.ch.Util$1) - locked <0x000000010805fd60> (a java.util.Collections$UnmodifiableSet) - locked <0x0000000107fd8068> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:158) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "Thread-0 (group:HornetQ-client-global-scheduled-threads-1458446590)" daemon prio=5 tid=0x00000001063b7800 nid=0x11800a000 runnable [0x0000000118009000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000010804a160> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 820198133, /127.0.0.1:50071 => localhost/127.0.0.1:5448)" daemon prio=5 tid=0x00000001063b6800 nid=0x117f07000 runnable [0x0000000117f06000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:77) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "hornetq-failure-check-thread" prio=5 tid=0x0000000102280800 nid=0x117e04000 in Object.wait() [0x0000000117e03000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x0000000108216e90> (a org.hornetq.core.remoting.server.impl.RemotingServiceImpl$FailureCheckThread) at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$FailureCheckThread.run(RemotingServiceImpl.java:519) - locked <0x0000000108216e90> (a org.hornetq.core.remoting.server.impl.RemotingServiceImpl$FailureCheckThread) "New I/O server boss #6 (channelId: 1647958666, /127.0.0.1:5447)" prio=5 tid=0x0000000102280000 nid=0x117d01000 runnable [0x0000000117d00000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136) at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00000001080e0fd0> (a sun.nio.ch.Util$1) - locked <0x00000001080e0fb8> (a java.util.Collections$UnmodifiableSet) - locked <0x000000010802fc30> (a sun.nio.ch.KQueueSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:223) at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) at org.jboss.netty.util.internal.IoWorkerRunnable.run(IoWorkerRunnable.java:46) at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:179) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:637) "hornetq-discovery-group-thread-dg1" daemon prio=5 tid=0x0000000106313800 nid=0x117bfe000 runnable [0x0000000117bfd000] java.lang.Thread.State: RUNNABLE at java.net.PlainDatagramSocketImpl.receive0(Native Method) - locked <0x00000001080de9e0> (a java.net.PlainDatagramSocketImpl) at java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:136) - locked <0x00000001080de9e0> (a java.net.PlainDatagramSocketImpl) at java.net.DatagramSocket.receive(DatagramSocket.java:712) - locked <0x00000001070d26d0> (a java.net.DatagramPacket) - locked <0x00000001080c9a78> (a java.net.MulticastSocket) at org.hornetq.core.cluster.impl.DiscoveryGroupImpl.run(DiscoveryGroupImpl.java:317) at java.lang.Thread.run(Thread.java:637) "Thread-2 (group:HornetQ-scheduled-threads-1987075368)" prio=5 tid=0x0000000106312800 nid=0x117afb000 runnable [0x0000000117afa000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000001080ee400> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Thread-0 (group:HornetQ-Asynchronous-Persistent-Writes1290936822-747881713)" daemon prio=5 tid=0x000000010227d800 nid=0x1179f8000 waiting on condition [0x00000001179f7000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000001081d7240> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Thread-1 (group:HornetQ-scheduled-threads-1987075368)" prio=5 tid=0x0000000102264000 nid=0x1178f5000 runnable [0x00000001178f4000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000001080ee400> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "hornetq-expiry-reaper-thread" prio=3 tid=0x0000000102263000 nid=0x1177f2000 in Object.wait() [0x00000001177f1000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x0000000108215e50> (a org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper) at org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper.run(PostOfficeImpl.java:1065) - locked <0x0000000108215e50> (a org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper) at java.lang.Thread.run(Thread.java:637) "hornetq-async-buffer" prio=5 tid=0x0000000102262800 nid=0x1176ef000 waiting on condition [0x00000001176ee000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000001081df250> (a org.hornetq.utils.VariableLatch$CountSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:905) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1217) at org.hornetq.utils.VariableLatch.waitCompletion(VariableLatch.java:111) at org.hornetq.core.journal.impl.TimedBuffer$CheckTimer.run(TimedBuffer.java:401) at java.lang.Thread.run(Thread.java:637) "Thread-0 (group:HornetQ-scheduled-threads-1987075368)" prio=5 tid=0x0000000106310000 nid=0x1175ec000 runnable [0x00000001175eb000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000001080ee400> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:637) "Old I/O client worker (channelId: 1257515479, /127.0.0.1:50053 => localhost/127.0.0.1:5450)" prio=5 tid=0x000000010208f000 nid=0x1174e9000 runnable [0x00000001174e8000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.net.SocketInputStream.read(SocketInputStream.java:182) at java.io.FilterInputStream.read(FilterInputStream.java:66) at java.io.PushbackInputStream.read(PushbackInputStream.java:122) at org.jboss.netty.channel.socket.oio.OioWorker.ru
