Bridge Leaking
As I suspected, BridgeImpl is leaking. If we can fix this we would speed up the testsuite.
This is easily replicated with MessageRedistributionTest::testRedistributionWhenConsumerIsClosed.
(Probably with the others Redistribution tests also, but this one above is the test I was using to debug it).
I - At the beggining, MessageRedistributionTest will setup three servers, with bridges between all of them.
II - When the test is finished, tearDown will stop Server 0, 1 and 2.
III - Server0 is stopped successfully without any problems.
IV - When server1 is stopped, the bridge between server1 and server0 will try a session.close(). Session.close will close the consumers which will never succeed, being hanging at this point on StopRunnable:
Thread-4 (group:JBM-server-threads1809605797-1381221365)" prio=10 tid=0x00007f80a81af400 nid=0x3a0d waiting on condition [0x00007f809fffe000..0x00007f809fffec70] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80e0600878> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2054) at org.jboss.messaging.core.remoting.impl.ChannelImpl.sendBlocking(ChannelImpl.java:292) - locked <0x00007f80e06008d8> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ClientConsumerImpl.doCleanUp(ClientConsumerImpl.java:745) at org.jboss.messaging.core.client.impl.ClientConsumerImpl.close(ClientConsumerImpl.java:315) at org.jboss.messaging.core.client.impl.ClientSessionImpl.closeChildren(ClientSessionImpl.java:1322) at org.jboss.messaging.core.client.impl.ClientSessionImpl.close(ClientSessionImpl.java:708) at org.jboss.messaging.core.client.impl.DelegatingSession.close(DelegatingSession.java:116) at org.jboss.messaging.core.server.cluster.impl.BridgeImpl$StopRunnable.run(BridgeImpl.java:702) - locked <0x00007f80e057df00> (a org.jboss.messaging.core.server.cluster.impl.BridgeImpl) at org.jboss.messaging.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619)
This thread will hung here even after the server is finished.
* I will provide the complete thread dump at the end of this post.
V - The test will finish allright, however ClientSession from the Bridge will still reference the Server, and threads/executors will just be hunging around.
VI - As you can see also, there are several threads trying to reconnect to the server: look at the complete dump:
Full thread dump Java HotSpot(TM) 64-Bit Server VM (11.3-b02 mixed mode): "Thread-30" prio=10 tid=0x00007f80a8088800 nid=0x3afb runnable [0x00007f80ae04c000..0x00007f80ae04c9f0] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00007f80de8fdf98> (a sun.nio.ch.Util$1) - locked <0x00007f80de8fdf80> (a java.util.Collections$UnmodifiableSet) - locked <0x00007f80de8fdc20> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:289) at java.lang.Thread.run(Thread.java:619) "server-timer" daemon prio=10 tid=0x00007f80a8086c00 nid=0x3afa in Object.wait() [0x00007f80ada46000..0x00007f80ada46a70] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00007f80de8fe5f0> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x00007f80de8fe5f0> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462) "Thread-5 (group:JBM-server-threads1809605797-1381221365)" prio=10 tid=0x00007f80a81b0c00 nid=0x3aed waiting for monitor entry [0x00007f80ac935000..0x00007f80ac935b70] java.lang.Thread.State: BLOCKED (on object monitor) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.removeSession(ConnectionManagerImpl.java:435) - waiting to lock <0x00007f80b5d8fe88> (a java.lang.Object) - locked <0x00007f80b5d8fe98> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ClientSessionImpl.doCleanup(ClientSessionImpl.java:1296) at org.jboss.messaging.core.client.impl.ClientSessionImpl.close(ClientSessionImpl.java:719) at org.jboss.messaging.core.client.impl.DelegatingSession.close(DelegatingSession.java:116) at org.jboss.messaging.core.server.cluster.impl.BridgeImpl$StopRunnable.run(BridgeImpl.java:702) - locked <0x00007f80b5d90030> (a org.jboss.messaging.core.server.cluster.impl.BridgeImpl) at org.jboss.messaging.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) "Thread-4 (group:JBM-server-threads1809605797-1381221365)" prio=10 tid=0x00007f80a81af400 nid=0x3a0d waiting for monitor entry [0x00007f809fffe000..0x00007f809fffec70] java.lang.Thread.State: BLOCKED (on object monitor) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.removeSession(ConnectionManagerImpl.java:435) - waiting to lock <0x00007f80b5d90708> (a java.lang.Object) - locked <0x00007f80b5d90718> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ClientSessionImpl.doCleanup(ClientSessionImpl.java:1296) at org.jboss.messaging.core.client.impl.ClientSessionImpl.close(ClientSessionImpl.java:719) at org.jboss.messaging.core.client.impl.DelegatingSession.close(DelegatingSession.java:116) at org.jboss.messaging.core.server.cluster.impl.BridgeImpl$StopRunnable.run(BridgeImpl.java:702) - locked <0x00007f80b5d908b0> (a org.jboss.messaging.core.server.cluster.impl.BridgeImpl) at org.jboss.messaging.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) "Thread-5 (group:JBM-client-global-threads-799449511)" daemon prio=10 tid=0x0000000042166c00 nid=0x39e1 waiting on condition [0x00007f80adb47000..0x00007f80adb47cf0] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.getConnectionWithRetry(ConnectionManagerImpl.java:790) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.reattachSessions(ConnectionManagerImpl.java:710) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failoverOrReconnect(ConnectionManagerImpl.java:611) - locked <0x00007f80b5d90708> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.handleConnectionFailure(ConnectionManagerImpl.java:483) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.access$2(ConnectionManagerImpl.java:481) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl$DelegatingFailureListener.connectionFailed(ConnectionManagerImpl.java:1151) at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:434) at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:254) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl$Channel0Handler$1.run(ConnectionManagerImpl.java:1094) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) "Thread-4 (group:JBM-server-threads2023202663-711090443)" prio=10 tid=0x0000000042182400 nid=0x39a2 waiting for monitor entry [0x00007f80ac32f000..0x00007f80ac32fbf0] java.lang.Thread.State: BLOCKED (on object monitor) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.removeSession(ConnectionManagerImpl.java:435) - waiting to lock <0x00007f80b5e1b878> (a java.lang.Object) - locked <0x00007f80b5e1b888> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ClientSessionImpl.doCleanup(ClientSessionImpl.java:1296) at org.jboss.messaging.core.client.impl.ClientSessionImpl.close(ClientSessionImpl.java:719) at org.jboss.messaging.core.client.impl.DelegatingSession.close(DelegatingSession.java:116) at org.jboss.messaging.core.server.cluster.impl.BridgeImpl$StopRunnable.run(BridgeImpl.java:702) - locked <0x00007f80b5df7598> (a org.jboss.messaging.core.server.cluster.impl.BridgeImpl) at org.jboss.messaging.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) "Thread-4 (group:JBM-client-global-scheduled-threads-1690235745)" daemon prio=10 tid=0x00007f80a81a6c00 nid=0x39a1 waiting on condition [0x00007f80ac430000..0x00007f80ac430c70] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80b5d98828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) "Thread-3 (group:JBM-client-global-threads-799449511)" daemon prio=10 tid=0x000000004216e400 nid=0x399e waiting on condition [0x00007f80ac531000..0x00007f80ac5319f0] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.getConnectionWithRetry(ConnectionManagerImpl.java:790) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.reattachSessions(ConnectionManagerImpl.java:710) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failoverOrReconnect(ConnectionManagerImpl.java:611) - locked <0x00007f80b5d8fe88> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.handleConnectionFailure(ConnectionManagerImpl.java:483) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.access$2(ConnectionManagerImpl.java:481) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl$DelegatingFailureListener.connectionFailed(ConnectionManagerImpl.java:1151) at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:434) at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:254) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl$Channel0Handler$1.run(ConnectionManagerImpl.java:1094) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) "Thread-0 (group:JBM-client-global-threads-799449511)" daemon prio=10 tid=0x0000000042137400 nid=0x3997 waiting on condition [0x00007f80acc38000..0x00007f80acc38d70] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.getConnectionWithRetry(ConnectionManagerImpl.java:790) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.reattachSessions(ConnectionManagerImpl.java:710) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failoverOrReconnect(ConnectionManagerImpl.java:611) - locked <0x00007f80b5e1b878> (a java.lang.Object) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.handleConnectionFailure(ConnectionManagerImpl.java:483) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.access$2(ConnectionManagerImpl.java:481) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl$DelegatingFailureListener.connectionFailed(ConnectionManagerImpl.java:1151) at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.callFailureListeners(RemotingConnectionImpl.java:434) at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:254) at org.jboss.messaging.core.client.impl.ConnectionManagerImpl$Channel0Handler$1.run(ConnectionManagerImpl.java:1094) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) "Thread-3 (group:JBM-client-global-scheduled-threads-1690235745)" daemon prio=10 tid=0x00007f80a819c400 nid=0x3992 waiting on condition [0x00007f80ad13d000..0x00007f80ad13dbf0] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80b5d98828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) "Thread-2 (group:JBM-client-global-scheduled-threads-1690235745)" daemon prio=10 tid=0x0000000042141c00 nid=0x3991 waiting on condition [0x00007f80ad23e000..0x00007f80ad23ec70] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80b5d98828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) "Thread-1 (group:JBM-client-global-scheduled-threads-1690235745)" daemon prio=10 tid=0x00007f80a406fc00 nid=0x3990 waiting on condition [0x00007f80ad33f000..0x00007f80ad33fcf0] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80b5d98828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) "Thread-0 (group:JBM-client-global-scheduled-threads-1690235745)" daemon prio=10 tid=0x00007f80a406e800 nid=0x398f waiting on condition [0x00007f80ad440000..0x00007f80ad440d70] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80b5d98828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:619) "Thread-2" prio=10 tid=0x00007f80a026e400 nid=0x397b runnable [0x00007f80ae85a000..0x00007f80ae85aaf0] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked <0x00007f80b5d9bff0> (a sun.nio.ch.Util$1) - locked <0x00007f80b5d9bfd8> (a java.util.Collections$UnmodifiableSet) - locked <0x00007f80b5d9bc00> (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80) at sun.net.httpserver.ServerImpl$Dispatcher.run(ServerImpl.java:289) at java.lang.Thread.run(Thread.java:619) "server-timer" daemon prio=10 tid=0x00007f80a0263000 nid=0x3979 in Object.wait() [0x00007f80ae95b000..0x00007f80ae95bb70] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00007f80b5d9c7c0> (a java.util.TaskQueue) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x00007f80b5d9c7c0> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462) "ReaderThread" prio=10 tid=0x00007f80a0270000 nid=0x3977 runnable [0x00007f80aeb64000..0x00007f80aeb64bf0] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:264) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158) - locked <0x00007f80b5ca74d0> (a java.io.InputStreamReader) at java.io.InputStreamReader.read(InputStreamReader.java:167) at java.io.BufferedReader.fill(BufferedReader.java:136) at java.io.BufferedReader.readLine(BufferedReader.java:299) - locked <0x00007f80b5ca74d0> (a java.io.InputStreamReader) at java.io.BufferedReader.readLine(BufferedReader.java:362) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner$ReaderThread.run(RemoteTestRunner.java:140) "Low Memory Detector" daemon prio=10 tid=0x00007f80a0002000 nid=0x3974 runnable [0x0000000000000000..0x0000000000000000] java.lang.Thread.State: RUNNABLE "CompilerThread1" daemon prio=10 tid=0x00007f80a8046400 nid=0x3973 waiting on condition [0x0000000000000000..0x00007f80af145570] java.lang.Thread.State: RUNNABLE "CompilerThread0" daemon prio=10 tid=0x00007f80a8044800 nid=0x3972 waiting on condition [0x0000000000000000..0x00007f80af2461f0] java.lang.Thread.State: RUNNABLE "JDWP Command Reader" daemon prio=10 tid=0x000000004209ac00 nid=0x3970 runnable [0x0000000000000000..0x0000000000000000] java.lang.Thread.State: RUNNABLE "JDWP Event Helper Thread" daemon prio=10 tid=0x00007f80a802f400 nid=0x396e runnable [0x0000000000000000..0x0000000000000000] java.lang.Thread.State: RUNNABLE "JDWP Transport Listener: dt_socket" daemon prio=10 tid=0x00007f80a802bc00 nid=0x396c runnable [0x0000000000000000..0x00007f80af54a750] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x00007f80a801cc00 nid=0x396a waiting on condition [0x0000000000000000..0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=10 tid=0x0000000042093800 nid=0x3969 in Object.wait() [0x00007f80af850000..0x00007f80af850c70] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00007f80b5cc28f8> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116) - locked <0x00007f80b5cc28f8> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) "Reference Handler" daemon prio=10 tid=0x0000000042091800 nid=0x3968 in Object.wait() [0x00007f80af951000..0x00007f80af951cf0] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00007f80b5cc2930> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:485) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) - locked <0x00007f80b5cc2930> (a java.lang.ref.Reference$Lock) "main" prio=10 tid=0x0000000042020c00 nid=0x3959 waiting on condition [0x00007f80f7732000..0x00007f80f7732eb0] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f80de8fd208> (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:905) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1217) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) at org.jboss.profiler.jvmti.JVMTIInterface.startMemoryProfiler(JVMTIInterface.java:975) at org.jboss.messaging.tests.util.UnitTestCase.tearDown(UnitTestCase.java:617) at org.jboss.messaging.tests.integration.cluster.distribution.ClusterTestBase.tearDown(ClusterTestBase.java:114) at org.jboss.messaging.tests.integration.cluster.distribution.MessageRedistributionTest.tearDown(MessageRedistributionTest.java:67) at junit.framework.TestCase.runBare(TestCase.java:136) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:120) at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197) "VM Thread" prio=10 tid=0x000000004208c000 nid=0x3967 runnable "GC task thread#0 (ParallelGC)" prio=10 tid=0x000000004202b800 nid=0x395f runnable "GC task thread#1 (ParallelGC)" prio=10 tid=0x000000004202d400 nid=0x3960 runnable "GC task thread#2 (ParallelGC)" prio=10 tid=0x000000004202ec00 nid=0x3961 runnable "GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000042030800 nid=0x3962 runnable "GC task thread#4 (ParallelGC)" prio=10 tid=0x0000000042032000 nid=0x3963 runnable "GC task thread#5 (ParallelGC)" prio=10 tid=0x0000000042033800 nid=0x3964 runnable "GC task thread#6 (ParallelGC)" prio=10 tid=0x0000000042035400 nid=0x3965 runnable "GC task thread#7 (ParallelGC)" prio=10 tid=0x0000000042036c00 nid=0x3966 runnable "VM Periodic Task Thread" prio=10 tid=0x00007f80a0004800 nid=0x3975 waiting on condition JNI global references: 31824 Heap PSYoungGen total 31424K, used 2359K [0x00007f80de720000, 0x00007f80e0ac0000, 0x00007f80f2c70000) eden space 28864K, 8% used [0x00007f80de720000,0x00007f80de96de48,0x00007f80e0350000) from space 2560K, 0% used [0x00007f80e0350000,0x00007f80e0350000,0x00007f80e05d0000) to space 2816K, 0% used [0x00007f80e0800000,0x00007f80e0800000,0x00007f80e0ac0000) PSOldGen total 77056K, used 2030K [0x00007f80b5c70000, 0x00007f80ba7b0000, 0x00007f80de720000) object space 77056K, 2% used [0x00007f80b5c70000,0x00007f80b5e6b8a8,0x00007f80ba7b0000) PSPermGen total 21248K, used 10560K [0x00007f80b0870000, 0x00007f80b1d30000, 0x00007f80b5c70000) object space 21248K, 49% used [0x00007f80b0870000,0x00007f80b12c0288,0x00007f80b1d30000)
