Dead Lock on a failover test
I got a dead lock during a testsuite run. I was doing some other stuff (had a KVM with window$ running) what may had made the behavior worse, but a dead lock shouldn't happen.
It happened once during org.jboss.messaging.tests.integration.cluster.failover.FailoverExpiredMessageTest
[junit] Found one Java-level deadlock: [junit] ============================= [junit] "Thread-6236": [junit] waiting to lock monitor 0x00007fdcea027d10 (object 0x00007fdd25a50e28, a java.lang.Object), [junit] which is held by "Thread-6235" [junit] "Thread-6235": [junit] waiting to lock monitor 0x00007fdcea027e00 (object 0x00007fdd25a5f790, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection), [junit] which is held by "Thread-6236" [junit] [junit] Java stack information for the threads listed above: [junit] =================================================== [junit] "Thread-6236": [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866) [junit] - waiting to lock <0x00007fdd25a50e28> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdd25a5f790> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] "Thread-6235": [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87) [junit] - waiting to lock <0x00007fdd25a5f790> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.destroy(RemotingConnectionImpl.java:470) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failover(ConnectionManagerImpl.java:587) [junit] - locked <0x00007fdd25a5f8b8> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionFailed(ConnectionManagerImpl.java:412) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.callListeners(RemotingConnectionImpl.java:574) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:453) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870) [junit] - locked <0x00007fdd25a50e28> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdd25a5fac0> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] [junit] Found one Java-level deadlock: [junit] ============================= [junit] "Thread-1433": [junit] waiting to lock monitor 0x00007fdce92c0ab0 (object 0x00007fdcf0dbe090, a java.lang.Object), [junit] which is held by "Thread-1432" [junit] "Thread-1432": [junit] waiting to lock monitor 0x00007fdce92c0c90 (object 0x00007fdcf0dbe1c0, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection), [junit] which is held by "Thread-1433" [junit] [junit] Java stack information for the threads listed above: [junit] =================================================== [junit] "Thread-1433": [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866) [junit] - waiting to lock <0x00007fdcf0dbe090> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdcf0dbe1c0> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] "Thread-1432": [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87) [junit] - waiting to lock <0x00007fdcf0dbe1c0> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:455) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870) [junit] - locked <0x00007fdcf0dbe090> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdcf0eabc20> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] [junit] Found one Java-level deadlock: [junit] ============================= [junit] "Thread-1355": [junit] waiting to lock monitor 0x00007fdcea4eb748 (object 0x00007fdcf0eadc30, a java.lang.Object), [junit] which is held by "Thread-1354" [junit] "Thread-1354": [junit] waiting to lock monitor 0x00007fdcea4eb838 (object 0x00007fdcf0eadd60, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection), [junit] which is held by "Thread-1355" [junit] [junit] Java stack information for the threads listed above: [junit] =================================================== [junit] "Thread-1355": [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866) [junit] - waiting to lock <0x00007fdcf0eadc30> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdcf0eadd60> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] "Thread-1354": [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87) [junit] - waiting to lock <0x00007fdcf0eadd60> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:455) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870) [junit] - locked <0x00007fdcf0eadc30> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdcf0ec6058> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] [junit] Found one Java-level deadlock: [junit] ============================= [junit] "Thread-471": [junit] waiting to lock monitor 0x0000000040401d60 (object 0x00007fdceffad6b8, a java.lang.Object), [junit] which is held by "Thread-470" [junit] "Thread-470": [junit] waiting to lock monitor 0x0000000040401f40 (object 0x00007fdceffad7e8, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection), [junit] which is held by "Thread-471" [junit] [junit] Java stack information for the threads listed above: [junit] =================================================== [junit] "Thread-471": [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866) [junit] - waiting to lock <0x00007fdceffad6b8> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdceffad7e8> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] "Thread-470": [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87) [junit] - waiting to lock <0x00007fdceffad7e8> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601) [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:455) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870) [junit] - locked <0x00007fdceffad6b8> (a java.lang.Object) [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97) [junit] - locked <0x00007fdceffb6b98> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161) [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167) [junit] [junit] Found 4 deadlocks. [junit]