0 Replies Latest reply on Feb 13, 2009 7:28 PM by Clebert Suconic

    Dead Lock on a failover test

    Clebert Suconic Master

      I got a dead lock during a testsuite run. I was doing some other stuff (had a KVM with window$ running) what may had made the behavior worse, but a dead lock shouldn't happen.


      It happened once during org.jboss.messaging.tests.integration.cluster.failover.FailoverExpiredMessageTest


      [junit] Found one Java-level deadlock:
       [junit] =============================
       [junit] "Thread-6236":
       [junit] waiting to lock monitor 0x00007fdcea027d10 (object 0x00007fdd25a50e28, a java.lang.Object),
       [junit] which is held by "Thread-6235"
       [junit] "Thread-6235":
       [junit] waiting to lock monitor 0x00007fdcea027e00 (object 0x00007fdd25a5f790, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection),
       [junit] which is held by "Thread-6236"
       [junit]
       [junit] Java stack information for the threads listed above:
       [junit] ===================================================
       [junit] "Thread-6236":
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866)
       [junit] - waiting to lock <0x00007fdd25a50e28> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdd25a5f790> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit] "Thread-6235":
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87)
       [junit] - waiting to lock <0x00007fdd25a5f790> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.destroy(RemotingConnectionImpl.java:470)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failover(ConnectionManagerImpl.java:587)
       [junit] - locked <0x00007fdd25a5f8b8> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionFailed(ConnectionManagerImpl.java:412)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.callListeners(RemotingConnectionImpl.java:574)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:453)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870)
       [junit] - locked <0x00007fdd25a50e28> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdd25a5fac0> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit]
       [junit] Found one Java-level deadlock:
       [junit] =============================
       [junit] "Thread-1433":
       [junit] waiting to lock monitor 0x00007fdce92c0ab0 (object 0x00007fdcf0dbe090, a java.lang.Object),
       [junit] which is held by "Thread-1432"
       [junit] "Thread-1432":
       [junit] waiting to lock monitor 0x00007fdce92c0c90 (object 0x00007fdcf0dbe1c0, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection),
       [junit] which is held by "Thread-1433"
       [junit]
       [junit] Java stack information for the threads listed above:
       [junit] ===================================================
       [junit] "Thread-1433":
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866)
       [junit] - waiting to lock <0x00007fdcf0dbe090> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdcf0dbe1c0> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit] "Thread-1432":
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87)
       [junit] - waiting to lock <0x00007fdcf0dbe1c0> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:455)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870)
       [junit] - locked <0x00007fdcf0dbe090> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdcf0eabc20> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit]
       [junit] Found one Java-level deadlock:
       [junit] =============================
       [junit] "Thread-1355":
       [junit] waiting to lock monitor 0x00007fdcea4eb748 (object 0x00007fdcf0eadc30, a java.lang.Object),
       [junit] which is held by "Thread-1354"
       [junit] "Thread-1354":
       [junit] waiting to lock monitor 0x00007fdcea4eb838 (object 0x00007fdcf0eadd60, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection),
       [junit] which is held by "Thread-1355"
       [junit]
       [junit] Java stack information for the threads listed above:
       [junit] ===================================================
       [junit] "Thread-1355":
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866)
       [junit] - waiting to lock <0x00007fdcf0eadc30> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdcf0eadd60> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit] "Thread-1354":
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87)
       [junit] - waiting to lock <0x00007fdcf0eadd60> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:455)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870)
       [junit] - locked <0x00007fdcf0eadc30> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdcf0ec6058> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit]
       [junit] Found one Java-level deadlock:
       [junit] =============================
       [junit] "Thread-471":
       [junit] waiting to lock monitor 0x0000000040401d60 (object 0x00007fdceffad6b8, a java.lang.Object),
       [junit] which is held by "Thread-470"
       [junit] "Thread-470":
       [junit] waiting to lock monitor 0x0000000040401f40 (object 0x00007fdceffad7e8, a org.jboss.messaging.core.remoting.impl.invm.InVMConnection),
       [junit] which is held by "Thread-471"
       [junit]
       [junit] Java stack information for the threads listed above:
       [junit] ===================================================
       [junit] "Thread-471":
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:866)
       [junit] - waiting to lock <0x00007fdceffad6b8> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdceffad7e8> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit] "Thread-470":
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:87)
       [junit] - waiting to lock <0x00007fdceffad7e8> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:601)
       [junit] at org.jboss.messaging.core.remoting.impl.RemotingConnectionImpl.fail(RemotingConnectionImpl.java:455)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.failConnection(ConnectionManagerImpl.java:870)
       [junit] - locked <0x00007fdceffad6b8> (a java.lang.Object)
       [junit] at org.jboss.messaging.core.client.impl.ConnectionManagerImpl.connectionDestroyed(ConnectionManagerImpl.java:198)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:187)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:97)
       [junit] - locked <0x00007fdceffb6b98> (a org.jboss.messaging.core.remoting.impl.invm.InVMConnection)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMConnector.disconnect(InVMConnector.java:161)
       [junit] at org.jboss.messaging.core.remoting.impl.invm.InVMAcceptor$Listener$1.run(InVMAcceptor.java:167)
       [junit]
       [junit] Found 4 deadlocks.
       [junit]