0 Replies Latest reply on Mar 3, 2015 3:31 PM by Jalandip Lepcha

    Deadlock in JCA during JDBC pool flush

    Jalandip Lepcha Newbie

      Hi

      I am using Jboss-EAP-6.1 with ironjacamar version 1.0.26.  During an outage of the database application stopped responding. Thread dump shows that there is a in the jca layer. We use autoreconnect and <flush-strategy>EntirePool</flush-strategy>.  Below is the stack trace of the offending threads. Is this a know issue fixed in later version ?

       

      Found one Java-level deadlock:

      =============================

      "http-/0.0.0.0:8080-6":

        waiting to lock monitor 0x0000000001e64678 (object 0x0000000781621fd0, a org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject),

        which is held by "ajp-xxx/10.155.78.226:8009-34"

      "ajp-xxx/10.155.78.226:8009-34":

        waiting to lock monitor 0x00007f7eb4004a88 (object 0x000000079ff923e8, a com.mysql.jdbc.jdbc2.optional.JDBC4MysqlXAConnection),

        which is held by "Thread-1006 (HornetQ-client-global-threads-1821340891)"

      "Thread-1006 (HornetQ-client-global-threads-1821340891)":

        waiting to lock monitor 0x0000000001e64678 (object 0x0000000781621fd0, a org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject),

        which is held by "ajp-xxx/10.155.78.226:8009-34"

       

      ==================================

       

      "http-/0.0.0.0:8080-6":

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.initLock(AbstractPool.java:235)

        - waiting to lock <0x0000000781621fd0> (a org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject)

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getLock(AbstractPool.java:267)

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getTransactionOldConnection(AbstractPool.java:443)

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:369)

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329)

        at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368)

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464)

        at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:139)

        at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:71)

        at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:446)

        at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:167)

       

      "Thread-1006 (HornetQ-client-global-threads-1821340891)":

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.flush(AbstractPool.java:315)

        - waiting to lock <0x0000000781621fd0> (a org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject)

        at org.jboss.jca.core.connectionmanager.listener.AbstractConnectionListener.connectionErrorOccurred(AbstractConnectionListener.java:367)

        at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.connectionErrorOccurred(TxConnectionListener.java:445)

        at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.broadcastConnectionError(BaseWrapperManagedConnection.java:647)

        at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.broadcastConnectionError(XAManagedConnection.java:199)

        at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection$1.connectionErrorOccurred(XAManagedConnection.java:87)

        at com.mysql.jdbc.jdbc2.optional.MysqlPooledConnection.callConnectionEventListeners(MysqlPooledConnection.java:248)

        - locked <0x000000079ff923e8> (a com.mysql.jdbc.jdbc2.optional.JDBC4MysqlXAConnection)

        at com.mysql.jdbc.jdbc2.optional.WrapperBase.checkAndFireConnectionError(WrapperBase.java:61)

        at com.mysql.jdbc.jdbc2.optional.PreparedStatementWrapper.executeQuery(PreparedStatementWrapper.java:858)

        at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:107)

        at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462)

       

       

      "ajp-xxx/10.155.78.226:8009-34":

        at com.mysql.jdbc.jdbc2.optional.ConnectionWrapper.close(ConnectionWrapper.java:830)

        - waiting to lock <0x000000079ff923e8> (a com.mysql.jdbc.jdbc2.optional.JDBC4MysqlXAConnection)

        at com.mysql.jdbc.jdbc2.optional.ConnectionWrapper.close(ConnectionWrapper.java:466)

        at com.mysql.jdbc.jdbc2.optional.JDBC4ConnectionWrapper.close(JDBC4ConnectionWrapper.java:72)

        at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.destroy(BaseWrapperManagedConnection.java:494)

        at org.jboss.jca.adapters.jdbc.xa.XAManagedConnection.destroy(XAManagedConnection.java:225)

        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.doDestroy(SemaphoreArrayListManagedConnectionPool.java:819)

        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.flush(SemaphoreArrayListManagedConnectionPool.java:568)

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.flush(AbstractPool.java:323)

        - locked <0x0000000781621fd0> (a org.jboss.jca.core.connectionmanager.pool.strategy.PoolBySubject)

        at org.jboss.jca.core.connectionmanager.listener.AbstractConnectionListener.connectionErrorOccurred(AbstractConnectionListener.java:367)

        at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.connectionErrorOccurred(TxConnectionListener.java:445)

        at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.broadcastConnectionError(BaseWrapperManagedConnection.java:647)

        at org.jboss.jca.adapters.jdbc.BaseWrapperManagedConnection.checkValid(BaseWrapperManagedConnection.java:522)

        at org.jboss.jca.adapters.jdbc.xa.XAManagedConnectionFactory.matchManagedConnections(XAManagedConnectionFactory.java:497)

        at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:284)

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getTransactionNewConnection(AbstractPool.java:495)

        at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:374)

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329)

        at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368)

        at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464)

        at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:139)

        at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:71)