jboss 4.2.3 Connection Pool Leak

[paata]

Hi all,

My Env :

     1. JBoss AS : 4.2.3.GA

     2. Database : Oracle 10g

     3. OS             : Sun Solaris

     4. Java          : BEA JRockit(R)  java version "1.6.0_05"

     5. JAVA OPTS : "-server -Xmx4g -Xms4g -d64 -Xnoopt -XXnojitinline -XgcPrio:deterministic -Xpausetarget=40ms"

I've two databases with <xa-datasource> and one database with <local-tx-datasource> . Sample xa datasouce looks like:

 <xa-datasource>
        <jndi-name>XAOracleDS</jndi-name>
        <track-connection-by-tx/>
        <isSameRM-override-value>false</isSameRM-override-value>
        <xa-datasource-class>oracle.jdbc.xa.client.OracleXADataSource</xa-datasource-class>
        <xa-datasource-property name="URL">jdbc:oracle:thin:@XXX.XXX.XXX.XXX:1521:STR</xa-datasource-property>
        <xa-datasource-property name="User">***</xa-datasource-property>
        <xa-datasource-property name="Password">***</xa-datasource-property>
        <min-pool-size>50</min-pool-size>
        <max-pool-size>100</max-pool-size>
        <track-statements>true</track-statements>
        <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.OracleExceptionSorter</exception-sorter-class-name>
        <no-tx-separate-pools/>
        <application-managed-security>true</application-managed-security>
        <metadata>
                <type-mapping>Oracle9i</type-mapping>
        </metadata>
  </xa-datasource>

Sample local datasource

<local-tx-datasource>
    <jndi-name>TimesTen1</jndi-name>
    <connection-url>jdbc:timesten:client:RTPPS1</connection-url>
    <driver-class>com.timesten.jdbc.TimesTenDriver</driver-class>
    <user-name></user-name>
    <password></password>
    <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
    <min-pool-size>3</min-pool-size>
    <max-pool-size>5</max-pool-size>
    <blocking-timeout-millis>5000</blocking-timeout-millis>
    <idle-timeout-minutes>15</idle-timeout-minutes>
    <prepared-statement-cache-size>32</prepared-statement-cache-size>
        <query-timeout>6000</query-timeout>
    </set-tx-query-timeout>
    <track-connection-by-tx/>
    <metadata>
       <type-mapping>TimesTen</type-mapping>                                                                                                                 
    </metadata>
  </local-tx-datasource>

I have approximately 200 messages (operations) per second. After a few days or maybe couple of hours  by xa connection pool size achieve to maximum and jboss doesn't work as it should, however after invoking flush method on ManagedConnectionPool throgh jmx console, pool size decreases and application server continues to work.

For better explanation I've attached a graph (Vertical - Number of connections in use. Horizontal - time.):

At the same time, there is only one active connection in database

entry from arjuna log:

[com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] 
Could not find new XAResource to use for recovering non-serializable XAResource

We restart jboss every 2 days to keep it running. Any help is appreciated.

Thanks in advance,

Paata Lominadze

[paata]

hello team again.

Can anybody help ?

More detail Description.

I have 2 Oracle Database with XA

     1. 10g - Version : 10.2.0.3.0 (64 bit Solaris)

     2. 9i  - Version : 9.2.0.6.0 (64 bit Solaris)

And Timesten In-Memory Databe with local-tx-datasource (I can't use XA with Timesten , because of Timesten DB Configuration)

     1. TimesTen Release 7.0.2.0.0 (64 bit Solaris) (tt70:17001) 2007-05-05T21:24:04Z

I've read many documentations. Ho i understand there is no problem to use one non-xa and multiple xa database (LRCO is implemented in jboss as by default).

I found some points of problem : Our database Admin makes database backups every 3 hour(I mean firs oracle database - 10g - it is our main database). When database backup is running my InUseConnectionCount for this database increases from  0-4 to 40-45.

and when database backup finished InUseConnectionCount never going down.

We have steal connection to this database.

I've read many forums and many blogs where described this problem but i can't resolve my problem till now.

I've changed some jboss as configuration parameters.

     1.jbossjta-properties.xml

     <properties depends="arjuna" name="jta">                                                                                                                                       
         <!-- specifying XA recovery for Oracle -->                                                                                                                                
         <property name="com.arjuna.ats.jta.recovery.XAResourceRecoveryJDBC" value="com.arjuna.ats.internal.jbossatx.jta.AppServerJDBCXARecovery;XAOracleDS"/>                     
         <property name="com.arjuna.ats.jta.recovery.XAResourceRecoveryJDBC1" value="com.arjuna.ats.internal.jbossatx.jta.AppServerJDBCXARecovery;XAOracleOldBillingDS"/>          
         <property name="com.arjuna.ats.jta.recovery.XAResourceRecoveryJDBC2" value="com.arjuna.ats.internal.jbossatx.jta.AppServerJDBCXARecovery;XAOracleScratchDS"/>             
         <property name="com.arjuna.ats.jta.xaRecoveryNode" value="*"/>                                                                                                            
        <!--                                                                                                                                                                       
        Support subtransactions in the JTA layer?                                                                                                                                  
        Default is NO.                                                                                                                                                             
      -->                                                                                                                                                                          
        <property name="com.arjuna.ats.jta.recovery.XAResourceRecovery.JBMESSAGING1" value="org.jboss.jms.server.recovery.MessagingXAResourceRecovery;java:/DefaultJMSProvider"/>  
        <property name="com.arjuna.ats.jta.supportSubtransactions" value="NO"/>                                                                                                    
        <property name="com.arjuna.ats.jta.jtaTMImplementation" value="com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionManagerImple"/>                               
        <!--                                                                                                                                                                       
                        com.arjuna.ats.internal.jta.transaction.jts.TransactionManagerImple                                                                                        
                        -->                                                                                                                                                        
        <property name="com.arjuna.ats.jta.jtaUTImplementation" value="com.arjuna.ats.internal.jta.transaction.arjunacore.UserTransactionImple"/>                                  
        <!--                                                                                                                                                                       
                        com.arjuna.ats.internal.jta.transaction.jts.UserTransactionImple                                                                                           
                        -->                                                                                                                                                        
    </properties>

Also i tried to set some connection parameters into my oracle connection configuration file:

     1. oracle-xa-ds.xml

        <idle-timeout-minutes>15</idle-timeout-minutes>                                                                                                                            
        <blocking-timeout-millis>20000</blocking-timeout-millis>

But nothing can help me.

I think that Recovery Manager can't release steal connection.

Also we can't move from JBoss 4.2.3.GA to JBoss 5.X.X for some reasons.

Any Idea will be appreciated.

Thanks in advance.

_________________

Paata Lominadze.

[paata]

Latest JBoss 4.3 is working fine.

Regards.

Paata Lominadze.