JBoss Connection pool not recovered after a database outage with back ground validation enabled

[kodali.rameshbabu]

Hi,

To give a littler backround of the problem,

Recently we had a data base outage in production due to some  locks applied on tables(due to abnormal termination of DB update jobs)  and it causes  all the connections in JDBC connection pool waiting at data base and waiting to get hold of tables.  After some time, it leads all the connections in the connection pool got exhausted and no more conenctions in the pool.  Aplication server started throrwing connection errors for new coming request as no more conenctions available in pool.

DBA killed those waiting threads and restarted data base instance.  Though after DB servers restarts,  still App servers can not recover the connections in connection pool and it forces us to restart App servers too.

After App servers restarts, problem solved.

Now question is:  Why JBoss App servers required to restart after DB  servers are back up?  Will JBoss connection pool does not have mechanism to recover connections automatically?

P.S:  Other team in our company not enabled the back ground validation in their ds.xml, their App servers does not require restart after DB back up.  Their App servers picked up connections automatically.

What makes the back ground validation different here?

Your valueble inputs are much apprecitated. Also if it a problem,  can you please suggest the solution to fix this.

We are using JBoss AS 5.0

Here below our JBoss ds.xml configuration:

We are using the Informix data base.

<local-tx-datasource>

        <jndi-name>JNDIName</jndi-name>

        <connection-url>Connection_URL</connection-url>

        <driver-class>com.informix.jdbc.IfxDriver</driver-class>

        <!-- Use the security domain defined in conf/login-config.xml -->

        <security-domain>Sample-POLICY</security-domain>

        <depends>jboss.security:service=JaasSecurityDomain,domain=ServerMasterPassword</depends>

        <new-connection-sql>SET LOCK MODE TO WAIT </new-connection-sql>

        <check-valid-connection-sql>select * from sysmaster:sysdual</check-valid-connection-sql>

        <min-pool-size>5</min-pool-size>

        <max-pool-size>200</max-pool-size>

        <idle-timeout-minutes>3</idle-timeout-minutes>

        <background-validation-millis>20000</background-validation-millis>

        <validate-on-match>false</validate-on-match>

        <query-timeout>15</query-timeout>

        <set-tx-query-timeout>true</set-tx-query-timeout>

        <prepared-statement-cache-size>20</prepared-statement-cache-size>

        <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.InformixExceptionSorter

        </exception-sorter-class-name>

        <metadata>

            <type-mapping>InformixDB</type-mapping>

        </metadata>

</local-tx-datasource>

[baijupadmanabhan]

This seems not be a problem. In almost all kinds of app servers we face this. The app server might have went in a hung state to send further request to db.

[kodali.rameshbabu]

No, this behavior is not showing always.  Asd I mentioned in my original post, if back ground validation is not turned on, then connection pool recovered automatically.

We have already seen this behaviour in production with both the cases.I am curoius to know the difference.