We have a web-application that we recently migrated from JBoss 4.3 to 5.1GA. The application was developed using JSP,Servlet and EJB2. All Servlet and JSP content is deployed on Tomcat and the EJB content on JBoss. EJB calls are made from Tomcat and those requests are load-balanced to 4 JBoss nodes using mod_jk. The EJBs (SFSB,SLSB,EB) are not clustered and so we use sticky-session at the load-balancer. Recently we upgraded the JBoss servers to 5.1, Tomcat version being the same. Also mod_jk was replaced by mod_cluster as the load_balancer.
Everything was fine during the migration and initial testing. Now the setup was moved to production and users started using it. During peak hours the load average of the database (stand-alone Oracle 11g) installed machine (RHELinux) increases and reaches peaks and JBoss starts throwing TransactionRolledBackExceptions and ORA-00028 errors. The number of connections from JBoss to the database are well below the max-connections defined int the database. When contacted, the DBA said that there was high concurrency in DB requests from the servers which is causing the increase in load-average.
We had to eventually revert to older mod_jk-JBoss 4.3 where the same application did well. But we have to move the mod_cluster-JBoss 5.1setup as soon as possible to production. Please let us know what could have gone wrong. I would post any information that may be needed like logs and configuration parameters used.
