We maintain a legacy (jboss 4.2.3 , ejb 3 ,jsp, jdk 1.6.45 64b ) application.
This application make heavy usage of database, and respond to
- User Http request
- Web service request
- Some CORBA connections
- Some external JSM call
- Some MDB internal process
Http connection pool is standard (tomcat 5.5 with 250 connections) , database pool connection is raised to 300 (instead of 60 standard).
The jboss service.xml (where default thread pool isdefined) is
<mbean code="org.jboss.util.threadpool.BasicThreadPool"name="jboss.system:service=ThreadPool">
<attribute name="Name">JBoss System Threads</attribute>
<attribute name="ThreadGroupName">System Threads</attribute>
<attribute name="KeepAliveTime">60000</attribute>
<attribute name="MaximumPoolSize">10</attribute>
<attribute name="MaximumQueueSize">1000</attribute>
<attribute name="BlockingMode">run</attribute>
</mbean>
One of our customer have some bad performance during part of the day.
There is a lot of threads generated without any relation with the users activities.
Normal activities threads count is 300/400 threads
- Sometimes it grows up to 4 k in seconds and decrease slowly in minutes
- Sometimes it can grows up to 25 k threads and takes hours to decrease and application become unresponsive (need a restart)
Thread dump show that all this threads are RUNNABLE , but there is no stack trace (other "normal" thread show usual stack trace).
All those threads are nammed "Thread-xxxxxx" and belong to the "JBoss Pooled Threads"
We are sure that those threads are not created by the application, but we cannot find what could be responsible for this threads generation.
Any ideas ?
