JBOSS 3.2.6RC2 mySQL performance issues

[tsnoke]

Hello,

We are trying to move our production environment application from JBoss 3.2.1 to 3.2.6RC2 to take advantage of Tomcat 5, et.al.

We are using a two server configuration with JBoss/Tomcat running on one and a mySQL running on the other.

The old config used:
JBoss 3.2.1
Java SDK 1.4.2_05
Red Hat Enterprise Server ES 3.0 Update 3
MySQL 4.0.21

The new config uses:
JBoss 3.2.6RC2
Java SDK 1.4.2_05
Red Hat Enterprise Server ES 3.0 Update 3
MySQL 4.0.21

We have an identical config for both 3.2.1 and 3.2.6RC2 on a robust server and we have developed a stress test that goes through a strictly defined process of accessing a series of webpages to emulate a user. some of the pages are exlusively on the JBoss server and some require sql calls to the other server. A single thread is a process composed of 12 tests that includes 3 sql calls. In our testing we instantiate 50 threads simultaneously and watch the stats on each test: mean time per thread, mean threads per second, max threads per second.

when we run stress tests we find that simple web acesses are much faster in 3.2.6RC2 (2 or 3 times), but every page that makes a database query, to the different server, is much slower (3 to 6 times).

our overall transactions per second is droped to 1/3 the volume that 3.2.1 can handle in a 15 minute period.

what can we do to troubleshoot the mysql integration into 3.2.6RC2, we want to take advantage of tomcat5, and 3.2.1 has performance issues that develop in longer stress tests.

thanks in advance,

Timur

[starksm64]

What profiling of the sql pages has been done either via logging or with a profiler?

[tsnoke]

We haven't done any profiling, this was working in 3.2.1 just fine.

The only change is running it under 3.2.6RC2.

If profiling would help, I am not sure where to go with this... but if you could point me at a FAQ for setting it up and I will post the results.

Thanks for looking at this,

Timur

[starksm64]

Go get a profiler. The following one is easy to use with jboss:
http://www.yourkit.com/download/index.jsp

Otherwise enable trace level logging of the org.jboss category in the log4j.xml file and be sure to include the thread id using the %t option so that you can identify where the time in a request is spent.

[genman]

The JBoss profiler is pretty good too, if you can get it working.

Refer to
http://sourceforge.net/cvs/?group_id=22866

This should work:
$ cvs -z3 -d:pserver:[e-mail]:/cvsroot/jboss co jboss-profiler

Take a look at readme.txt

[tsnoke]

This was very instructive, by looking at the traces we found a problem with our mySQL keep alive...

instead of an "are you there" we were asking for a complete dump of the DB.

so now we are trying to continue to squeeze every last bit of performance out of this solution, playing with the acceptCount and the maxProcessors to try to get the most robust results. Is there a best practice for setting these numbers? we want to be able to handle 100K+ users an hour.

Thanks in Advance,

-Timur