Improving speed app with concurrent transactions/hits

[sunkishorej2ee]

Hi guys,
I had been using JBOSS4.2.0GA in my development environ for the past 9-12months....now when concurrent hits reach the app in production environ, it slows down(in fact nothin happens and comes down to a hault for a while i.e., 5-10sec)....I had tried to improve the JVM opts in run.bat(for windows)...like in my run.bat i have the following...

rem JVM memory allocation pool parameters. Modify as appropriate.
set JAVA_OPTS=%JAVA_OPTS% -Xms1024m -Xmx1024m

rem Memory configuration to delay PermGen: OutOfMemoryError(s)
set JAVA_OPTS=-XX:MaxPermSize=1024m %JAVA_OPTS%

rem Garbage Collection configuration to fix PermGen: OutOfMemoryError(s)
set JAVA_OPTS= -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSPermGenSweepingEnabled
-XX:+CMSClassUnloadingEnabled %JAVA_OPTS%

rem With Sun JVMs reduce the RMI GCs to once per hour
set JAVA_OPTS=%JAVA_OPTS% -Dsun.rmi.dgc.client.gcInterval=600000 -Dsun.rmi.dgc.server.gcInterval=600000

Let me be clear , as the concurrent hits increases the hault time becomes more... Thought the above steps could be sufficient for heavy transactions under concurrent usage,but afraid that it doesnt impact much on performance enhancement.....In fact it is fine by large when no concurrent hits even with heavy transaction size....
So, I request you guys to guide me through any options for solving my performance slow down in production environ.....

Thanks and Regards
Kishore

[peterj]

Have you gathered garbage collection statistics using -verbose:gc or -XX:PrintHeapAtGC? Have you analyzed/graphed that data to see what the collector is doing?

What JVM are you using?

For graphing and interpreting GC data, see http://www.cecmg.de/doc/tagung_2007/agenda07/24-mai/2b3-peter-johnson/index.html

[sunkishorej2ee]

Hi Peter,
Thanks very much for your reply.....really sorry from me to reply this much late and to you for late response due to unavoidable reasons....
Frankly speaking regarding your reply about garbage collection statistics using -verbose:gc or -XX:PrintHeapAtGC with a web application.
So I tried out the following JVM options and its working little faster without any stoppage in the application/sys(provided no consecutive hits)

set JAVA_OPTS=%JAVA_OPTS% -Xms1024m -Xmx1024m
set JAVA_OPTS=-XX:MaxPermSize=1024m %JAVA_OPTS%
set JAVA_OPTS= -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSPermGenSweepingEnabled
-XX:+CMSClassUnloadingEnabled %JAVA_OPTS%

problem arises when after sometime of concurrent hits system strucks up for a while without knowing for the customer...hope i am clear about my req.....request you if u could provide some suggestions regarding the same.....
Our system is a client side installable web app with JBOSS 4.2.0GA and MYSql 5.0.45-community-nt....Although client allocates one dedicated server for the entire app, his RAM is gonna be 1GB max.....regarding this i have goine through this following post where u mentioned abt the combination of Jboss and mysql
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=133322


Thanks and Regards
Kishore

[peterj]

For the memory settings you have, the client needs to have 2GB RAM. The combination of heap size and permgen size is too large for a 1GB machine. Windows itself will use up about 1/2GB, leaving only 1/2GB for your app, so you will find that the system is doing a lot of thrashing (swapping memory areas between RAM and the swap file).

If you are seeing large pauses when using the concurrent collector, you are probably running into a full collection because the concurrent collector is falling behind on cleaning the heap. If this is the case, it could indicate a memory leak in your application. The GC statistics should be able to tell you what is going on.

How many CPUs (or cores) are on this machine? I hope at least 2, but 4 would be better.

[sunkishorej2ee]

Hi Peter,
Once again thx very much for your reply...
As you've given that the mem settings are too high for my client who would be havin a max of 1GB....So in such cases using concurrent collector would lead to memory leaks....I really hope if i change my mem settings it would affect the performance again slowing it (for single user at a time)drastically apart from concurrent hits slowness.....
So, I juz wanna ask u or any of the guys watchin this post here, to give me ur valuable suggestions/links on how to collect gc details on a web application without doin much with the app....mean to say the configurations/steps i need to do for getting the GC details output for my web app at run time....Really I dont have any exposure till now about how to get GC details...plz do guide me in this....

Thanks and Regards
Kishore

[peterj]

You might want to consider hiring someone to do this for you, it can get fairly complicated if you have never done this before.

Also, I do have another resource that has more details about tuning, but that resource is not free. I can provide the URL if you like.

[sunkishorej2ee]

Hi Peter,
Once again thx very much for your reply.....yes i would definitely like to know the url....

Thanks and Regards
Kishore

[peterj]

Chapter 14 of JBoss in Action (http://manning.com/jamae) is all about performance tuning. While the book is geared towards JBossAS 5.0, most of chapter 14 applies to any Java application, including JBoss AS 4.2.x.