Performance issues on JBoss 5

[croy]

Hi,

Quick question on performance of JBoss 5. We are currently moving our products from jboss 4.0.5 to jboss 5.0.0.

The load average on the production machine for the same code base is giving us reason for concerns.

Has anyone else had issues with performance?
Any pointers on performance ?
we currently use these runtime setting for JBoss

-Xms2048m -Xmx2048m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djava.net.preferIPv4Stack=true

More details follow, we appreciate any help you can give us.

With JBoss 4.0.5 we see the load average of 0.3 and cpu usage around 15%.

With JBoss 5.0.0 we see the load average of 2.0 and cpu usage around 60%, with spikes to 100% CPU usage and load average over 10.

Machine spec info:

quad core xeon.
top - 14:22:43 up 139 days, 6:54, 3 users, load average: 0.83, 0.77, 0.77
Tasks: 101 total, 1 running, 100 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.2%us, 0.2%sy, 0.0%ni, 98.5%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3631912k total, 3507244k used, 124668k free, 9016k buffers
Swap: 4192956k total, 211100k used, 3981856k free, 1422380k cached

processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.00GHz
stepping : 10
cpu MHz : 2992.943
cache size : 2048 KB

processor : 1
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.00GHz
stepping : 10
cpu MHz : 2992.943
cache size : 2048 KB

processor : 2
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.00GHz
stepping : 10
cpu MHz : 2992.943
cache size : 2048 KB

processor : 3
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.00GHz
stepping : 10
cpu MHz : 2992.943
cache size : 2048 KB

[mwohlf]

I am also having performance issues, my machine is

Ubuntu 8.10 2.6.27-9-server #1 SMP
AMD Athlon(tm) 64 Processor 3700+
1 GByte memory

Running JBoss AS5, jboss-seam-2.1.1.GA, richfaces-ui-3.3.0.GA and a simple hello world program with some JTA stuff.

Performance is OK right after restart, but the performance degrades after running for several hours, what I tried so far to speedy things up:

* removed most of the additional stuff form /deploy
* removed the hypersonic default DB & quartz service
* in jboss-service.xml disabled RMI and the AJP connector
* in web.xml set the org.ajax4jsf.xmlparser.ORDER parameter to NONE
* in components.xml set <core:init debug="false">

I did some debug logging which seems to indicate the problem could be the VFSScanner and/or the HotDeployment feature since I loose 2-3 sec there for each request:

11:58:23,891 DEBUG [org.ajax4jsf.webapp.BaseFilter] (http-xxx-80-4) Finished request processing total time 75ms for uri: /charms/a4j/g/3_3_0.GAjquery.js
11:58:24,136 DEBUG [org.jboss.seam.init.Initialization] (http-xxx-80-5) Using Java hot deploy
11:58:25,484 DEBUG [org.jboss.seam.deployment.DeploymentStrategy] (http-xxx-80-5) Using org.jboss.seam.integration.jbossas.vfs.VFSScanner@6616b3
11:58:26,081 DEBUG [org.jboss.seam.deployment.DeploymentStrategy] (http-xxx-80-5) Adding org.jboss.seam.bpm.PageflowDeploymentHandler as a deployment handler
11:58:26,497 DEBUG [org.jboss.seam.deployment.DeploymentStrategy] (http-xxx-80-5) Using org.jboss.seam.integration.jbossas.vfs.VFSScanner@f1f2f3
11:58:26,788 DEBUG [org.jboss.seam.util.Resources] (http-xxx-80-5) Loaded resource from servlet context: jndi:/localhost/charms/pages/admin/charmsRoleList.page.xml

One thing that might be related is that I also found hundreds of files in my tmp/vfs-nested.tmp directory like described here:
http://ourcraft.wordpress.com/2009/01/05/plague-of-jar-files/

now I wonder:
* how to turn off the hot deployment feature?
* is there a way to run without the VFSScanner?
since one of them seems to really mess up the performance on my system.

Any help or comments on this are very much appreciated.

[alesj]

"mwohlf" wrote:

One thing that might be related is that I also found hundreds of files in my tmp/vfs-nested.tmp directory like described here:
http://ourcraft.wordpress.com/2009/01/05/plague-of-jar-files/

I have a much better working solution here:
- http://www.jboss.com/index.html?module=bb&op=viewtopic&t=148563
but as you can see, it requires the usage of snapshots - vfs and deployers (which I already put into our mvn repo).

"mwohlf" wrote:

* how to turn off the hot deployment feature?

This is really for Seam guys to answer.

"mwohlf" wrote:

* is there a way to run without the VFSScanner?

I doubt it, since vfs protocol is used all over JBoss5.

[pmuir]

"alesj" wrote:

"mwohlf" wrote:

* how to turn off the hot deployment feature?

This is really for Seam guys to answer.

Set debug to false in components.xml