JBossDeveloper

I don't think it's the case. The slow down occurs after some time the server was started. For example now, I have this war file with my code posted in the first message here. The Jboss is running for several days and it's really slow. Even if I undeploy the main application, the server remains slow. If I try to deploy it back, the deploy takes very long time. However, if I just shut down the JBoss and start it again, it will be fast.

I wouldn't say that whole virtual machine is running slow. If I run the code above in seperate JVM (just compile it with javac and run it) it's fast. This is bizarre.

Here's yet another observation: when the server is after fresh start and I run the bubble sort, It utilizes 100% of one processor core. It never switches to another core or drops utilization below 95%. However after some time the server is running and I'm experiencing the performance problems, the method above is utilizing CPU core usually 100%, however I just found out from htop that this task is being switched very often to other cores. That is, at the beginning it's running on core #1, after say 2 seconds it's running on #5 then after say 2 seconds #8 etc. Furthermore, the utilization is not kept at 100% at the core but sometimes drops to 80% or even lower.

For the server after fresh start, even though If I simulate a load, it never switches the task to another core.

Is this a symptom of something?

It doesn't seem that that's the case. If I run the bubble sort in separate JVM (by running java PerformanceTest), it's behaving normally, it's fast, it's not switching processors. If there is a problem with VM, it should be a global problem. But this is issue only with Jboss 7 JVM. I was looking at the VMWare data with system administrator today and it doesn't show any issue. RAM/CPU are basically dedicated only to the machine where JBoss is running. It doesn't use swap either.

What about this data from top command for jboss 7 process?

VIRT 9093m, RES 2.6g, SHR 20m, %MEM 16.8

I believe this is normal.

Yes, it does. It's working correctly

Here are JVM parameters;

-Xms1280m -Xmx2048m -XX:MaxPermSize=2048m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Duser.timezone=UTC -Djboss.server.default.config=standalone-full.xml -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n 

Thanks, I will do that.

Here is the thread dump: https://dl.dropbox.com/u/51364356/threadDump.txt

The code is executed in  blah.blahhh.data.PerformanceTest.bubbleSort not that the thread is either in IN_VM state or IN_JAVA, it changes over time.

So we are upragded application server from JBoss 7.1.1.Final to SNAPSHOT of JBoss AS 7.1.4 and that issue didn't show up again. We are running it for a month right now. So the problem was with JBoss AS 7.1.1.Final.

Thanks for your help