war (jetty) feature consumes ~100% cpu without load
transporeon-ernst May 11, 2011 3:10 AMhi,
we had this situation a few times, the servicemix java process consumes nearly 100% cpu.
this happend a few times on productive systems, but also on internal test and my workstation.
it happend again this night on my local system, i started servicemix yesterday, installed one of our feature for debugging and let it run over night. there was no load in this time.
i tried to solve this problem like i did once before (with version 4.3.0-fuse-03-00), i uninstalled features one by one to find out what could cause the problem. uninstalling the war feature fixed in once on my workstation, the process stopped to consume cpu power. but
it didn't work today, the process just hangs, no more logs, no connection via jmx. i attached a screenshot of jconsole before it lost the connection.
here is what i did in the karaf console
karaf@esb01> features:list -i
State Version Name Repository Description
spring karaf-2.2.0-fuse-SNAPSHOT
spring-web karaf-2.2.0-fuse-SNAPSHOT
spring-dm karaf-2.2.0-fuse-SNAPSHOT
spring-dm-web karaf-2.2.0-fuse-SNAPSHOT
config karaf-2.2.0-fuse-SNAPSHOT
jetty karaf-2.2.0-fuse-SNAPSHOT
jetty-jaas karaf-2.2.0-fuse-SNAPSHOT
http karaf-2.2.0-fuse-SNAPSHOT
management karaf-2.2.0-fuse-SNAPSHOT
activemq activemq-5.5-fuse-SNAPSHOT
activemq-blueprint activemq-5.5-fuse-SNAPSHOT
transaction karaf-enterprise-2.2.0-fuse-SNAPSHOT OSGi Transaction Manager
karaf@esb01> features:uninstall war
org.ops4j.pax.web.service.jetty.internal.JettyServerWrapper@7f188fd STOPPED
+-qtp1351746529{8<=2<=8/254,0}
+-org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection@2589e1e started
the output after the command i saw for the first time
and than nothing , i had to kill the process.
there was no trace of a problem in the log file.
i'm currently using apache-servicemix-4.4.0-fuse-20110405.154730-4
i run 1 cxf soap webservice in an embedded jetty, and a vaadin webapp in the osgi http service.
karaf@esb01> info
Karaf
Karaf home E:\srv\apache-servicemix-4.4.0-fuse-SNAPSHOT
Karaf base E:\srv\apache-servicemix-4.4.0-fuse-SNAPSHOT
OSGi Framework org.eclipse.osgi - 3.6.0.v20100517
JVM
Java Virtual Machine Java HotSpot(TM) 64-Bit Server VM version 19.0-b09
Version 1.6.0_23
Vendor Sun Microsystems Inc.
Uptime 5.611 seconds
Total compile time 9.397 seconds
Threads
Live threads 37
Daemon threads 32
Peak 37
Total started 40
Memory
Current heap size 145,938 kbytes
Maximum heap size 734,016 kbytes
Committed heap size 734,016 kbytes
Pending objects 0
Garbage collector Name = 'ParNew', Collections = 0, Time = 0.000 seconds
Garbage collector Name = 'ConcurrentMarkSweep', Collections = 2, Time = 0.000 seconds
Classes
Current classes loaded 3,424
Total classes loaded 3,426
Total classes unloaded 0
Operating system
Name Windows 7 version 6.1
Architecture amd64
Processors 4
my JAVA_OPTS
-server -Xmx768m -Xms768m -Xmn512m -Xss128k -XX:MaxPermSize=192M -XX:ParallelGCThreads=20 -XX:UseLargePages -XX:UseConcMarkSweepGC -XX:UseParNewGC -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:AggressiveOpts
i think this will not be easy to reproduce, i just wanted to let you know that there is a problem.
something offtopic, i don't think this could solve the problem, but when will fuse esb 4.4 be released? (hopefully bundled with camel 2.7.1).
the wiki page is outdated
http://fusesource.com/wiki/display/FUSEESB/ESBReleases
thanks
-
jconsole.gif 56.6 KB