5 Replies Latest reply on Feb 7, 2013 3:57 PM by viva_div

    jboss as 4.2.2 high cpu usage

    viva_div

      Hi!

      We need some help with our jboss as version 4.2.2

      For unknown reason java process uses 100% CPU after startup which makes applications unusable.

      Platfort is virtualized SLES 10.

      There were no changes made to OS or JBOSS/App.

      Where to start?

       

      Thanks

        • 1. Re: jboss as 4.2.2 high cpu usage
          wdfink

          Might be that you have an issue with GC.

          Check a thread dump whether a GC thread will consume all CPU.

          Switch the gc logging on (see the oracle java documentation for that)

           

          Maybe it is simple and you have less memory configured for the JVM

          • 2. Re: jboss as 4.2.2 high cpu usage
            viva_div

            Looked at heap dump and Permanent space is 99%. Also polling process shows a lot of time accounted for GC.

            I guess you are right on GC issue. Now how do I fix it?

            Tried to bump Xmx and xx:maxpermsize(512mb) but Permanent allocation is non resposive. Stays around 50mb.

            I got this enviroment recently and not well familiar with sun jdk 1.5_15 issues.

            We can throw more memory at it but not sure how.

             

            Thanks

            • 3. Re: jboss as 4.2.2 high cpu usage
              peterj

              Running the permgen at 99% is not a problem - the JVM will usually allocate just barely enough space for it because it is not volatile. So if you say it stays around 50MB, than a maxpermgen of 64MB should be more than enough. (Allocating more can actually work against you because it reduces the amount of memory that can be allocated to other areas of the JVM.)

               

              If you suspect GC is the issue, you need to set one of the GC logging options, such as "-verbose:gc -Xloggc:gc.log", the data will appear in gc.log. Analyzing that data will tell you what is going on. To get some ideas on doing this, see my white papers at:

              http://www.cmg.org/cgi-bin/search.cgi?q=java+peter+johnson&x=30&y=10

              The 101 and 201 papers are handy, as well as the "Analyzing GC with R" (the R script calculates the % of time the process sent in GC).

              • 4. Re: jboss as 4.2.2 high cpu usage
                dvayanu

                I think you should start by creating ThreadDumps and monitoring GC. Monitoring gc is easy, just add following to your start script:

                -XX:+DisableExplicitGC -verbose:GC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:logs/gc.log

                check if you have multiple "Full" gcs which last long time after each other. (grep -e Full gc.log)

                 

                to create ThreadDumps you can send

                kill -QUIT pid on linux

                use jstack <pid> > filename

                or use a tool like moskito (moskito.anotheria.net) which can provide you embedded thread dumps.

                 

                You should create 10 thread dumps in short distance, maybe 3 seconds inbeetween and check for threads that are doing something strange. If you have an infinite loop in your code, you will be able to find it this way.

                 

                regards

                Leon

                • 5. Re: jboss as 4.2.2 high cpu usage
                  viva_div

                  Did thread dump. Nothing special in there.

                  In the end it was a virtualization issue. After restarting vmware host os everything got back to normal. Thanks