5 Replies Latest reply on Nov 15, 2011 5:21 AM by clebert.suconic

    java.lang.OutOfMemoryError: GC overhead lim

    srazza

      Hi,

      in our environment, based on standalone HornetQ 2.2.5, we have got this error:

       

      -------------------------------------------------------------------------------------------------------------------------------------------------------------------

      [Old I/O server worker (parentId: 963293928, [id: 0x396ab2e8, /xxx.xxx.xxx.xxx:5445])] 02:30:43,275 WARNING [org.hornetq.core.protocol.core.ServerSessionPacketHandler]  Sending unexpected exception to the client
      java.lang.OutOfMemoryError: GC overhead limit exceeded

      -------------------------------------------------------------------------------------------------------------------------------------------------------------------


      The problem is that the while HornetQ process was still running, the service was not working and all the publishers and the

      subscibers was not able to connect.

      We had to kill the process and restart it to solve the issue.

      What is the cause of this error?

      There are some configuration parameters to activate to avoid this kind of problems?

       

      Best regards

       

      Stefano

        • 1. Re: java.lang.OutOfMemoryError: GC overhead lim
          clebert.suconic

          You are joking, right?

           

           

          You didn't get us any information besides... I have an OME.... how can we help you?

           

           

          If you only send messages to a message system without consuming them.. you will definitely have an OME.

           

           

           

          Look at paging... besides this..

           

          If you think you're hitting a bug you will have to provide us more data...  at least some memory information, what objects are taking most of your memory, a profiler.. anything, otherwise you are just telling us:

           

          - I have a problem, help me please!

           

           

          And I will have to answer you that we don't have how to help you.

           

           

          Sorry about the candid answer, but it's true.. there's little we can do to help you with the information you gave us.

          • 2. Re: java.lang.OutOfMemoryError: GC overhead lim
            srazza

            Hi Clebert,

            sorry for the lack of information.

            We have a production environment with two nodes cluster configuration (active-standby), with about 100 Topics and persistence enabled..

            We start HornetQ we the following JVM parameters:

            -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -Xms10240M -Xmx10240M

             

            In the hornetq-configuration.xml we use this settings:

            ----------------------------------------------------------------------------------------------------------

               <address-settings>

                  <!--default for catch all-->

                  <address-setting match="#">

                     <dead-letter-address>jms.queue.DLQ</dead-letter-address>

                     <expiry-address>jms.queue.ExpiryQueue</expiry-address>

                     <redelivery-delay>0</redelivery-delay>

                     <max-size-bytes>1048576000</max-size-bytes>

                     <page-size-bytes>104857600</page-size-bytes>

                     <message-counter-history-day-limit>10</message-counter-history-day-limit>

                     <address-full-policy>PAGE</address-full-policy>

                     <max-delivery-attempts>-1</max-delivery-attempts>

                  </address-setting>

               </address-settings>

            ----------------------------------------------------------------------------------------------------------

             

            Let me know if you need any other information.

            At the moment the error has happened only one time, but we'd like to undestand the causes, in order to prevent

            other similar situations.

             

            Best Regards

            • 3. Re: java.lang.OutOfMemoryError: GC overhead lim
              leosbitto

              I would add -XX:+HeapDumpOnOutOfMemory and analyze the produced heap dump with the Eclipse memory analyzer (or any other tool, but MAT seems to be the most helpful one to me).

              • 4. Re: java.lang.OutOfMemoryError: GC overhead lim
                clebert.suconic

                I'm not sure how many of these are paging...

                 

                 

                if you get every topic paging, you will need MAX-BYTES * 100 + some other stuff eventually.... I doubt you would have enough memory if everything is paging.

                 

                 

                Please, look at how to take memory snapshots from JVM. That's really what you need to find out what's going on.

                 

                You may add -XX:+PrintClassHistogram  to your VM, and do a kill -3

                • 5. Re: java.lang.OutOfMemoryError: GC overhead lim
                  clebert.suconic

                  Leos Bitto wrote:

                   

                  I would add -XX:+HeapDumpOnOutOfMemory and analyze the produced heap dump with the Eclipse memory analyzer (or any other tool, but MAT seems to be the most helpful one to me).

                   

                  Right.. Also PrintClassHistogram may help you.