14 Replies Latest reply on Jan 19, 2010 4:43 AM by Jeff Mesnil

    many org.hornetq.core.postoffice.impl.AddressImpl objects

    Vladimir Prodan Newbie

      Hi,

       

      I have a program who sends messages with temporary queues inside and waits for an answer written in the temporary queue - after this it closes the consumer, deletes the temporary queue - anyway after a time I notice the memory usage increases and when I look I see a lot of AddressImpl objects - could be that this is in connection with the temporary queue and this is somehow not cleared?

       

      Thanks a lot.

       

      BTW: I'm using 2.0.0.BETA5

        • 1. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
          Tim Fox Master

          Can you post a test program that demonstrates the issue?

           

          BTW Creating a temp queue for each request-response is a very common anti-pattern. Don't do it, your system will perform very poorly!

           

          This is mentioned in the user manual section on performance tuning.

          • 2. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
            Vladimir Prodan Newbie

            Hi,

            I will try to provide a testcase. Regarding the temporary queue I followed the request-response example - I'll try to reuse the temporary queue.

            Thank you for the answer

            • 3. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
              Tim Fox Master

              In the example, of course the session, producer, consumer and temp queue need to be re-used, sorry if we didn't make that clear in the example.

               

              Also you can get rid of the temp queue altogether and use correlation id and a map.

               

              There are many examples of this on the web if you google it.

              • 4. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                Vladimir Prodan Newbie

                Of course you are right, makes sense to not create every time a new queue and I'm reusing now the queues but in my prototype I came first to the memory problem :-)

                I attached a small program that reproduces this problem: I executed this with -Xmx32m and after a short time I start to receive messages like:

                 

                17.12.2009 13:07:18 org.hornetq.core.logging.Logger warn
                WARNUNG: Less than 25%
                free memory:      1,22 MiB
                max memory:       30,88 MiB
                total memory:     30,75 MiB
                available memory: 4,37%

                 

                You are in danger of running out of RAM. Have you set paging parameters on your addresses? (See user manual "Paging" chapter)

                 

                and immediately I got also OutOfMemory - as you can see in the dump (screenshot attached) I see not any more a big number of AddressImpl objects but this time PagingStoreImpl - I noticed this objects before as well but I thought the root problem is AddressImpl.

                 

                Is there a problem with my program? I'm doing something wrong?

                Thanks again!

                • 5. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                  Jeff Mesnil Master

                  in your code, you still create a temp queue and a consumer for each message sent. this is an anti-pattern.

                   

                  If you want to use RPC semantic with JMS, you could use a QueueRequestor instead.

                  • 6. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                    Vladimir Prodan Newbie

                    Yes, I know the code is still with the anti-pattern  - this is my old code which produces problems - the new is working fine. I'll look also at QueueRequestor, thank you for the hint.

                    I think it would be nice also to find out why I get OutOfMemory event if this is an anti-pattern - if it's a bug or if some extra cleanup should be done for each step.

                    • 7. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                      Tim Fox Master

                      Ah, you're using beta5.

                       

                      Please first check to see if the problem occurs with CR2

                      • 8. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                        Tim Fox Master

                        I just ran your program with CR2 and it doesn't run out of memory for me.

                         

                        Other comments about your test program:

                         

                        Don't use ObjectMessage (also see perf tuning chapter) just to store a string - this is very wasteful - use TextMessage instead.

                         

                        You're not using netty so don't use netty transport params, it's pointless:

                         

                                params.put("hornetq.remoting.netty.host","org.hornetq");
                                params.put("hornetq.remoting.netty.useinvm",true);

                         

                        (You can delete these ^^)

                         

                        In general you will get much, much better performance by using request response pattern in a more sensible way.

                         

                        I usually recommend not to use a temp queue at all, and have all your responses sent to the same response queue and use correlation id to de-multiplex them. In this way you can pipeline many over the same connection.

                        • 9. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                          Tim Fox Master
                          (Note to Jeff: Ok, ok I will go back to holidays now )
                          • 10. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                            Vladimir Prodan Newbie

                            I tried with CR2 and I still get out of memory - see bellow. As I said I limited the memory to 32M - maybe you have more?

                            There are:

                             

                            13551 PagingStoreImpl objects

                            13550 ClientProducerCreditManagerImpl objects

                             

                            And the rest see attached.

                             

                            I will follow the sugestions you posted and my program is already performing much better and without exceptions, thank you! Still I would like to find what's the problem with the posted program :-)

                             

                            ###################

                             

                             

                            java.lang.OutOfMemoryError: GC overhead limit exceeded
                                at java.util.ArrayList.<init>(ArrayList.java:112)
                                at sun.reflect.generics.parser.SignatureParser.parseClassTypeSignature(SignatureParser.java:241)
                                at sun.reflect.generics.parser.SignatureParser.parseFieldTypeSignature(SignatureParser.java:228)
                                at sun.reflect.generics.parser.SignatureParser.parseTypeSignature(SignatureParser.java:359)
                                at sun.reflect.generics.parser.SignatureParser.parseTypeSig(SignatureParser.java:157)
                                at sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:367)
                                at sun.reflect.annotation.AnnotationParser.parseAnnotation(AnnotationParser.java:181)
                                at sun.reflect.annotation.AnnotationParser.parseParameterAnnotations2(AnnotationParser.java:133)
                                at sun.reflect.annotation.AnnotationParser.parseParameterAnnotations(AnnotationParser.java:110)
                                at java.lang.reflect.Method.getParameterAnnotations(Method.java:751)
                                at org.hornetq.core.management.impl.MBeanInfoHelper.getOperationInfo(MBeanInfoHelper.java:112)
                                at org.hornetq.core.management.impl.MBeanInfoHelper.getMBeanOperationsInfo(MBeanInfoHelper.java:54)
                                at org.hornetq.core.management.impl.QueueControlImpl.fillMBeanOperationInfo(QueueControlImpl.java:708)
                                at org.hornetq.core.management.impl.AbstractControl.getMBeanInfo(AbstractControl.java:88)
                                at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:321)
                                at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:307)
                                at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
                                at org.hornetq.core.server.management.impl.ManagementServiceImpl.registerInJMX(ManagementServiceImpl.java:537)
                                at org.hornetq.core.server.management.impl.ManagementServiceImpl.registerQueue(ManagementServiceImpl.java:284)
                                at org.hornetq.core.server.impl.HornetQServerImpl.createQueue(HornetQServerImpl.java:1351)
                                at org.hornetq.core.server.impl.HornetQServerImpl.createQueue(HornetQServerImpl.java:770)
                                at org.hornetq.core.server.impl.ServerSessionImpl.handleCreateQueue(ServerSessionImpl.java:464)
                                at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:133)
                                at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:466)
                                at org.hornetq.core.remoting.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:411)
                                at org.hornetq.core.remoting.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:360)
                                at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:432)
                                at org.hornetq.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:131)
                                at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
                                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
                                at java.lang.Thread.run(Thread.java:619)

                            • 11. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                              Tim Fox Master

                              Vladimir-

                               

                              Could you file a JIRA report? Then someone will take a look.

                              • 13. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                                Vladimir Prodan Newbie

                                Hi,

                                 

                                I tested this again with GA version and I still get OutOfMemory - I attayh the profiler screenshot and the new java file (GA).

                                 

                                https://jira.jboss.org/jira/browse/HORNETQ-255 is closed as "done" - should be maybe reopened?

                                 

                                Regards,

                                Vladimir

                                • 14. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
                                  Jeff Mesnil Master
                                  I'll look at this. From your profiling, it seems the server resources associated to the temporary queues are not all properly removed when the queue is deleted.