-
1. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
timfox Dec 16, 2009 1:13 PM (in response to vlpp)Can you post a test program that demonstrates the issue?
BTW Creating a temp queue for each request-response is a very common anti-pattern. Don't do it, your system will perform very poorly!
This is mentioned in the user manual section on performance tuning.
-
2. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
vlpp Dec 17, 2009 3:14 AM (in response to timfox)Hi,
I will try to provide a testcase. Regarding the temporary queue I followed the request-response example - I'll try to reuse the temporary queue.
Thank you for the answer
-
3. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
timfox Dec 17, 2009 6:23 AM (in response to vlpp)In the example, of course the session, producer, consumer and temp queue need to be re-used, sorry if we didn't make that clear in the example.
Also you can get rid of the temp queue altogether and use correlation id and a map.
There are many examples of this on the web if you google it.
-
4. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
vlpp Dec 17, 2009 7:18 AM (in response to timfox)Of course you are right, makes sense to not create every time a new queue and I'm reusing now the queues but in my prototype I came first to the memory problem :-)
I attached a small program that reproduces this problem: I executed this with -Xmx32m and after a short time I start to receive messages like:
17.12.2009 13:07:18 org.hornetq.core.logging.Logger warn
WARNUNG: Less than 25%
free memory: 1,22 MiB
max memory: 30,88 MiB
total memory: 30,75 MiB
available memory: 4,37%You are in danger of running out of RAM. Have you set paging parameters on your addresses? (See user manual "Paging" chapter)
and immediately I got also OutOfMemory - as you can see in the dump (screenshot attached) I see not any more a big number of AddressImpl objects but this time PagingStoreImpl - I noticed this objects before as well but I thought the root problem is AddressImpl.
Is there a problem with my program? I'm doing something wrong?
Thanks again!
-
AddressImpl.java.zip 1.4 KB
-
hornetqprofiling.gif 39.3 KB
-
-
5. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
jmesnil Dec 17, 2009 8:01 AM (in response to vlpp)in your code, you still create a temp queue and a consumer for each message sent. this is an anti-pattern.
If you want to use RPC semantic with JMS, you could use a QueueRequestor instead.
-
6. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
vlpp Dec 17, 2009 8:07 AM (in response to jmesnil)Yes, I know the code is still with the anti-pattern - this is my old code which produces problems - the new is working fine. I'll look also at QueueRequestor, thank you for the hint.
I think it would be nice also to find out why I get OutOfMemory event if this is an anti-pattern - if it's a bug or if some extra cleanup should be done for each step.
-
7. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
timfox Dec 17, 2009 8:08 AM (in response to vlpp)Ah, you're using beta5.
Please first check to see if the problem occurs with CR2
-
8. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
timfox Dec 17, 2009 8:18 AM (in response to timfox)I just ran your program with CR2 and it doesn't run out of memory for me.
Other comments about your test program:
Don't use ObjectMessage (also see perf tuning chapter) just to store a string - this is very wasteful - use TextMessage instead.
You're not using netty so don't use netty transport params, it's pointless:
params.put("hornetq.remoting.netty.host","org.hornetq");
params.put("hornetq.remoting.netty.useinvm",true);(You can delete these ^^)
In general you will get much, much better performance by using request response pattern in a more sensible way.
I usually recommend not to use a temp queue at all, and have all your responses sent to the same response queue and use correlation id to de-multiplex them. In this way you can pipeline many over the same connection.
-
9. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
timfox Dec 17, 2009 8:19 AM (in response to timfox)(Note to Jeff: Ok, ok I will go back to holidays now ) -
10. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
vlpp Dec 17, 2009 9:30 AM (in response to timfox)I tried with CR2 and I still get out of memory - see bellow. As I said I limited the memory to 32M - maybe you have more?
There are:
13551 PagingStoreImpl objects
13550 ClientProducerCreditManagerImpl objects
And the rest see attached.
I will follow the sugestions you posted and my program is already performing much better and without exceptions, thank you! Still I would like to find what's the problem with the posted program :-)
###################
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.ArrayList.<init>(ArrayList.java:112)
at sun.reflect.generics.parser.SignatureParser.parseClassTypeSignature(SignatureParser.java:241)
at sun.reflect.generics.parser.SignatureParser.parseFieldTypeSignature(SignatureParser.java:228)
at sun.reflect.generics.parser.SignatureParser.parseTypeSignature(SignatureParser.java:359)
at sun.reflect.generics.parser.SignatureParser.parseTypeSig(SignatureParser.java:157)
at sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:367)
at sun.reflect.annotation.AnnotationParser.parseAnnotation(AnnotationParser.java:181)
at sun.reflect.annotation.AnnotationParser.parseParameterAnnotations2(AnnotationParser.java:133)
at sun.reflect.annotation.AnnotationParser.parseParameterAnnotations(AnnotationParser.java:110)
at java.lang.reflect.Method.getParameterAnnotations(Method.java:751)
at org.hornetq.core.management.impl.MBeanInfoHelper.getOperationInfo(MBeanInfoHelper.java:112)
at org.hornetq.core.management.impl.MBeanInfoHelper.getMBeanOperationsInfo(MBeanInfoHelper.java:54)
at org.hornetq.core.management.impl.QueueControlImpl.fillMBeanOperationInfo(QueueControlImpl.java:708)
at org.hornetq.core.management.impl.AbstractControl.getMBeanInfo(AbstractControl.java:88)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getNewMBeanClassName(DefaultMBeanServerInterceptor.java:321)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:307)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.hornetq.core.server.management.impl.ManagementServiceImpl.registerInJMX(ManagementServiceImpl.java:537)
at org.hornetq.core.server.management.impl.ManagementServiceImpl.registerQueue(ManagementServiceImpl.java:284)
at org.hornetq.core.server.impl.HornetQServerImpl.createQueue(HornetQServerImpl.java:1351)
at org.hornetq.core.server.impl.HornetQServerImpl.createQueue(HornetQServerImpl.java:770)
at org.hornetq.core.server.impl.ServerSessionImpl.handleCreateQueue(ServerSessionImpl.java:464)
at org.hornetq.core.server.impl.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:133)
at org.hornetq.core.remoting.impl.ChannelImpl.handlePacket(ChannelImpl.java:466)
at org.hornetq.core.remoting.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:411)
at org.hornetq.core.remoting.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:360)
at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:432)
at org.hornetq.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:131)
at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)-
hornetqdump2.jpg 429.4 KB
-
-
11. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
timfox Dec 17, 2009 4:14 PM (in response to vlpp)Vladimir-
Could you file a JIRA report? Then someone will take a look.
-
13. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
vlpp Jan 19, 2010 4:13 AM (in response to vlpp)Hi,
I tested this again with GA version and I still get OutOfMemory - I attayh the profiler screenshot and the new java file (GA).
https://jira.jboss.org/jira/browse/HORNETQ-255 is closed as "done" - should be maybe reopened?
Regards,
Vladimir
-
AddressImpl.java.zip 1.3 KB
-
HORNETQ-255-GA.png 76.7 KB
-
-
14. Re: many org.hornetq.core.postoffice.impl.AddressImpl objects
jmesnil Jan 19, 2010 4:43 AM (in response to vlpp)I'll look at this. From your profiling, it seems the server resources associated to the temporary queues are not all properly removed when the queue is deleted.