Entries for temporary queues are not being removed from addressSettingsRepository?

[fbroek]

I'm running into frequent OutOfMemory problems with Jboss5 in combination with Hornetq 2.1.1.Final and 2.1.2.Final. The application using the JMS server creates (and deletes) a lot of temporary JMS queues. I know it has been stressed on these forums already that this is considered to be an anti-pattern, but it's not in my power to change the application at this time. Furthermore the fact that it all works fine on JBoss4 and Weblogic as JMS providers, does not leave much arguments for changes to the application.

Memory profiling of Jboss5/HornetQ reveals that the biggest object (think e.g. 1.5 GB with a maximum heap size of 1.8 GB) is always an instance of org.hornetq.core.settings.impl.HierarchicalObjectRepository : the addressSettingsRepository in org.hornetq.core.server.impl.HornetQServerImpl. The culprit is as far as I can determine the cache HashMap in the addressSettingsRepository: it contains a huge amount of references to temporary queues, but the queues are no longer there (at least not visible in the JMX console).

I have done a bit of testing by adding some log messages to the methods in HierarchicalObjectRepository which contain a cache.clear() statement (addMatch(), removeMatch(), setDefault() and clear()) and creating (and deleting) a lot of temporary queues from some test code  None of the methods seem to be called during normal running of the server, which would explain the growth of that HashMap.

Am I missing something obvious here, or is there a call missing to e.g. removeMatch() in the server code somewhere when temporary queues are deleted on the client side?

Thanks,
Frederik

P.S. Almost identical behaviour was seen for the cache HashMap in org.hornetq.core.security.impl.SecurityStoreImpl but when I added <security-enabled>false</security-enabled> that one disappeared as memory-hog, only to reveal the problem described above.

I'm running into frequent OutOfMemory problems with Jboss5 in combination with Hornetq 2.1.1.Final and 2.1.2.Final. The application using the JMS server creates (and deletes) a lot of  temporary JMS queues. I know it has been stressed on these forums  already that this is considered to be an anti-pattern, but it's not in  my power to change the application at this time. Furthermore the fact  that it all works fine on JBoss4 and Weblogic as JMS providers, does not  leave much arguments for changes to the application.

Memory profiling of Jboss5/HornetQ reveals that the biggest object  (think e.g. 1.5 GB with a maximum heap size of 1.8 GB) is always an  instance of org.hornetq.core.settings.impl.HierarchicalObjectRepository :  the addressSettingsRepository in  org.hornetq.core.server.impl.HornetQServerImpl. The culprit is as far as  I can determine the cache HashMap in the addressSettingsRepository: it  contains a huge amount of references to temporary queues, but the queues  are no longer there (at least not visible in the JMX console).

I have done a bit of testing by adding some log messages to the methods  in HierarchicalObjectRepository which contain a cache.clear() statement  (addMatch(), removeMatch(), setDefault() and clear()) and creating (and  deleting) a lot of temporary queues from some test code  None of the  methods seem to be called during normal running of the server, which  would explain the growth of that HashMap.

Am I missing something obvious here, or is there a call missing to e.g.  removeMatch() in the server code somewhere when temporary queues are  deleted on the client side?

Thanks,
Frederik

P.S. Almost identical behaviour was seen for the cache HashMap in  org.hornetq.core.security.impl.SecurityStoreImpl but when I added  <security-enabled>false</security-enabled> that one  disappeared as memory-hog, only to reveal the problem described above.

[fbroek]

After having had a good look at the code, I have decided to create a JIRA issue: https://issues.jboss.org/browse/HORNETQ-641

Thanks,

Frederik

[clebert.suconic]

Did you look at trunk?

[fbroek]

As far as I can see this bit of code/functionality is (still) the same in trunk as in 2.1.1.Final and 2.1.2.Final.

[clebert.suconic]

It's not about the matching. There's a cache inside the Repository that's only cleared when you create new matches, what will never happen with temporary queues.

I have locally added a test and a fix. It will be committed by the end of my day.

[clebert.suconic]

thank you very much for reporting it.

[clebert.suconic]

As you probably saw on the JIRA I closed as done.

It's not on trunk yet, because most of the current work is being done on a different Branch. (which is the equivalent for trunk now).

We will merge all the changes from that branch on trunk. Feel free to try it.

Thanks again

[fbroek]

Thanks for the quick response to the JIRA issue! As far as I can see, your fix should do the trick of clearing the cache (by calling the new addressSettingsRepository.clearCache()).

As I mentioned in my original post, we saw similar behaviour for the cache hashmap in org.hornetq.core.security.impl.SecurityStoreImpl even though org.hornetq.core.security.impl.SecurityStoreImpl.checkCached(..) should clear that one regularly. At the moment I can't check that in detail as we added <security-enabled>false</security-enabled> to the configuration as a workaround, but it might be worthwhile checking if the cache gets cleared there with the creation/deletion of temporary queues.

Thanks,

Frederik