-
1. Re: FailoverTests consuming a lot of memory...
clebert.suconic Dec 18, 2008 3:29 PM (in response to clebert.suconic)It's actually quite funny...
MultiThreadFailvoerTest was taking about 7 minutes to run before disabling JMX on the test, and 3 minutes after the change.
(I ran all these without my changes for largeMessage & clustering)
I have seen this MessageReferences following this path:
MessagingReferenceImpl <- ConcurrentHashMaps <- QueueImpl <- MessageCounter <- ManagementImpl <- Somewhere_I_couldn't find (Probably JMX on the JVM)
(This is a summarized graph of course.... the real profiler shows many more objects in between)
I was only seeing MessageReferenceImpl growing on those tests, like the old JMXManagement Object was not being released. Maybe it would worth taking a look if management is not leaving objects after a server is stopped. -
2. Re: FailoverTests consuming a lot of memory...
timfox Dec 18, 2008 3:37 PM (in response to clebert.suconic)"clebert.suconic@jboss.com" wrote:
I'm changing RandomFailoverTest and MultiThreadRandomFailoverTest to disable JMX on Management.
.
How about fixing management, rather than disabling it in the tests? :) -
3. Re: FailoverTests consuming a lot of memory...
clebert.suconic Dec 18, 2008 3:50 PM (in response to clebert.suconic)I'm not sure if it's broken yet and JMX is not part of those tests.
And I posted a little bit more of information, so we could keep looking to why it happened. -
4. Re: FailoverTests consuming a lot of memory...
clebert.suconic Dec 18, 2008 4:01 PM (in response to clebert.suconic)"timfox" wrote:
"clebert.suconic@jboss.com" wrote:
I'm changing RandomFailoverTest and MultiThreadRandomFailoverTest to disable JMX on Management.
.
How about fixing management, rather than disabling it in the tests? :)
BTW: I"m not disabling Management. I"m just disabling JMX. We should verify if there there is a leak on registering Objects on JMX. (queue... etc) -
5. Re: FailoverTests consuming a lot of memory...
jmesnil Dec 19, 2008 2:21 AM (in response to clebert.suconic)i'll investigate the JMX leak
-
6. Re: FailoverTests consuming a lot of memory...
timfox Dec 19, 2008 4:02 AM (in response to clebert.suconic)"clebert.suconic@jboss.com" wrote:
I'm not sure if it's broken yet and JMX is not part of those tests.
It's certainly broken if it's leaking after the servers have been shut down.
Also, JMX is a useful part of those tests - it helped Jeff and I catch several issues - so don't disable it, fix the problem. -
7. Re: FailoverTests consuming a lot of memory...
jmesnil Dec 19, 2008 4:19 AM (in response to clebert.suconic)it's definitely a leak in the management code.
when the server is stopped, all the resources registered in JMX were not unregistered.
Clebert noticed that in the failover tests because they start/stop the server many times and registered many resources (e.g. one resource for each consumer queue) while never unregistering any.
I'll fix the code so that the resources are properly unregistered when the server is stopped.
I'll also reenable JMX in the failover tests