Cascading SlowConsumer Advisories

[jbullock_jbullock]

I recently experienced an activemq failure where the service became unresponsive.  A thread dump showed a large number of threads whose names indicated they were related to ActiveMQ.Advisory.SlowConsumer.Topic.xxx  But the weird thing is that the topic that was being alerted on was another slow consumer advisory topic!  This was repeated over and over with deeper and deeper nesting.  An example:

"Topic ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Advisory.SlowConsumer.Topic.ActiveMQ.Adviso .....

This is an indication to me that ActiveMQ is recursively creating SlowConsumer advisory topics on it own advisory topics.

My only solution was to restart the process.

Does anyone have any idea what was going on?  This has happened occasionally before and I would really like to know what is happening.

[garytully]

please open an issue to track this. The broker should not produce advisories for advisory destinations. The slow consumer advisory is relatively new so it may be slipping through. In any event this needs investigation.

Do you have a consumer for that advisory?

Can you attach your broker config and the thread dump to the jira?

[jbullock_jbullock]

Thanks.  I've created jira DEV-4216 describing this condition as a bug and attached the full thread dump and the activemq.xml.

I don't have any consumers on the SlowConsumer advisories right now.  So for the time being, I'll set will try setting the advisory topic policy as advisoryForSlowConsumers="false".  This might avoid the bug for the time being.

[jbullock_jbullock]

Converted to jira bug on advice that "activemq shouldn't be creating advisories on advisories".