MDB consumer not consuming messages in the queue after startup.

[skionsnow]

I have an application that has several Hornetq queues.  About 98% of the time when the application starts, the queues produce and consume messages as expected.  About 2% of the time, the MDB consumer does not appear to consume the messages.  I can find no indication in the log as to what is the issue. I see the same indications in the log that the MDB has started up and is bound to the correct queue.  If I restart the application, the queues will operate as I expect. What debugging would you suggest I can turn on to detect this situation?  I would really like to understand what is happening that causes this behavior any suggestions would be appreciated.

[jbertram]

The first thing I would do is collect a few thread dumps when the it appears the MDB isn't consuming messages to see if something might be stuck somewhere.

[skionsnow]

Justin,

I have not reproduced the startup issue yet to compare that jstack, but I left the server up over night and in the morning it was in the same non-consuming state.  I have attached the jstack and a sample my of how I am producing messages and consuming messages on the one queue I know for sure was stuck(the first one that gets events).  I don't see any blocked threads.  I do however see far fewer Hornetq threads then when things are operating as expected. SynergyEventQueue is a singleton ejb that instantiates a SynergyQueueProducer to place the messages on the queue. SynergyMDB is the listener MDB.

Thanks,

Jim

[skionsnow]

One other thing.  I have a number of other queues.  They all follow the same basic pattern.  A singleton ejb instantiating a queueProducer and an MDB listener. 

[skionsnow]

Justin,

I just noticed that I forgot to designate SynergyEventQueue as a singleton.  That would explain things.

Sorry,

Jim

[skionsnow]

Actually, I my last comment doesn't make sense.  I see the messages being queued in my logs.  So I still don't understand why the MDB didn't see it.

[skionsnow]

Justin,

One more piece of data.  If I create a session for every message, I never see this issue.  Is there some timeout for a Session, or MessageProducer that has no messages. The queues were not getting events for long periods of time.

Jim

[jbertram]

If you're producing messages from within the application server (which it appears that you are) then you should be using a pooled-connection-factory rather than caching your connection. When you use a pooled-connection-factory you create the connection, session, producer, etc. every time you send the message because you know behind the scenes the pooled-connection-factory is managing the resources for you.  Once you update your code to use this recommended pattern let me know if you still see the problem.

[skionsnow]

Okay thanks will do that.  Couple of questions:  So I should not cache the queue-JNDI lookup as well?   I am using this in a domain cluster how do I configure the pooled-connection for that?

[skionsnow]

FYI: I figured out that no caching is required. Thanks for the pointers.

[jbertram]

I'd need more details on your server/domain setup to comment on how that might impact the pooled-connection-factory configuration.

[skionsnow]

Justin,

Not sure what you are asking.  I am running in JBOSS EAP6.3.  I have an HA-singleton service that runs on a single selected node in a configuration that for now has only two nodes.  It receives events from another application via an SDK interface.  It produces messages and places the messages on a queue.  I would like for messages that my application receives from the singleton-service and placed on a JMS queue to be available to message consumers on both nodes.  The jgroups configuration is set up so that it uses TCP transportation.

Thanks,

Jim

[jbertram]

Messages put on any queue of any node of a symmetric cluster will be available to any consumer connected to any node of that cluster.  Therefore you can just use the default pooled-connection-factory to send your messages to local queues.

[skionsnow]

Justin,

Thanks for all the help.  Things seem to be working really well now.  I have a question about the message group feature message.setStringProperty("JMSXGroupID", event.getConfId()); I noticed that if the queue empties subsequent messages with the same group id  may be processed on the other node.  This has presented no issues for my implementation, but I wanted to make sure that the grouping is expected to only persist if the queue has multiple messages associated with the group to verify that it is working as expected.

Thanks,

Jim