HornetQ Cluster: Problem in message delivery

[vpothnis]

The setup:
1. HornetQ Cluster with 2 HornetQ servers running on the same machine. Server Discovery using UDP.

2. There is a producer that does the following in each invokation:
- Gets a connection from the connection pool (of 2 connections)
- Creates a temporary queue (say Tq)
- Posts a message to a well known queue (RequestQueue) on the HornetQ cluster. In the message, the JMSReplyTo is assigned the temporary queue.
- Creates a consumer that is associated with the temporary queue.
- Waits for the response/reply on the temporary queue and prints out the response.

3. There is a consumer that has a connection pool (of 2 connections) as well as a pool of message listeners.
- The message listeners are associated with the well known queue (RequestQueue)
- When a message is delivered to the listener, the listener extracts the JMSReplyTo and then sends a reply to that destination.

The producer and consumer are also on the same machine.
I was trying to test this setup using JMeter scripts. The following are the observations:

 Single HornetQServer HornetQCluster(2 servers)
Test 1:
(1 Thread 3000 runs) Test Complete Test Complete

Test 2:
(2 Threads 200 runs/thread) Test Complete Test Stops after some arbitrary number of runs.

When the logs are analysed, the producer is waiting for reply on one of the temporary queue. Looked like the message posted on the RequestQueue for this run was not consumed by the consumer at all.


This behaviour is consistently reproducible although it stops after different number of runs.
Is there anything that I am doing wrong? Any pointers as to why this might be happening?

Thanks,
Vinay

[vpothnis]

Update:

The observed behavior is the following:

1. The producer creates a temporary queue.
2. Creates a consumer for the temporary queue.
3. Posts the message on the "RequestQueue" (With JMSReplyTo set to the temporary queue).
4. Waits for the response on the temporary queue by calling consumer.receive().

The Consumer on the other end:
1. listens to the "RequestQueue"
2. Extracts the JMSReplyTo destination
3. Posts a reply to the destination (the temporary queue).

From what I can see in the logs, the Consumer has already posted a message to the temporaryQueue. But the Producer does not receive this and is still waiting for the message.

I looked at the temporary queue attributes in the MBeans via JConsole.
The temporary queue has a consumer registered to it. But the "message count" and "delivered count" are 0.

I am thinking the message posted to the temporary queue is somehow lost.

Any inputs/thoughts on this problem?

Thanks
Vinay

[timfox]

Please post a short test program that demonstrates the issue and someone will take a look.

[vpothnis]

Hello,

I have the test programs to reproduce the issue. But I have them as 4 java class files. It would be cumbersome to post it within the code blocks here.

Is there a way I can attach these files? Else, I can post the code as separate replies to this thread.

[timfox]

You can create a JIRA and attach it to that

[vpothnis]

https://jira.jboss.org/jira/browse/HORNETQ-236 created.

Thanks!
Vinay