1 2 Previous Next 22 Replies Latest reply on May 6, 2013 5:20 PM by clebert.suconic Branched to a new discussion.

consumer-window-size - should it be set at the server side or the client side?

newway Apr 8, 2013 7:41 AM

Hello,

I have a standalone java client that connects to queues running on JBoss 7.1.1 Final.

I'm trying to set the consumer window size to no buffering and I thought this is set by the consumer, so i use the following code:

transportConfiguration = new TransportConfiguration(NettyConnectorFactory.class.getName(), p);
connectionFactory = HornetQJMSClient.createConnectionFactoryWithoutHA(JMSFactoryType.QUEUE_CF, transportConfiguration);
connectionFactory.setConsumerWindowSize(0);

and only later on I create a session and a consumer.

my assumption was that this will take care of the buffering, but it doesn't - I have 15 consumers and only 5 of them are receiving messages - while the other 10 are informing that no messages are available.

and when I look in the jboss admin console I see that there are 10 messages in the queue and that they are all in delivery - and this is the same behavior I had before I started setting the window size.

So I think I misunderstood something - because in http://docs.jboss.org/hornetq/2.2.14.Final/user-manual/en/html/flow-control.html I read that it has to be set at the connection factory creation and this is what I do here.

please help me understand what I misunderstood.

Thanks,

Noa

1. Re: consumer-window-size - should it be set at the server side or the client side?

ataylor Apr 8, 2013 8:27 AM (in response to newway)

yes, that is correct, you set it on the client side when using the core API.if you think it is not working can you provide a test?
Actions
2. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 8, 2013 11:56 AM (in response to ataylor)
attached is a zip with an example.

I didn't manage to reproduce exactly the same issue - but I did have several runs where while I only have 10 consumers I see that there are 12 messages in delivery.
same was when i run with 5 consumers - i saw 9 messages in delivery.

a screen shot is inside the zip as well.

Example.zip 3.0 MB
Actions
3. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 14, 2013 11:50 AM (in response to newway)
Hi,

I didn't get any response to the example I loaded here - and I prepared a more refined test that shows how several consumers are not getting info from the queue while the queue is not empty and than after about a minute another consumer gets data.

in the zip file there are:
1. batch file to run the test.
2. example to the results of running the test.

please let me know if I am looking in the right direction or I misdiagnosed.

thanks,
Noa

Example - take2.zip 3.0 MB
Actions
4. Re: consumer-window-size - should it be set at the server side or the client side?

clebert.suconic Apr 15, 2013 9:20 AM (in response to newway)

Can't you make a very simple test without an EAR. You are not using any MDBs at all...?

anyway.. I briefly looked at your test.. and you are just starting several consumers. We do that kind of test all the time. I'm not sure what you are doing there.
Actions
5. Re: consumer-window-size - should it be set at the server side or the client side?

clebert.suconic Apr 15, 2013 9:20 AM (in response to clebert.suconic)

Use one of our examples as the basis for your test... or maybe even an UnitTest would be great.
Actions
6. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 17, 2013 9:44 AM (in response to clebert.suconic)

I'm using an EAR to create the messages because I wanted to simulate a situation as close as possible to my real app.

my scenario is that I have some code that runs inside JBoss and generates messages that are consumed by standalone java clients.
as a result of the content of the message the client will do some work.

I'm trying to overcome a situation where I see that in the queue there are 20 messages and now 2 out of 10 clients are taking 6 hours to handle the current message that each of them is are working on and the other 8 clients are not receiving messages.
and when I look in the jboss admin console I see that all the 20 messages are in delivery.

so for this specific test I only receive a message from the queue and sleep for a random amount of seconds to simulate the delay, I also created the messages in an EAR to avoid the most remote possibility that the connection factory of the sender will effect the one of the receivers.

Can you please help me understand what is the point you are criticizing here? the fact that my test is complicated to run because I create the messages using an EAR?

thanks,
Noa
Actions
7. Re: consumer-window-size - should it be set at the server side or the client side?

ataylor Apr 17, 2013 9:53 AM (in response to newway)

Can you please help me understand what is the point you are criticizing here? the fact that my test is complicated to run because I create the messages using an EAR?
I dont think any one is criticizing you here, all clebert is asking is if you could provide something that we could *very quickly* run in our framework, either an example or a unit test.
Actions
8. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 17, 2013 9:58 AM (in response to ataylor)

Thanks,
I didn't mean to sound offended or to complain on the attitude - when I wrote the post I realized that I'm using a strong word - but I wasn't able to find another one and I guess I should have added some disclaimer there.

I will try to produce a test that suites your framework - this basically means something that doesn't require starting up JBoss in order to run the test? something that runs on a standalone hornetq server?

Thanks,
Noa
Actions
9. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 17, 2013 2:19 PM (in response to newway)
OK - I hope this time the example will do.

I based it on the example at \hornetq-2.3.0.CR2\examples\jms\no-consumer-buffering

in the pom.xml you can define the number of messages, number of consumers, min time to handle a message and max time to handle a message.

in the attached zip you can also see the results of a run that I did and the test failed.

Basically my assumption is that if there are N concurrent consumers - after each consumer handles it's first message - than all the other messages should be handled in an ordered manner.

so all the consumer push into a java.util.concurrent.ConcurrentLinkedQueue the number of the current message they are working on and when all the messages are processed - I read from the queue, if one item in the queue is smaller than the one that preceded it than it is an indication that the test failed.

Let me know if I got some wires crossed and there is something fundamentally wrong in my understanding (actually let me know even if it isn't so fundamental)

Thanks,
Noa

no-cuncurrent-consumers-buffering.zip 92.8 KB
Actions
10. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 21, 2013 6:25 AM (in response to ataylor)

Hi,

Can you let me know if the recent example I gave explains my issue enough.
An answer like "yes we saw the example but we have our priorities and you are somewhere in our queue" is good enough.
I just wanted to know if there is more input needed from my side.

Thanks,
Noa
Actions
11. Re: consumer-window-size - should it be set at the server side or the client side?

ataylor Apr 21, 2013 6:46 AM (in response to newway)

we are all in the process of preparing a release, once done i will take a look at your issue, probably end of the week.
Actions
12. Re: consumer-window-size - should it be set at the server side or the client side?

newway Apr 21, 2013 8:42 AM (in response to ataylor)

Thank you.

Good luck in the relase.
Actions
13. Re: consumer-window-size - should it be set at the server side or the client side?

newway May 2, 2013 7:45 AM (in response to ataylor)

Hi,

I just wanted to let you know that I downloaded the 2.3.0.Final and verified that the same behavior is presented there.

Noa
Actions
14. Re: consumer-window-size - should it be set at the server side or the client side?

clebert.suconic May 2, 2013 9:51 AM (in response to newway)

It seems to me just a counter's issue. That counter is not meant to be atomic at that level. The best way to validate if there's an issue with consumer-window-size is to stop one of the consumers and see if any message will arrive on that consumer or not.

That means...a message wouldn't be delivered to other consumers.
Actions

1 2 Previous Next

Go to original post