14 Replies Latest reply on Jul 24, 2008 11:19 AM by ataylor

Producer hangs after 20M sent

ataylor Jul 19, 2008 6:29 AM

Ive come across what i think may be a bug in JBM 2.

If you run the perf sender and set the message size to 64k, the producer stops sending after 32000 messages, if the message size is 32k it stops sending after 64000 messages, and so forth.

So basically it looks like it stops working after 20M of messages. Can any of you guys reproduce this problem, if you can i'll raise a JIRA.

NB pinging needs to be disabled to send lge messages.

1. Re: Producer hangs after 20M sent

timfox Jul 19, 2008 6:39 AM (in response to ataylor)

Sounds like something to do with producer flow control.

What's your producer window size set to? What happens if you disable producer flow control or change the window size?
Actions
2. Re: Producer hangs after 20M sent

clebert.suconic Jul 19, 2008 9:01 PM (in response to ataylor)

That's a big coincidence for me...

I also ran the same test on friday...

I got the stack traces and it had to do with Producer Flow Control.
Actions
3. Re: Producer hangs after 20M sent

ataylor Jul 21, 2008 3:26 AM (in response to ataylor)

Yeah, it's definitely producer flow control. If you set it to -1 however, you quite quickly get an out of memory exception. I'll raise a JIRA
Actions
4. Re: Producer hangs after 20M sent

timfox Jul 21, 2008 4:50 AM (in response to ataylor)

"ataylor" wrote:
Yeah, it's definitely producer flow control. If you set it to -1 however, you quite quickly get an out of memory exception. I'll raise a JIRA

That's to be expected, since you're not limiting flow to the server any more (with -1).
Actions
5. Re: Producer hangs after 20M sent

ataylor Jul 21, 2008 5:16 AM (in response to ataylor)

altho, the exception is happening on the client which i wouldn't have expected!
Actions
6. Re: Producer hangs after 20M sent

timfox Jul 21, 2008 5:24 AM (in response to ataylor)

"ataylor" wrote:
altho, the exception is happening on the client which i wouldn't have expected!

It happens because you're putting packets on the MINA write queue faster than MINA can take if off the other end and write it to the socket.
Actions
7. Re: Producer hangs after 20M sent

ataylor Jul 21, 2008 5:52 AM (in response to ataylor)

ahhh, ok, i get it now.
Actions
8. Re: Producer hangs after 20M sent

timfox Jul 24, 2008 10:53 AM (in response to ataylor)

Is there a JIRA for this?
Actions
9. Re: Producer hangs after 20M sent

ataylor Jul 24, 2008 10:58 AM (in response to ataylor)

nope, do we need one as its bad configuration or do we need to force flow control to always be enabled?
Actions
10. Re: Producer hangs after 20M sent

timfox Jul 24, 2008 10:59 AM (in response to ataylor)

We don't know what the problem is yet do we?
Actions
11. Re: Producer hangs after 20M sent

ataylor Jul 24, 2008 11:07 AM (in response to ataylor)

It happens because you're putting packets on the MINA write queue faster than MINA can take if off the other end and write it to the socket.

I thought this was the problem!
Actions
12. Re: Producer hangs after 20M sent

timfox Jul 24, 2008 11:08 AM (in response to ataylor)

"ataylor" wrote:

I thought this was the problem!

No, that was an explanation of why you run out of memory when you disabled flow control, not an explanation of the reason for the original problem.
Actions
13. Re: Producer hangs after 20M sent

ataylor Jul 24, 2008 11:14 AM (in response to ataylor)

ah yes, memory, old age etc :)

I'll raise a JIRA
Actions
14. Re: Producer hangs after 20M sent

ataylor Jul 24, 2008 11:19 AM (in response to ataylor)

https://jira.jboss.org/jira/browse/JBMESSAGING-1405
Actions

Go to original post