-
1. Re: Why hornetq paging very slow?
ataylor Mar 5, 2012 4:43 AM (in response to bjrenyong74)it takes longer because the messages are persisted to disk
-
2. Re: Why hornetq paging very slow?
bjrenyong74 Mar 5, 2012 4:49 AM (in response to ataylor)But I should say it's too slow. I think it's not normal. I use persistent queue, why write journal so fast, but so slow to write paging?
-
3. Re: Why hornetq paging very slow?
ataylor Mar 5, 2012 5:06 AM (in response to bjrenyong74)there are extra overheads with paging
-
4. Re: Why hornetq paging very slow?
clebert.suconic Mar 5, 2012 10:32 AM (in response to bjrenyong74)There were some improvements on the latest versions. Why don't you try the version that comes with AS7? you could just download the latest AS7 and try it out as it was a standalone or you could download the branch and build it yourself.
-
5. Re: Why hornetq paging very slow?
bjrenyong74 Mar 6, 2012 5:41 AM (in response to clebert.suconic)Hi, I used iostat to check:
Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 0.00 0.00 0.00 61.00 0.00 30.50 1.00 0.22 3.54 2.92 17.80
cciss/c0d0p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d0p2 0.00 0.00 0.00 61.00 0.00 30.50 1.00 0.22 3.54 2.92 17.80
cciss/c0d0p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d0p4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d0p5 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d0p6 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d0p7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d0p8 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
cciss/c0d1 0.00 123.00 0.00 152.00 0.00 1096.00 14.42 0.99 6.51 6.52 99.10
cciss/c0d1p1 0.00 123.00 0.00 152.00 0.00 1096.00 14.42 0.99 6.51 6.52 99.10paging datas on c0d1.
Is that means my disk IO performance bottleneck? And I have no choice to tuning?
-
6. Re: Why hornetq paging very slow?
clebert.suconic Mar 6, 2012 12:51 PM (in response to bjrenyong74)There were some improvements as when we sync and making it more asynchronous. Did you get a chance to test the latest version?
How are you commiting your data? How are you producing? How are you consuming? (General observations that will help on any message system)
-
7. Re: Why hornetq paging very slow?
luismcosta Mar 22, 2012 12:54 PM (in response to clebert.suconic)Hi,
I have a question that relates with this one. Besides the documentation (http://docs.jboss.org/hornetq/2.2.5.Final/user-manual/en/html_single/index.html#paging) I didn’t find any other source of information where I could find an answer.
When in Paging Mode, which means that “max-size-bytes” has been reached, what happens to the new messages that are delivered in that destination? Are they paged one by one? I would expect that newest “page-size-bytes” messages would be paged in blocks, and the old ones (who are the first to be delivered) to be kept in memory. This implies that “page-size-bytes” should be less or equal to “max-size-bytes”.
Thanks for the help,
Luís M. Costa
-
8. Re: Why hornetq paging very slow?
clebert.suconic Mar 22, 2012 4:20 PM (in response to luismcosta)That is correct...
if you actually size page-size-bytes > max-size that would be a configuration error (you would immediately starve the memory if the page-size > max-page-size whenever you added stuff to the memory).
Paging is a overflow of the memory, so the first messages are kept in memory.
-
9. Re: Why hornetq paging very slow?
luismcosta Mar 23, 2012 10:49 AM (in response to bjrenyong74)Hi Clebert!
Thank you for your response, but... what is correct?
My assumption that 1) the newest "page-size-bytes" messages are stored in blocks or, instead, 2) they are paged one by one on receive?
Thanks,
Luís M. Costa
-
10. Re: Why hornetq paging very slow?
stefan_fink Jun 15, 2012 4:15 AM (in response to clebert.suconic)I have exactly the same timeout of about 3 seconds when messages are paged out.
I tested with 2.2.14 Final, using the core API to generate the messages and also the core API to consume the messages. Consuming isn't a issue.
With my testing I found that the buffers which is paged out ist quite small (385 messages which look like "Messsage: 1234").
Every time this Buffer is wirtten to disk the delay occurs. I checked it with wireshark and the delay is also there visible.
Regards
Stefan
-
11. Re: Why hornetq paging very slow?
clebert.suconic Jun 15, 2012 12:06 PM (in response to stefan_fink)There's a timed buffer writing to the pages. If you have a single producer, you need to wait the conclusion of that write.. it should scale when you have more than one producer... and you can configure the timed buffer.
-
12. Re: Why hornetq paging very slow?
stefan_fink Jun 18, 2012 9:19 AM (in response to clebert.suconic)Ok I have to wait until the write has finished. But why does this take 3 seconds. It can't be a write (IO) bound problem. Something else ist waiting here.
The only place I find timed buffer in the docs ist the
journal-buffer-timeout
andjournal-buffer-size
and here a max of 490k are written many hundred times a second!Any idea where the paging buffer is configured?