HornetQ 2.2.2: Divert issues
ronnys Mar 31, 2011 9:45 AMHi,
first of all, thanks a lot for the new HornetQ release.
We had some issues with OOM conditions when using diverts in non-exclusive mode some months ago and used exclusive diverts as workaround as suggested. This works quite well, but needs more queues, complicates the setup and potentially impacts performance. So I tested non-exclusive diverts again with HornetQ 2.2.2.Final. Unfortunately, there are still some issues.
For the test, 3 queues are configured like this:
<address-setting match="jms.queue.divtest_main">
<max-delivery-attempts>-1</max-delivery-attempts>
<max-size-bytes>33554432</max-size-bytes>
<page-size-bytes>16777216</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
<address-setting match="jms.queue.divtest_sel1">
<max-delivery-attempts>-1</max-delivery-attempts>
<max-size-bytes>33554432</max-size-bytes>
<page-size-bytes>16777216</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
<address-setting match="jms.queue.divtest_sel2">
<max-delivery-attempts>-1</max-delivery-attempts>
<max-size-bytes>33554432</max-size-bytes>
<page-size-bytes>16777216</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
Diverts are configured like that:
<diverts>
<divert name="msgduplicate-divert-1">
<address>jms.queue.divtest_main</address>
<forwarding-address>jms.queue.divtest_sel1</forwarding-address>
<filter string="copy='1'"/>
<exclusive>false</exclusive>
</divert>
<divert name="msgduplicate-divert-2">
<address>jms.queue.divtest_main</address>
<forwarding-address>jms.queue.divtest_sel2</forwarding-address>
<filter string="copy='1'"/>
<exclusive>false</exclusive>
</divert>
</diverts>
HornetQ is configured to run with a maximum heap size of 1.5Gb.
Messages are being sent to jms.queue.divtest_main, 80% of all messages are matching the filter and are being copied by the diverts and then received from all 3 queues. A test client (PerfTest.java) is attached.
A few seconds after the test has been started with a clean data directory, the client connection to the server fails with "Timed out waiting for response when sending packet 43". The HornetQ server CPU usage goes up to 400% on a Quad-Core system. Log output including some thread dumps taken in 5s intervals is attached.
Btw, I also saw some OOM errors when running the same setup with address-full-policy = BLOCK on all 3 queues, but I cannot reliably reproduce it. Just FYI.
Best regards,
Ronny
-
PerfTest.java.zip 2.1 KB
-
hornetq.log.zip 143.6 KB