I wanted to ask about everyones experience with using <diverts>.
Here is our set up, and what I've noticed.
We have a system of record, and many systems that are working copies (stores) of our system of record. We are using HornetQ as our JMS transport layer to manage replication between these systems.
Originally, we were using a single topic to send messages to 1/several/or all of these store copies. The store copies maintain a durable subscription to the topic so that no messages are lost. We have run into the issue where 1 or more of the store systems are down, and hence messages get stored in HornetQ and eventually roll to paging files. Once paging happens, we are no longer able to get messages to the rest of our stores copies. I understand that this is a known behavior of HornetQ.
Our solution was to use Diverts and create a topic for each of the working copies, allowing us to manage the memory and paging parameters in a way that a single store being down for a period of time does not prevent other stores from receiving their messages.
What I've noticed, is that when a message from the system of record is intended to go to all stores, the HornetQ throughput severely bogs down (from our typical rate of 250 records/second to less than 20 records/second). My suspicion is that this is due to needing to write a copy of each message for each topic to the journal, so 10 messages sent to 500 stores means 5000 messages written to the journal.
Is there a better strategy for managing the memory and maintaining throughput using HornetQ?
We do require that no messages be lost (hence the durable subscriptions), and have set the <block-on-non-durable-send> to true because we noticed instances where a message was assumed to have been sent successfully, but actually lost due to the timing in receiving the error via the listener.
thanks,
John
