New dispatch model for Persistent messages.
Prior releases to 5.0 have kept references for all the messages that could be dispatched to an active Durable Topic Consumer or a Queue in memory. Whilst a reference itself was not large, it did impose a limit on the maximum number of messages that could be pending delivery.
A typical approach for messaging systems dispatching persistent messages is to pull them in batches from long term storage when a client is ready to consume them, using a cursor to maintain the next to dispatch position. This is a robust and very scalable approach, but not the most performant for cases when the consumer(s) can keep up with the producer(s) of messages.
Hence FUSE Message Broker 5.0 has a hybrid approach, allowing messages to pass from producer to consumer directly (after the messages have been persisted) but switches back to using cursors if the consumer(s) fall behind.
New Default Message Store
This is based on a combination of the journal and Kaha.
Slow Consumers
Version 5.0 will a combination of outstanding messages and memory consumption to decide when a consumer can be sent more messages (message prefetch). This will allow us to determine when consumption is slow, regardless of the message size.
Spooling to disk for non-persistent slow consumers has also been optionally introduced.
