OOM errors when using chained diverts
ronnys Jul 20, 2010 3:46 AMHi,
we need to distribute messages to 2 different destinations. Originally we wanted to use a simple topic and 2 durable subscribers w/o filters, but due to the nature of the paging mechanism, this does not work very well if the backlog of both subscribers gets too different (i.e. the gap gets too large), as the subscriber with the bigger backlog stops the subscriber with the smaller backlog.
So I tried diverts and (ab-?)used it in the following way:
- Client posts to queue A
- HornetQ exclusively diverts any messages coming in on queue A to queue B
- HornetQ non-exclusively diverts any messages coming in on queue B to queue C
When posting messages to queue A and reading them concurrently from queue B, the HornetQ server fails with OOM errors (tested with the latest trunk version as of Friday, July, 16th). Server config files are attached, the memory and Java settings are the default except enabled remote JMX access. The queues used are named jms.queue.msgduplicate*.
In order to reproduce the issue:
- delete HornetQ data directory and start HornetQ; watch memory consumption
- start Publisher
- LoadTestPublisher org.jnp.interfaces.NamingContextFactory jnp://host:port ConnectionFactory queue/msgduplicate_t1_inbound 241200000 10 2048 5000 0
- Wait until publisher posted 200000 messages, then start consumer (do not consume messages from the other queue!)
- LoadTestConsumer org.jnp.interfaces.NamingContextFactory jnp://host:port ConnectionFactory queue/msgduplicate_t1_outbound_1 241200000 5000
- The server should consume a lot of memory and dump OOM errors within the next 2 minutes.
Still looking for a better way to distribute messages to multiple different queues ...
Best regards,
Ronny
-
LoadTestPublisher.java.zip 2.6 KB
-
LoadTestConsumer.java.zip 2.2 KB
-
hornetq-users.xml 341 bytes
-
hornetq-jms.xml 2.9 KB
-
hornetq-configuration.xml 7.0 KB
-
hornetq-beans.xml 2.0 KB