HornetQ java.lang.OutOfMemoryError while sending small messages at high frequency

[crytek]

Hello,

I've encountered the outOfMemoryError inside the JbossAS 7.1.2 logs while I was sending from my remote client small messages (64-100 bytes) to HornetQ queues, at a high frequency (2000 msgs/s).

My testcase description:

One cluster with two Jboss AS 7.1.2 nodes with HornetQ 2.2.16.Final.

Two clustered queues (named "testHq_bug_1017_1" , "testHq_bug_1017_2"): each queue has the max-size-bytes=1048577 (~1MB), address-full-policy=PAGE, and page-size-bytes=1048576 (1MB)

Each cluster node has the Xmx=2048 (2GB)

Message size: 64-100 bytes and of type MapMessage.

I am opening only one connection to a single node inside the cluster, and start to send the messages on 150 threads / queue  ( 2 queues => 300 threads) . Each thread starts a new session.

The queues are paused, so that messages won't be consumed by the registered MDBs.

Messages are sent to a single node and load-balanced across the cluster nodes in a Round-Robin fashion.

Actual:

While sending messages (~2000 msgs/s), after 10-20 minutes, the server becomes unresponsive and inside the server logs I get:  "java.lang.OutOfMemoryError: Java heap space" on the node  which I am sending to. Also, my client starts to throw exceptions "Exception in thread "pool-2-thread-263" java.lang.RuntimeException: javax.jms.JMSException: Timed out waiting for response when sending packet 43".

Is this a bug specific to the HornetQ 2.2.16.Final , maybe fixed in later versions, or there is something wrong inside my configuration ?

When I am sending bigger messages (1KB -  2KB), the msg sending frequency is implicitly smaller and this issue doesn't reproduce anymore.

I've attached the server logs and also the Hq config files.

Regards,

Florin

[clebert.suconic]

Are you sending it transactionally or non transactionally?

[crytek]

I am sending messages transactionally, with a batch size of 100.

[clebert.suconic]

Paging and transactions don't mean infinite pages.. each transaction still has a record on the Journal to control atomic.

This is something we will change on a next major version. Paging is not a database and it doesn't have transactional capabilities, so we use the journal for that and that still would use memory.