Bug with AutoCommitAcks in HornetQ 2.1.2?

[chasan]

I've started to evaluate HornetQ 2.1.2 embedded, and found a problem with ClientSession having autoCommitAcks enabled.

I've to manually invoke ClientSession.commit() after ClientMessage.acknowledge(), even when the session was created with AutoCommitAcks=true, otherwise the consumer will stop receiving messages. I tried using ClientConsumer.receive() and ClientConsumer.setMessageHandler(), in both modes, the consumer gets stuck if ClientSession.commit() isn't called.

I triggered this problem by configuring a very small queue in hornetq-configuration.xml (max-size-bytes=1024, page-size-bytes=512, address-full-policy=PAGE).

I'm attaching sample code that triggers the problem (needs hornetq-core-2.1.2.Final.jar and netty-3.2.1.Final.jar).

I don't know if I'm doing something wrong or it's a bug in HornetQ 2.1.2.

Regards,

Carlos

[clebert.suconic]

The message will be in memory until you acknowledge the message.

On the case, you're using auto-ack, auto-commit, and you're using small sizes.

You should use a lower window size.

when you play with small sizes like that, you need to have your sizing done properly, since the system will respect the maxSize on paging until it is acknowledged.

[chasan]

I'm acknowledging every message the consumer receives, the problem is that I also have to invoke session.commit() even when auto-commit-acks is enabled in the session:

// create session with autoCommitAcks=true

ClientSession session = sessionFactory.createSession(true, true);

try {

   session.start();

   ClientConsumer consumer = session.createConsumer("jms.queue.TestQueue");

   try {

      for (int counter = 1; counter <= numMessages; counter++) {

         ClientMessage message = consumer.receive();

         message.acknowledge();

         // commit shouldn't be automatic?

         // session.commit();

      }

   } finally {

      consumer.close();

   }

   session.stop();

} finally {

   session.close();

}

According to the JavaDoc documentation, the session.commit() statement shouldn't be needed when the session is created with autoCommitAcks=true:

   /**

    * Creates a session.

    *

    * @param autoCommitSends <code>true</code> to automatically commit message sends, <code>false</code> to commit manually

    * @param autoCommitAcks <code>true</code> to automatically commit message acknowledgement, <code>false</code> to commit manually

    * @return a ClientSession

    * @throws HornetQException if an exception occurs while creating the session

    */

   ClientSession createSession(boolean autoCommitSends, boolean autoCommitAcks) throws HornetQException;

I used a very small queue to trigger the problem quickly in the sample code (I first encountered this problem with a 16MB queue and 10MB page size).

I don't yet know how to configure auto-ack using the HornetQ Core API (I didn't even knew I was using it :P).

I'll keep reading the documentation.

Thanks.

[clebert.suconic]

When you use auto-commit, the ACKs will be flushed back to the server based on sizing. You probably want ackBatchSize = 0 (or something smaller than the default), so every ACK will be flushed right away what would fix this issue.

ackBatchSize is defaulted at 1M. If your page size is < 1M, that means you will never flush memory.

If you make pageSize lower, you have to also use a lower ackBatchSize:

Example:

factory.createSession(true, true, 0); // ackBatchSize = 0

On our testsuite for example, we use lower pageSizes in some tests, but we also need to use lower ackBatchSizes.

[chasan]

The sample code now works perfectly with ackBatchSize = 0 and autoCommitAcks = true.

I will study this batch buffer for acks feature in more detail. Mmm, I'm now wondering if ackBatchSize is related to ClientConsumerImpl.ackIndividually.

Thanks!

[chasan]

In the HornetQ 2.1 user manual, section 24.9, I found this:

"If not acknowledging explictly messages are acknowledged according to the ack-batch-size setting. Be careful not to set your paging max size to a figure lower than ack-batch-size or your system may appear to hang!"

That's what happened to me. I think. The wording is a bit weird. :-S

Well, maybe it'd be a good idea that maybe HornetQ could throw an exception or log a warning message when page-size-bytes < ack-batch-size (just like when max-size-bytes <= page-size-bytes) to avoid this pitfall.

Thanks!

[clebert.suconic]

I thought about that before.. however the ACKBatchSize is a client property, and we won't know about that at the server's side..  while the pageSize is a server's property while it won't know about the client ACKBatch sizes.

[zont]

I don't really understand why the memory will be never flushed if ackbatch-size is greater than page-size.

Could you explain it on an example please. Thank you.

[ataylor]

If the page size is smaller than the ackbatch size then the server will start paging before any message are ever acknowledged, its acknowledging messages that causes depaging to occur.