Understanding the nuances of "Pre-Acknowledge" mode

[bnc119]

Hi,

I am running HornetQ 2.1.2 Final.  The details of my system configuration are as follows: 

• I have configured HornetQ for "Pre-ACK" mode.
• I have disabled the Expiry Queue.
• I have a non-durable subscriber that subscribes to a topic but does not ever call receive().  Instead, the subscriber is busy doing other work.
• All messages published have a TimeToLive of 10 seconds. 
• I am using jConsole to monitor each topics' "MessageCount" value  - that is, the number of messages yet to be ACK'd by a client.

I am running a number of simple experiments and have a few questions:

When I publish a small number of messages, say, 10, to a particular topic, I see that the MessageCount field stays at 0, as I expect it to.  All messages are immediately "Pre-Ack'd" by HornetQ.

However, if I "burst" many (say, 1000) messages to a topic, I see that the MessageCount jumps approx 70.  For some reason,  930 messages are "Pre-Ack'd" immediately, but 70 are not.  After approximately 10 seconds,  (the message TTL) the messages are ACK'd, and MessageCount drops to 0.  If I immediately send another 1000 messages, the behavior is a little different: All 1000 messages are thrown onto MessageCount.  (That is, none of them are Pre-Ack'd until about 10 seconds later)

Can anyone explain what is happening here?  Again, my consumer is not ever calling recieve().  Am I hitting a limit on the local client buffer?

My second (related) question is:

Is there a limit (ceiling) to the value represented in MessageCount?  If my producer sends 50,000 messages with a TTL of 60 seconds, I typically see the MessageCount max out at approx. 38,000.  It takes my producer far less than 60 seconds to publish all 50,000 messages to the server.  Where are my remaining (approx 12,000) messages?  Did they get ACK'd immediately, and therefore don't show up in MessageCount?

Thanks,

[clebert.suconic]

Pre-ACK will be called when the server is delivering the message to the client.

The message may go to the client buffer... at that point if you don't consume your client's buffer, the buffer may be up to its maximum size (consumer-window-size) and messages will be then batched at the server.

You should consume messages if you want to have more messages delivered. Or you could also set consumerWindowSize=-1.. but you could eventually OutMememoryException at the client if you don't consume your buffer.

[bnc119]

Actually, I have a quick follow-up question:

The details of my system configuration remain the same:

• I have configured HornetQ for "Pre-ACK" mode.
• I have disabled the Expiry Queue.
• I have a non-durable subscriber that subscribes to a topic but does not ever call receive().  Instead, the subscriber is busy doing other work.

Under these constraints, I'm playing around with different <consumer_window_size> values.  The resulting behavior is strange:  the true client buffer size appears to be less than half of whatever I specify in the consumer_window_size field.

For example, if I specify  <consumer-window-size>5242880</consumer-window-size>  (5MB), I see the server-side queues start filling up after a publisher pushes approx 2.1 MB of message data.  These results are consistent with just about any non-negative value I try for  <consumer-window-size>.  Any idea what's going on here?

(P.S:  Setting consumer_window_size = -1 appears to work correctly.  In this case, I never see any messages get buffered at the server)

Thanks,

[clebert.suconic]

The size is based on the wiring size / size of the buffer. Maybe you're using a different counting for your message size?

The thing will be basically taken of Consumer credits and the wire protocol.