Paging discussion:

[clebert.suconic]

I have started a WIKI for paging:

http://wiki.jboss.org/wiki/JBossMessaging2Paging

The basic idea is to have a FileBasedCollection where we store Messages, having an eviction cache based on watermark, and a FileBasedQueue for messageReferences.

Both FileBasedQueue and FileBasedCollection would share the same implementations.

I'll use SequentialFile to access the disk, so if we decide to go AIO later we could easily change it.

One thing I'm doing differently from what I suggested on the face2face meeting was regarding the MessageReferences.

During the f2f I thought about not requiring a separated file for MessageReferences where those references would just be a pointer on the main DataFile. but then I realized that in some events the queue could eventually get out of order. (such as redelivers caused by rollbacks). So, on that case we need a separated data structure for MessageReferences.

The wiki so far only has some ideas and requirements. I will update it as soon as I have some UML diagram.

[timfox]

I have started a WIKI for paging:

http://wiki.jboss.org/wiki/JBossMessaging2Paging

The basic idea is to have a FileBasedCollection where we store Messages, having an eviction cache based on watermark, and a FileBasedQueue for messageReferences.

Both FileBasedQueue and FileBasedCollection would share the same implementations.

I'll use SequentialFile to access the disk, so if we decide to go AIO later we could easily change it.

One thing I'm doing differently from what I suggested on the face2face meeting was regarding the MessageReferences.

During the f2f I thought about not requiring a separated file for MessageReferences where those references would just be a pointer on the main DataFile. but then I realized that in some events the queue could eventually get out of order. (such as redelivers caused by rollbacks). So, on that case we need a separated data structure for MessageReferences.

The wiki so far only has some ideas and requirements. I will update it as soon as I have some UML diagram.

Can you elaborate more why you need different files for messages and message references? I found it hard to understand from your wiki.

Also I didn't really understand the part about indexes.

Some deeper explanation on the wiki would be good. I think it's currently quite hard to work out your intentions from that page, and how they differ from what was discussed in Paris.

Also, please can you create a new branch for this work?

[clebert.suconic]

Can you elaborate more why you need different files for messages and message references? I found it hard to understand from your wiki.

I'll reformulate the WIKI, but let me explain this part here.


- Say you have one Address with 10 Messages:

Msg1, 2, 3, 4, 5, 6, 7, 8, 9, 10


- You have two consumers on that Binding. One of them consumed messages 1,2 and 3, the other one consumer 4, 5 and 6. Now the first consumer rolled back the ACKs while the second consumer committed the messages:


After these two Consumers operation, the Binding will have:

Msg1,2,3, 7, 8, 9 (4,5 and 6 are gone)


While another Binding of the same address could still have:

Msg 1,2,3,4,5,6,7,8,9,10


so... because of that we will need the references on a separated data structure.

[timfox]

What is the "Binding Queue"?

[timfox]

What you're saying is that different queues bound under the same address could end up with message references in different orders after a cancel condition.

That's correct, but I don't see what relation it has to paging.

Paging occurs at the address level - *before* the message is routed. Paging is done at the message and address level, not the queue and message reference level.

Once a message has been routed there's no paging involved.

[clebert.suconic]

If paging is done before routing, this is much simpler then.

Paging will be just a disk buffer before PostOffice routing.

I was aways thinking of an edge case condition what would force us to do some control by Binding, and not by Address.

[timfox]

Yes, exactly.

Paging works on the address and message level, not on the queue and message reference level.

Paging comes before routing. You'll also need some kind of callback mechanism to trigger reloading from paging store when space frees up in an address.

[clebert.suconic]

One thing I did on the Paging Branch was to create three new methods on ServerMessage to control reference counting.

int decrementRefCount();

 int incrementRefCount();

 int getRefCount();

With that I can discount the size of a queue when refCount == 0, and start depaging as soon as the size reaches some minimal watermark.

The paging logic is already working on the branch. I still have some work to do such as configuring paging size individually and some other small things, but I'm almost there.

[timfox]

One thing I did on the Paging Branch was to create three new methods on ServerMessage to control reference counting.

With that I can discount the size of a queue when refCount == 0, and start depaging as soon as the size reaches some minimal watermark.

Can you explain more, I didn't get that.

[clebert.suconic]

I have added some size controlling per Address on the PostOffice.

Whenever a message is routed, I'm adding the getEncodeSize() of the message to the PostOffice, and when the message is develiered I'm subtracting the queue size on the PostOffice.

When the size hits the max size I set the address in page mode, and after that moment all the messages are being sent to the page instead of routing.

To perform the "when the message is delivered" control, I am adding the reference counting logic. When the counter==0 on the ACK of the message, I'm calling a method on the postOffice informing that the message was delivered and that method is subtracting the encode size of the address. If the size < watermark I start a depage thread.

The address will be in page mode until all the page files are consumed, and at that point the Address comes back to its regular state automatically. (There is a startPaging method on PagingManager, and there is no stopPaging method. The "stopPaging" event happens when everything is dequeued).

[timfox]

I have added some size controlling per Address on the PostOffice.

Whenever a message is routed, I'm adding the getEncodeSize() of the message to the PostOffice, and when the message is develiered I'm subtracting the queue size on the PostOffice.

You should do the subtraction when the message is completely *acknowledged* (i.e. acked by all references) not when *delivered*. When it's delivered it's still in memory (the deliveries list in the session).

To perform the "when the message is delivered" control,

You mean "when the message is acked" :)

I am adding the reference counting logic. When the counter==0 on the ACK of the message, I'm calling a method on the postOffice informing that the message was delivered and that method is subtracting the encode size of the address. If the size < watermark I start a depage thread.

This is done on a separate thread?

ServerMessage already does reference counting - but only for durable messages. Instead of adding new methods you could just extend what is already there.

[clebert.suconic]

This is done on a separate thread?

Yes...

ServerMessage already does reference counting - but only for durable messages. Instead of adding new methods you could just extend what is already there.

My first try was to have a single ref-counting for everything, and an attribute for isDurable on the message, but I realized that you could have some of the bindings durable, and some nonDurable.

So.. you would need to differentiate the two ref-countings. The Durable-ref-counting is used to delete the message from store, while the other one is used to subtract the size of the Address what will happen regardless of the message being durable or not.

[clebert.suconic]

You should do the subtraction when the message is completely *acknowledged* (i.e. acked by all references) not when *delivered*. When it's delivered it's still in memory (the deliveries list in the session).

You mean "when the message is acked" :)

Yes.. when the message is acked.

[timfox]

My first try was to have a single ref-counting for everything, and an attribute for isDurable on the message, but I realized that you could have some of the bindings durable, and some nonDurable.

Yes, durable is effectively an attribute of the reference not the message

boolean durable = ref.getMessage().isDurable() && ref.getQueue().isDurable().

So.. you would need to differentiate the two ref-countings.

Yes, the message can maintain a durable count and a non durable count, but you can use the same methods for incrementing / decrementing.

[clebert.suconic]

Yes, the message can maintain a durable count and a non durable count, but you can use the same methods for incrementing / decrementing.

I would need a parameter on increment / decrement to determine if it is durable or not. The ServerMessage don't know about that

Besides based on where the calls are made it is easier to keep two separate methods for each increment and decrement.