JBossDeveloper

"timfox" wrote:

"clebert.suconic@jboss.com" wrote:

I would need a parameter on increment / decrement to determine if it is durable or not. The ServerMessage don't know about that

Yes, I'd add a parameter. Otherwise for durable messages you going to have to call two methods - incrementDurable() and incrementPaging() or whatever you called it.

private void doAck(final MessageReference ref) throws Exception
 {
 ServerMessage message = ref.getMessage();

 Queue queue = ref.getQueue();

 if (message.decrementRefCount() == 0)
 {
 postOffice.messageDone(message);
 }

 if (message.isDurable() && queue.isDurable())
 {
 int count = message.decrementDurableRefCount();

 if (count == 0)
 {
 storageManager.storeDelete(message.getMessageID());
 }
 else
 {
 storageManager.storeAcknowledge(queue.getPersistenceID(), message.getMessageID());
 }
 }

 queue.referenceAcknowledged(ref);
 }

As we were talking over IRC, transactions on paging are a little bit more complicated.

We could have multiple destinations in page-mode and we need to guarantee a single commit/rollback point for those.


To correctly deal with transactions on those cases, I'm planning to use the journal to control the completeness of the transaction.

This is the current solution I have in mind:

I - When a message is sent transactionally to an address in page-mode, I will aways send it with the TransactionID

II - When (before) the transaction is completed (Prepare/Commit) I will add a new record on the Journal that will contain the number of Messages on Page (I'm calling this PageTransaction). (We will hold the PageTransactions somewhere on the memory).


III - When depaging, I will verify if there is a PageTransaction available,
.......If not, I will verify if there is a transaction pending. (i.e. transaction has started but committed wasn't called yet).
.......If pending transaction -> The depaged messages will be on wait mode until the commit is called.
.......else (no pending transaction) -> The messages will be just discarded as that means the transaction was aborted somehow (rollback or server aborted on another server's execution).
.......if evertyhing was OK while depaging I will update the PageTransaction on the journal with the numberOfMessages consumed, or delete the PageTransaction if numberOfMessages = 0, so that would release the record for reclaiming.


In summary three is:

During depage:
.......It has a page-transaction
..............- update page-transaction NRMessage = NRMessage - NRMessagesOnThePage
.....................(Delete it if 0)
..............- send the messages to the PostOffice/Journal
.......It doesn't have a page-transaction
..............if there is a pending transaction.. put those messages waiting for the transaction complete
..............else... transaction is aborted



The only downside of this is if the user has 1 transaction for each message, while the user is sending 1 million messages. (1 million transactions on the memory). But that would be an anti-pattern and I don't know how we could guarantee atomicity without the journal at this point.

Saying that in English (less Java):

What I'm doing is recording the transactionIDs along with the Messages on Paging.

When depaging I verify if the transactionID was already completed or aborted on the Journal, and to ensure the Transaction will live longer I have to add some Record on the journal.

I have updated the WIKI with the current state of Paging on the Branch:

http://wiki.jboss.org/wiki/JBossMessaging2Paging

I have configured a server with -Xms64M and -Xms64M, configured MaxSize at 10M on my destination and I was able to send 500K messages to the server, without having a consumer, and later consume then as nothing special was happening.

It worked so nicely that I thought I configured something wrongly, but I double checked all the configs and everything is working fine.


I tested the 500K using 4 scenarios:

- NonPersistent, NonTransactional
- Persistent, NonTransactional
- NonPersistent, Transactional
- Persistent, Transactional


I will start merging this on trunk now, as it will be easier to fix EasyMock tests on trunk (as some other refactoring may be happening till then).

And BTW: Let me know if you plan any change around PostOffice at Trunk on the next few days.

As part of the merging into trunk I will also merge Pager and PageManager into a single class.

Also is it configurable on both an address and a global level?

"clebert.suconic@jboss.com" wrote:
I have configured a server with -Xms64M and -Xms64M, configured MaxSize at 10M on my destination and I was able to send 500K messages to the server, without having a consumer, and later consume then as nothing special was happening.

I just committed paging on trunk. It was a big commit, so I would advise everybody to update.

Things I'm still missing:

- Global size control
- Thread Pool (as discussed here: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=141551)
- I still want to change journal reload to use a FileChannel instead of loading the whole file in memory. Maybe this will be done as part of the journal optimization task later.

"timfox" wrote:
I found this cryptic code in ServerSessionImpl after your commit:

 finally
 {

 }
 //finally (TODO: enable this back)
 {
 //Now unlock

 for (Queue queue: locked)
 {
 queue.unlock();
 }
 }

Is it intentional?