-
1. Re: Do we need JournalStorageManager::ADD_MESSAGE?
timfox May 8, 2008 12:13 PM (in response to clebert.suconic)If you get rid of JournalStorageManager.ADD_MESSAGE how will you know if the record is an add when you reload the journal?
-
2. Re: Do we need JournalStorageManager::ADD_MESSAGE?
clebert.suconic May 8, 2008 12:23 PM (in response to clebert.suconic)Yes.. you're right.
But I want to get rid of that copy anyway.
I want to add a RecordType to the interface. When you read the class RecordType would store type, so no need to replicate the buffers. -
3. Re: Do we need JournalStorageManager::ADD_MESSAGE?
timfox May 8, 2008 12:35 PM (in response to clebert.suconic)Can you explain your suggestion in more detail? I'm not sure I understood it.
-
4. Re: Do we need JournalStorageManager::ADD_MESSAGE?
clebert.suconic May 8, 2008 12:47 PM (in response to clebert.suconic)Looking at JournalStorageManager, every time you use appendAddRecord or appendUpdateRecord (transactional or not), you are aways adding the RecordType to the Byte Array.
Instead of using the ByteArray to store that information I would change the signatures on addRecord, addRecordTransaction, updateRecord, updateRecordTransaction.
Something like:
appendAddRecordTransactional(long txID, byte recordType, long id, byte[] record, IOCallback callback)
appendUpdateRecordTransactional(long txID, byte recordType, long id, byte[] record, IOCallback callback) throws Exception;
And same thing for non transactional.
(Just for the record, for obvious reasons you don't need that on delete)
I would then add the byte recordType to org.jboss.messaging.core.journal.RecordInfo. -
5. Re: Do we need JournalStorageManager::ADD_MESSAGE?
clebert.suconic May 8, 2008 4:04 PM (in response to clebert.suconic)The Buffers were being created about 3 times for the AddMessage
MessagingBuffer buffer = new BufferWrapper(1024); // 1st buffer.putByte(ADD_MESSAGE); // this would buffer.putBytes(message.encode().array()); //2nd messageJournal.appendAddRecord(message.getMessageID(), buffer.array()); // 3rd
With this code, a class calling JournalStorageManager directly was performing about 30K inserts, with intermediate commits (what would require them to wait).
I have made few changes:
I have introduced an interface EncondingSupport:public interface EncodingSupport { int encodeSize(); void encode(MessagingBuffer buffer); }
And on the Journal it's possible to pass an Encodingsupport. With that it is possible to avoid at least few copies, writing to the final and necessary buffer only. (before calling journal.append).
I could get twice about 60K writes/second.
I know the transport now won't let me go above 8K / second based on NonPersistentMessages (on my computer at least)... but even that way I got some higher numbers on few tests I made.
I'm still not ready to commit though... it should be another day :-) -
6. Re: Do we need JournalStorageManager::ADD_MESSAGE?
timfox May 9, 2008 6:06 AM (in response to clebert.suconic)Sounds good . The less copies the better :)