JBoss Messaging Circular Journal Layout
Header Id
Every file on the journal contains a 4 bytes (integer) header with the current identification of the file. Every time the file is reused over reclaiming this field is updated with the unique ID.
When the journal is created the first created file will be 1, and when it is loaded the global counter will contain the max value found over the journal.
Basic Record Definition
Every record used by JournalImpl will have this basic definition:
FieldName | Size |
|---|---|
RecordType | 1 Byte |
FileID | 4 Bytes (Integer) |
TransactionID (If transaction record) | 8 Bytes (Long) |
RecordID | 8 Bytes (Long) |
VariableSize (Used on Add and Update records) | 4 Bytes (Integer) |
RecordType (Used on Add and Update records) | 1 Byte |
Record Body (Used on Add and Update records) | VariableSize Bytes |
Check Size | 4 Bytes (Integer) |
File Id
Every record will also contain the FileID that should be the same as HeaderID. If FileID != HeaderID that means the record belongs to a previous usage so this record should be ignored.
Variable Size
There is a possibility of this data being corrupted because of interrupted rights (caused by a failure for example). Every time we read a record we need to verify this VariableSize is consistent, what means not negative. The CheckSize is also to validate the health of the record. (Expected record size + VariableSize = CheckSize).
Check Size
We record the number of bytes written for each record to make sure the entire record was recorded.
Record Health
A record is only considered ok to be reloaded if:
RecordType between 11 and 19 on position 1
FileID = HeaderID on position 2
CheckSize on the latest 4 bytes matches the number of bytes written. (i.e. last position -4 CheckSize = Position where the record started)
A record is also considered unhealthy if VariableSize is negative CurrentPosition + VariableSize + ExtraBodySize > FileLength.
Calculated Hash
We could maybe replace the CheckSize by calculated hash over the body. Such calculation needs to be defined but this could be easily done. The idea of the CheckSize is to guarantee the health of a record and a calculated hash could also bring the same benefit.
-
Reading Records
A record could be anywhere on the file. There is no guarantee records will be placed side by side on the files.
So Reload will aways have to look for the next RecordType between 11 and 19, and when that record is found the health of the record should be tested.
Reload will keep looking for any health record on the file.
This is because of alignment on AIO, and possible holes caused by interruptions on the process of reading the file.
Holes on Transactions
A record on a complete transaction (commit or prepare) holds the number of elements used on the transaction/ per journal file.
Say.. if a transaction was spread between 3 different journal files (because of the size, or the time it took to be completed (because of the user's interaction maybe)), it will contain the number of elements used by each record.
When the journal is reloaded, all the transactions are supposed to be reloaded before we can consider the transaction safe. However if a file which was part of the transaction was gone that means the file was safely reclaimed and we could still consider the transaction safe.
Transaction Layout:
Field Name | Size |
|---|---|
Record Type (Commit = 18 or PREPARE = 17) | 1 Byte |
File Id | 4 Bytes (Integer) |
Transaction Id | 8 Bytes (Long) |
Number Of Files | 4 Bytes (Integer) |
Repeated by NumberOfFiles:
File ID | 4 Bytes (Integer) |
Number of records on that file | 4 Bytes (Integer) |
And at the latest 4 bytes:
Check Size | 4 Bytes (Integer) |
Comments