This is what I'm planning to do on the backup synchronization:
I - Journal
I will use some of the compacting logic for this.
1st - Disable any reclaiming, compacting
2nd - Force a move to a next file.
3rd - Starting copying the previous files to the backup node.
Any add/delete while the copy is being done can be already replicated to the backup side. The backup will have write those to the disk, but buffers what was done in parallel to update the counters (positivies and negatives) after the backup is finished. Just like compacting.
II - Large Message
I will copy any file to the backup side. The only issue is if a delete happens while the backup is being done. On that case i will have to buffer deletes that happened while the copy was being done.
III - Paging...
I still need to put a little thought on paging. Especially in regard to concurrency with the journal copy. my initial thoughs are:
- Disable depaging while the copy is being done.
- Copy all the files...
- If anything is paged while the copy is being done... .that should be buffered also
I have a few ideas about concurrent updates with the journal, but I will think about later. I have enough work to do with Journal alone and I think I know what to do on that part already.
Just an implementation detail on I:
I'm adding an operation on Journal called copyTo(Journal). I will be able to use the ReplicatedJournal (same one used at the StorageManager). So this will also bring another feature to the Journal.