Version 3

    State transfer or rehashing are the means by which newly started nodes bring data from other nodes in replicated and distributed cache respectively. In the case of state transfer, this means bringing the entire cache contents from the coordinator of the replicated cluster, and in the case of rehashing, it means that certain data will migrate from a neighbouring node to the newly joined node in order to maintain data in the node that consistent hashing mandates. Regardless of the method used, in both cases data needs to go over the network to these new nodes and if the data being shipped around is big, this can put some strain in the network infrastructure. So, this wiki tries to come up with new ways to do state transfer that do not fully rely on the network infrastructure. For example:


    1. When a node is stopped, it could have a local cache store configured to which it dumps the memory contents.
    2. Upon restart, the node could:
      1. If using state transfer, get a digest of the keys and versions (lamport or vector clocks, or similar) from both the coordinator and the local store (concurrently), and decide:
        1. which data has not been updated and so can be loaded from the local cache store, and
        2. which new data or more up-to-date data the coordinator has and retrieve that from the coordinator over the network.
      2. If using rehashing, the digest of keys and versions would be pushed to this node, and the rest would work exactly the same.
        1. An interesting optimisation for the rehashing case is that the newly started node could decide to get a particular key's data from a preferred data owner, such as a node that's running on the same machine or the local rack, as opposed to a node accross a WAN.
    3. If the node crashed or any failure was encountered while dumping the memory contents, it would fallback on retrieving everything from the network.


    Clearly, this functionality would benefit a lot the following use cases:

    • Situations where nodes are just simply restarted for maintenance, where the deltas between what's been stored locally and retrieved from the network are small.
    • Storing big amounts of data, for example DVDs. If the data set is small, the additional computation required would just add latency.