How to implement TreeCache synchronization?

[kwatsen]

My prototype app is unstable around when adding/removing nodes to/from parents while other cluster members are walking the tree.

For instance:

1. adding a new node to the tree cache
   • one cluster member might be adding a new child node under root using:
     • treeCache.put("/foobar", key1, val1)    // this line adds the "child" node
     • treeCache.put("/foobar", key2, val2)    // this line augments the just-added "child" node
     • treeCache.put("/foobar", key3, val3)    // same as above
   • another cluster member runs:
     • Set<Object> names = root.getChildrenNames();
     • Iterator<Object> iterator = names.iterator();
     • while (iterator.hasNext()) {
       • String key = (String)iterator.next();
       • MyObject myObject = new MyObject(key)    // this constructor reads out out the cached node's fields (i.e. key1-key3)
       • etc.
     • }
   • the error is due to the 2nd cluster node thinking that the child is "fully" in the cache when it isn't.  For instance, getChildrenNames() will return the the "/foobar" node even if if it only has "key1" (i.e. it hasn't gotten key2 or key3 yet, so the MyObject() constructor gets an error when trying to dereference one of the missing keys.  I suppose the constructor could throw an exception and then the could above could step-over the entry, but I was hoping there would be a better way within treecache - and ideas?
2. removing a node from the tree cache
   • one cluster member removes a node running:
     • treeCache.removeNode(fqn)
   • another cluster member is iterating over the fields of the node being deleted
   • the error is due to a missing reader-writer lock - the removeNode should block until all readers are out and block new readers from getting in.   If it was on the same JVM, I'd use standard synchronization mechanism, but being distributed, what is the best solution?

Thanks in advance,

Kent

[galder.zamarreno]

Is this question about Infinispan's tree module, or JBoss Cache?

[kwatsen]

Sorry it wasn't clear, this is with Infinispan's tree module.

[kwatsen]

Anyone?

[mircea.markus]

the error is due to the 2nd cluster node thinking that the child is "fully" in the cache when it isn't.  For instance, getChildrenNames() will return the the "/foobar" node even if if it only has "key1" (i.e. it hasn't gotten key2 or key3 yet, so the MyObject() constructor gets an error when trying to dereference one of the missing keys.  I suppose the constructor could throw an exception and then the could above could step-over the entry, but I was hoping there would be a better way within treecache - and ideas?

I think that's your best option when working with the tree API. If working with lower level API (e.g. the Cache directly) you could use the locking API to prevent concurrent modifications. Actually thiking more about it, another approach would be to have a convention that, when either reading or writing a tree node, to acquire a certain lock (e.g. the FQN).

the error is due to a missing reader-writer lock - the removeNode should block until all readers are out and block new readers from getting in.   If it was on the same JVM, I'd use standard synchronization mechanism, but being distributed, what is the best solution?

again the synthetic lock described above should do.