I think I have isololated this as a ModeShape issue versus Infinispan in that in my application, I am first establishing an Infinispan CacheMgr and binding to JNDI, then passing that JNDI name to ModeShape. So, using that cacheMgr, I can directly set a key/value pair in Infinispan itself using a put("mykey", "myvalue") and have verified that the value is distributed to the other JBoss node. Same, if the 2nd JBoss node changes that value, it is reflected at the 1st JBoss node as expected. So, at the base Infinispan level within my application, clustering is working.
So, has anyone had experience wit this use case with ModeShape and trying to cluster the Infinispan backend source?
With ModeShape 1.1.0, I know it was at least trying to copy the data between Infinispan nodes as it was throwing Serialization errors. Since that was fixed in v1.2.0, those errors are no longer seen, but the data is not being copied between nodes at all. Any ideas?
Hmm. There's no caching of values on the Infinispan connector, so that's not the issue. There is caching of values within the JCR session, so that would keep you from seeing updates to existing nodes unless you explicitly refreshed the session. Somehow, I doubt that's the problem here and that wouldn't have any impact on new nodes (i.e., read non-existent node /a in session 1, create node /a in session 2, read node /a in session 1 should work).
I will take a look at this tonight, Any chance that you could post your code to give me a bit of a head start?
Thanks for digging into this!
Thanks Brian. I PM'd you my email and I can then email you the source.
1 of 1 people found this helpful
Got your PM, but I am going to make an educated guess that you did not set the rootNodeUuid property on your sources. The default rootNodeUuid behavior varies from connector to connector at the moment. The Infinispan connector happens to use a random UUID. If you have two sources clustered together with different root node UUIDs, the two sources will share all node data, but path-based lookups won't work.
Disclaimer: I have only tested this at the graph layer so far, but it worked fine consistently once I set the rootNodeUuid to the same value on both sources. Well, after I set the rootNodeUuid property and after I set -Djava.net.preferIPv4Stack=true.
I am going to open up a defect to add better default behavior for the Infinispan connector rootNodeUuid property and add some documentation to the Reference Guide on how to set this up.
I just committed a fix for MODE-764 that changes the default value for the rootNodeUuid to be a constant. It also adds two test cases (one at the graph level and one at the JCR level) that demonstrate the Infinispan source working in a cluster. I based it on your configuration from the original post, so you should be good to go after you set the rootNodeUuid property.
Sorry for the delay in figuring this out. Thanks for catching and reporting this!
Thanks Brian, that was it!