-
1. Re: Data integrity after cluster merge
galder.zamarreno Jul 24, 2012 7:13 AM (in response to mattw)If you have a reference to the cache, you could call the following in order to make sure that the clear was only local:
cache.getAdvancedCache().withFlags(Flag.CACHE_MODE_LOCAL).clear()
I'm not 100% sure about the timing of the @MergeEvent wrt the state transfer of data though.
-
2. Re: Data integrity after cluster merge
mattw Jul 24, 2012 11:03 AM (in response to galder.zamarreno)HI there,
Thanks for the response, that works a treat.
I had done the same by getting the data container (getDataContainer()) and clearing that but your way looks better.
I think you're right about the state transfer, it works fine on caches with few records. However, if I stress my test setup with 90000 records across 2 caches (One replicated, one distributed) with backing stores, I sometime see inconsistent caches where records are missing. I assume the clear is still processing when the state transfer starts (and are therefore wiped). Is there any way to delay the state transfer? Or force it to happen at some known point?
Are there any other recommended ways of handling the state merge?
Thanks again for your help,
Matt
-
3. Re: Data integrity after cluster merge
galder.zamarreno Jul 25, 2012 4:51 AM (in response to mattw)If you focus only in the distributed cache for a sec, you could implement @DataRehashed annotated method and do the clear when the event.isPre=true. This even gets sent before any rehashing happens.
-
4. Re: Data integrity after cluster merge
mattw Jul 27, 2012 3:29 AM (in response to galder.zamarreno)Thanks for that suggestion but I am seeing the @DataRehashed event after a @ViewChanged event so I don't think we can use that...
Matt
-
5. Re: Data integrity after cluster merge
galder.zamarreno Aug 14, 2012 12:10 PM (in response to mattw)Did you try doing the clearing in @ViewChanged then?
-
6. Re: Data integrity after cluster merge
mattw Aug 15, 2012 3:28 AM (in response to galder.zamarreno)The @ViewChanged event gets fired on any view change, including new nodes joining. I don't want to be clearing the cache on a new new node joining...Only on the merge.
Cheers,
Matt
-
7. Re: Data integrity after cluster merge
dex80526 Aug 15, 2012 10:58 AM (in response to mattw)I think this is also related to ISPN-1586, which is specifically for replication cluster. FYI.
-
8. Re: Data integrity after cluster merge
dan.berindei Aug 21, 2012 6:19 AM (in response to dex80526)@dex Yes, ISPN-1586 is certainly related conceptually - you could regard each joiner with preexisting data in the cache store as a partition re-joining the cache. However, handling these situations is quite different.
@Matthew The local node won't send any data before @DataRehashed ends. On the other hand the local node will still accept data from the other partitions, so your clear() call may delete more than necessary.
I think we don't have a proper solution for your problem at the moment. You could probably do it by hacking JGroupsTransport to call your code before the other listeners, so that our internal listener only ever called after you do the clear(), but that has its own share of risks (and may stop working in 5.2).
Please create an issue for this, although we certainly won't have full conflict resolution for merges in 5.2 we should give you the chance to just clear the local data store before sending or receiving anything.