We have a unit test that simply places 30K keys in the cache. This is a distributed cache with numOwners = 2. So after the cache starts up, there are a total of 60K objects across the 3 nodes. These are counted by cache.entrySet().size().
Total number of entries: 29128
Total number of entries: 9463
Total number of entries: 21409
Sum: 60000
The test then removes a node and waits for the LEAVE rehash to finish:
After LEAVE rehash. Total number of entries: 30000
After LEAVE rehash. Total number of entries: 30000
The test then adds a new node back into the cluster. Here are the counts (again, through cache.entrySet().size() ).
After JOIN rehash. Total number of entries: 14969
After JOIN rehash. Total number of entries: 30000
After JOIN rehash. Total number of entries: 30000
Sum: 74969
The test is expecting only 60K objects, so it fails. If I disable this check and try to clear() each cache instance, I get the following output (waiting 10 seconds for operations to complete).
After cache.clear(). Total number of entries: 0
After cache.clear(). Total number of entries: 7588
After cache.clear(). Total number of entries: 6628
Sum: 14216
That isn't what I'm expecting at all.
The unit test is attached to https://issues.jboss.org/browse/ISPN-1012