There's no atomicmap replication in this configuration

[nikolay1981]

Hi,

inifinispan                5.1.0 Beta1

JAVA                       1.6.0_23

useLockStriping            false

isolationLevel             REPEATABLE_READ

stateRetrieval             fetchInMemoryState="true"

clustering mode            replication async asyncMarshalling="true" useReplQueue="true" replQueueInterval="200"

networking                 TCP

TM                         JBossStandaloneJTAManager

deadlockDetection          false

Here's a simple test.

The same code is used by both nodes.

Each node runs loop of iterations, each iteration modifies only one element randomly chosen from 2 elements with the keys: '0' and '1'

and then prints immediately values of both elements key: '0' or key: '1'. Node puts random value 0-9 and its name A or B.

2 nodes competes for these elements with keys 0 and 1.

Output like this "2-A,7-B" means element key 0 has been updated by node A with value 2, key 1 has been updated by node B with value 7

1st phase - Map is being updated

2nd phase - cache itself is used to store elements

during 1st phase changes made by nodes are NOT propagated to the opposite node. This is wrong behaviour. Why is it happening?

during 2nd phase changes made by nodes are being properly propagated to the opposite node. This is correct behaviour.

Code, logs and ouputs are zipped and attached.

Thank you.

     [java] Fri Oct 07 21:49:16 EDT 2011 *******************************************
     [java] Fri Oct 07 21:49:16 EDT 2011 ************* PHASE 1     MAP   ***********
     [java] Fri Oct 07 21:49:16 EDT 2011 *******************************************
     [java] Fri Oct 07 21:49:17 EDT 2011 null,7-A,
     [java] Fri Oct 07 21:49:17 EDT 2011 4-A,7-A,
     [java] Fri Oct 07 21:49:17 EDT 2011 4-A,8-A,
     [java] Fri Oct 07 21:49:17 EDT 2011 3-A,8-A,
     [java] Fri Oct 07 21:49:18 EDT 2011 4-A,8-A,
     [java] Fri Oct 07 21:49:18 EDT 2011 4-A,6-A,
     [java] Fri Oct 07 21:49:18 EDT 2011 4-A,4-A,
...skipped

     [java] Fri Oct 07 21:49:27 EDT 2011 *******************************************
     [java] Fri Oct 07 21:49:27 EDT 2011 ************* PHASE 2     CACHE ***********
     [java] Fri Oct 07 21:49:27 EDT 2011 *******************************************
     [java] Fri Oct 07 21:49:27 EDT 2011 null,4-A,
     [java] Fri Oct 07 21:49:27 EDT 2011 null,4-A,
     [java] Fri Oct 07 21:49:27 EDT 2011 null,0-A,
     [java] Fri Oct 07 21:49:28 EDT 2011 null,2-A,
...skipped

     [java] Fri Oct 07 21:49:30 EDT 2011 1-A,0-A,
     [java] Fri Oct 07 21:49:30 EDT 2011 5-A,0-A,
     [java] Fri Oct 07 21:49:30 EDT 2011 5-A,0-A,
     [java] Fri Oct 07 21:49:31 EDT 2011 6-B,8-A,
     [java] Fri Oct 07 21:49:31 EDT 2011 1-B,5-A,
     [java] Fri Oct 07 21:49:31 EDT 2011 4-A,5-A,
     [java] Fri Oct 07 21:49:31 EDT 2011 4-A,2-A,
     [java] Fri Oct 07 21:49:32 EDT 2011 8-B,3-A,
     [java] Fri Oct 07 21:49:32 EDT 2011 1-B,6-A,
...skipped

[galder.zamarreno]

Hmmm, I'm sorry but I don't understand what behaviour exactly you're trying to assert.

If the issue is to do with ordering of operations, then asyncMarshalling should be set to false.

If the issue is about operations not reaching the other node, you need to remember that this is asynchronous mode and you need to factor some wait time from the moment you make modifications in one node (i.e. call cache.put()) to seeing the changes in the other node.

[nikolay1981]

Thank you for a reply.

The behavior is replication of modifications of atomcmap elements.

node A

map.put(random.nextInt(2), 'A')

node B

map.put(random.nextInt(2), 'B')

values dump should be A,A  A,B  B,A  B,B but not only A,A or B,B

Two nodes modifing a map with 2 elements. Each transaction modifies only 1 element. So situations when different elements get updated by different nodes happens quite often and map values dump looks like A, B or B, A. But if there was no replication then map values dump will looks like A, A or B, B (each time transaction modifies random element 1st of 2nd and assign node name A or B).

Replication queue is flushed every 200ms. There's 200ms pause between transactions and each transaction lasts atleast 20ms.

The example shows that cache.put(key, value) is being propagated properly to another node, but atomcMap.put(key, value) is not. I tested that in both modes udp, tcp. I modified the code, adding

            for(;cache.getCacheManager().getMembers().size() < 2;){

                Thread.sleep(1000);

            }

to make sure that merging is done before 1st stage.

I dont care about the ordering and have asyncMarshalling="false"
<async asyncMarshalling="false" useReplQueue="true" replQueueInterval="200"/>

Result is the same.

Take a look please at the zip file attached.

Thank you.

[nikolay1981]

Ok. I've found an interesting thing. If I change useReplQueue from "true"  to "false" I get atomic map replicated.

Actually this is very strange to see useReplQueue configuration option. Async replication in itself imply some queue to keep replication chunks.  So why is there posibility to assign false to useReplQueue?

[galder.zamarreno]

async means JGroups (the clustering communications toolkit) does a fire and forget

async + repl queue means that operations are queued and passed down to JGroups layer, which in turns does fire and forget

In the 2nd option, the requests return earlier than in the 1st option.

[nikolay1981]

Thank you for explanation.

By the way did you try to run my example with useReplQueue true/false ?

[galder.zamarreno]

No, I have not tried your example yet. If you think there's a bug, I'd suggest you open a JIRA in https://issues.jboss.org/browse/ISPN and attach the example you have. Even better, if you could transform the example into a test case where you can assert a failure, that'd be great. You can use our own test case maven archetype to make your life easier: https://docs.jboss.org/author/display/ISPN/Infinispan+Maven+Archetypes#InfinispanMavenArchetypes-WritingatestcaseforInfinispan