consistent hash of distriubtion with relay behaviour
dex80526 Jul 23, 2012 5:14 PMAcooring to Jgroups document (http://www.jgroups.org/manual/html/user-advanced.html#RelayAdvanced), the change to a local cluster will be replicated by the RELAY to other cluster. But, in my testing, the behaviour is different (detailed below). I used gui-demo code from Infinispan distribution. The observed behaviour is indeed like the 2 local cluster formed ONE big cluster.
Here is my configuration.
1) nyc cluster 1 (2 nodes with distribution and numOwners = 2); zrh cluster 2 (single node cluster, distribution and numbOwners=2)
2) I use TCP for both local and bridge
After I ran Infinispan demo instance on 3 different nodes to form 2 local clusters and bridge them with RELAY, the GUI shows 2 node from site nyc and one from zrh, but only one Coordinator.
Then, I tried to add data entries from node1 (nyc), the data will be distributed to antoher node, only one (based the hash order). This could be the second node node2 (nyc) or node(zrh), but, the data never apper on all 3 nodes.
This seems to me is the expected behavior of all 3 node form a single cluster with DISTRIBUTION and numOwners=2.
Expected behavior of RELAY: the data should appear on all 3 node in this case.
Do I misss somthing here? or it is a bug?
Here are the configuration:
relay1.xml:
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
xmlns="urn:infinispan:config:5.1">
<global>
<transport clusterName="c1"
machineId="c1-node1"
siteId="nyc"
rackId="r1" nodeName="hanode1"
>
<properties>
<property name="configurationFile" value="./jgroups-tcp-relay1.xml" />
</properties>
</transport>
<globalJmxStatistics enabled="true"/>
</global>
<default>
<locking
isolationLevel="READ_COMMITTED"
lockAcquisitionTimeout="20000"
writeSkewCheck="false"
concurrencyLevel="5000"
useLockStriping="false"
/>
<deadlockDetection enabled="true" spinDuration="1000"/>
<jmxStatistics enabled="true"/>
</default>
<namedCache name="keychain" >
<clustering mode="distribution">
<l1 enabled="false" lifespan="10000"/>
<hash numOwners="2" rehashWait="5000" rehashRpcTimeout="10000" />
<async/>
</clustering>
</namedCache>
</infinispan>
jgroups-tcp-relay1.xml:
....
<TCPPING timeout="3000"
initial_hosts="${jgroups.tcpping.initial_hosts:node1[7800],node2[7800]}"
port_range="1"
num_initial_members="2"
ergonomics="false"
/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD timeout="3000" max_tries="3"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK
use_mcast_xmit="false"
retransmit_timeout="300,600,1200,2400,4800"
discard_delivered_msgs="false"/>
<UNICAST2 timeout="300,600,1200"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
max_bytes="400000"/>
<pbcast.GMS print_local_addr="false" join_timeout="7000" view_bundling="true"/>
<UFC max_credits="2000000" min_threshold="0.10"/>
<MFC max_credits="2000000" min_threshold="0.10"/>
<FRAG2 frag_size="60000"/>
<RELAY site="nyc" bridge_name="dr_bridge" level="TRACE" bridge_props="./tcp.xml" />
</config>
tcp.xml:
....
<TCPPING timeout="3000"
initial_hosts="${jgroups.tcpping.initial_hosts:node1[8900], node2[8900], node3[8900]}"
port_range="1"
num_initial_members="3"
ergonomics="false"
/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD timeout="3000" max_tries="3"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK
use_mcast_xmit="false"
retransmit_timeout="300,600,1200,2400,4800"
discard_delivered_msgs="false"/>
<UNICAST2 timeout="300,600,1200"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
max_bytes="400000"/>
<pbcast.GMS print_local_addr="false" join_timeout="7000" view_bundling="true"/>
<UFC max_credits="2000000" min_threshold="0.10"/>
<MFC max_credits="2000000" min_threshold="0.10"/>
<FRAG2 frag_size="60000"/>
</config>