Basic Replication Configuration Question
monty-temboo Jun 16, 2011 6:45 PMI've been playing with the 2 node example on github. I modified it to have Node0 just print out the size of the cache every few seconds. Node1 puts 100 random strings as keys and values in. I also modified the waitForClusterToForm() method to listen for the cacheStarted() method and then sleep a second before allowing the node to start reading or writing.
So, in playing around, I started the reader node, then a writer and saw 100 things in there. Started another writer and saw 200 things in there. Started another reader, and that one showed 0 items. Ran another writer and the first reader then showed 300, second reader 100 items.
I have replication turned on. Here is my config, pretty much straight out of the example directory, but with no jgroups xml referenced.
<?xml version="1.0" encoding="UTF-8"?>
<infinispan
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:infinispan:config:5.0 http://www.infinispan.org/schemas/infinispan-config-5.0.xsd"
xmlns="urn:infinispan:config:5.0">
<global>
<globalJmxStatistics
enabled="true"
jmxDomain="org.infinispan"
cacheManagerName="SampleCacheManager"/>
<transport
clusterName="infinispan-cluster"
machineId="m1"
rackId="r1" nodeName="Node-A">
</transport>
</global>
<default>
<locking
isolationLevel="READ_COMMITTED"
lockAcquisitionTimeout="20000"
writeSkewCheck="false"
concurrencyLevel="5000"
useLockStriping="false"
/>
<jmxStatistics enabled="true"/>
<clustering mode="replication">
<stateRetrieval
timeout="240000"
fetchInMemoryState="false"
alwaysProvideInMemoryState="false"
/>
<sync replTimeout="20000"/>
</clustering>
</default>
</infinispan>
My cache manager is created like this:
this.cacheManager = new DefaultCacheManager("infinispan.xml");
I imagine that there is some other configuration I need to do to make this work. After looking again at the config, I decided to try setting fetchInMemoryState and alwaysProvideInMemoryState to true. If I do that I get an exception like this for anything other than the first node up:
15:35:26,374 INFO JGroupsTransport -- ISPN00078: Starting JGroups Channel
15:35:26,374 INFO JGroupsTransport -- ISPN00088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
15:35:26,424 INFO JChannel -- JGroups version: 2.12.0.Final
15:35:27,027 WARN UDP -- receive buffer of socket java.net.DatagramSocket@1fea6a1c was set to 20MB, but the OS only allocated 65.51KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
15:35:27,027 WARN UDP -- receive buffer of socket java.net.MulticastSocket@56dc64a2 was set to 25MB, but the OS only allocated 65.51KB. This might lead to performance problems. Please set your max receive buffer in the OS correctly (e.g. net.core.rmem_max on Linux)
15:35:30,208 INFO JGroupsTransport -- ISPN00094: Received new cluster view: [Node-A-16741|1] [Node-A-16741, Node-A-63371]
15:35:30,309 INFO JGroupsTransport -- ISPN00079: Cache local address is Node-A-63371, physical addresses are [fe80:0:0:0:21f:5bff:feb6:9ae3%5:61547]
15:35:30,310 INFO GlobalComponentRegistry -- ISPN00128: Infinispan version: Infinispan 'Pagoa' 5.0.0.CR5
15:35:30,367 WARN GMS -- Node-A-63371: not member of view [Node-A-16741|2] [Node-A-16741]; discarding it
15:35:30,582 INFO CacheJmxRegistration -- ISPN00031: MBeans were successfully registered to the platform mbean server.
15:35:30,583 INFO RpcManagerImpl -- ISPN00074: Trying to fetch state from Node-A-16741
15:35:30,597 WARN STREAMING_STATE_TRANSFER -- State reader socket thread spawned abnormaly
java.net.NoRouteToHostException: No route to host
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
at java.net.Socket.connect(Socket.java:525)
at org.jgroups.util.Util.connect(Util.java:276)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:510)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:462)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:223)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:189)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:891)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:613)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:294)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703)
at org.jgroups.protocols.BARRIER.up(BARRIER.java:119)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:177)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:275)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:209)
at org.jgroups.protocols.Discovery.up(Discovery.java:291)
at org.jgroups.protocols.PING.up(PING.java:66)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1102)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1658)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1640)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:637)
15:35:30,599 WARN STREAMING_STATE_TRANSFER -- Could not connect to state provider. Closing socket...
All of this is running on the same machine, by the way.