I have a 2 node cluster setup in replication mode (synchronous). Additionally I'm using JTA (standalone with jbossjta 4.15.3.Final). I'm using infinispan 5.1.5.FINAL. My code does the following in a loop:

begin tx:

long now = System.currentTimeMillis();

// create key using time of next closest 100 millisecond

String key = String.valueOf(now + (100 - now % 100) == null);

if (cache.putIfAbsent(key, key)) {

  add++

} else {

  dup++;

}

end tx

I run this on two different machines, same subnet, and write the add/dup stats to a log file. The clocks are in sync so they both average about the same percentage of adds and dups. After it runs for a minute or so I compare the add stats between machines. What I'm seeing is a small number of duplicate adds between the nodes, meaning that each node applied the same key to the map, which violates the putIfAbsent idiom. Is this expected behavior? If not, how can I prevent it? The Jgroups config is using TCP/TCPPING. Here's my Infinispan config:

<?xml version="1.0" encoding="UTF-8"?>

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"

xmlns="urn:infinispan:config:5.1">

<global>

  <transport clusterName="cluster" nodeName="node1">

  <properties>

  <property name="configurationFile" value="jgroups-tcp.xml" />

  </properties>

  </transport>

  <globalJmxStatistics enabled="false" />

  <shutdown hookBehavior="DONT_REGISTER" />

</global>

<default>

  <deadlockDetection enabled="true" spinDuration="1000" />

  <jmxStatistics enabled="false" />

</default>

<namedCache name="test">

  <eviction strategy ="NONE"/>

  <clustering mode="replication">

  <stateTransfer fetchInMemoryState="true" timeout="240000" />

  <sync replTimeout="15000" />

  </clustering>

  <transaction

  transactionMode="TRANSACTIONAL"

  useSynchronization="false"

  cacheStopTimeout="30000"

  syncCommitPhase="true" 

  syncRollbackPhase="false"

  transactionManagerLookupClass="org.infinispan.transaction.lookup.JBossStandaloneJTAManagerLookup"

  lockingMode="OPTIMISTIC" />

  <locking

  isolationLevel="READ_COMMITTED"

  lockAcquisitionTimeout="15000" 

  useLockStriping="false" />

  <expiration lifespan="-1" maxIdle="-1" wakeUpInterval="5000"/>

</namedCache>

</infinispan>