Cross site replication - Infinispan 7.2/8

[srinivasan_ss]

Hi

This thread is related to cross site replication.

1. Looking for Infinispan 7.2/8 examples for cross site replication. The Infinispan 7.2/8 user guide points to old version of cross site XMLs.
2. Can cross site example be deployed in the same machine and tested?
3. I tried to run 2 JBoss servers on a single machine and deployed a WAR with embedded Infinispan with different Infinispan site names - CHARLOTTE and DENVOR. Got the below error while Infinispan tried to replicate the data from Charlotte site to Denvor site:

09:48:31,229 ERROR [org.springframework.batch.core.step.AbstractStep] (TestJobThread-1) Encountered an error executing step PROCESS in job TestJob: The local cache ___defaultcache failed to backup data to the remote sites:
DENVOR: org.infinispan.remoting.CacheUnreachableException: UnreachableException:
member=SiteMaster(DENVOR)

    Charlotte site xmls:

•     infinispan.xml:

<?xml version="1.0" encoding="UTF-8"?>
<infinispan
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="urn:infinispan:config:7.2 http://www.infinispan.org/schemas/infinispan-config-7.2.xsd"
      xmlns="urn:infinispan:config:7.2">
  <jgroups transport="org.infinispan.remoting.transport.jgroups.JGroupsTransport">
    <stack-file  name="external-file" path="jgroups.xml"/>           
  </jgroups> 
    <cache-container default-cache="default" statistics="true">
    <transport cluster="TveApp" stack="external-file" site="CHARLOTTE"/>
    <distributed-cache name="default" mode="SYNC">
  <backups>
  <backup site="DENVOR" failure-policy="FAIL" strategy="SYNC" timeout="12000" enabled="true" two-phase-commit="true"/>
  </backups>
  </distributed-cache>
    </cache-container>
</infinispan>

• relay2.xml

<?xml version="1.0" encoding="UTF-8"?>
<RelayConfiguration xmlns="urn:jgroups:relay:1.0">
  <sites>
        <site name="CHARLOTTE" id="CHARLOTTE">
            <bridges>
                <bridge config="jgroups-relay2.xml" name="global"/>
            </bridges>
        </site>
        <site name="DENVOR" id="DENVOR">
            <bridges>
                <bridge config="jgroups-relay2.xml" name="global"/>
            </bridges>
        </site>
  </sites>
</RelayConfiguration>

• jgroups.xml

<config xmlns="urn:org:jgroups" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:org:jgroups file:schema/JGroups-3.1.xsd">
    <TCP    bind_addr="${jgroups.tcp.address:127.0.0.1}" bind_port="${jgroups.tcp.port:7800}" loopback="true" port_range="30" recv_buf_size="20m"
            send_buf_size="640k" discard_incompatible_packets="true" max_bundle_size="64000" max_bundle_timeout="30" enable_bundling="true"
            use_send_queues="true" enable_diagnostics="false" bundler_type="old" thread_naming_pattern="pl" thread_pool.enabled="true"
            thread_pool.min_threads="2" thread_pool.max_threads="30" thread_pool.keep_alive_time="60000" thread_pool.queue_enabled="true"
            thread_pool.queue_max_size="100" thread_pool.rejection_policy="Discard" oob_thread_pool.enabled="true" oob_thread_pool.min_threads="2"
            oob_thread_pool.max_threads="30" oob_thread_pool.keep_alive_time="60000" oob_thread_pool.queue_enabled="false" oob_thread_pool.queue_max_size="100"
            oob_thread_pool.rejection_policy="Discard" internal_thread_pool.enabled="true" internal_thread_pool.min_threads="2" internal_thread_pool.max_threads="4"
        internal_thread_pool.keep_alive_time="60000" internal_thread_pool.queue_enabled="true" internal_thread_pool.queue_max_size="100" internal_thread_pool.rejection_policy="Discard"/>

    <MPING bind_addr="${jgroups.bind_addr:127.0.0.1}" break_on_coord_rsp="true"  mcast_addr="${jgroups.mping.mcast_addr:228.2.4.6}"
          mcast_port="${jgroups.mping.mcast_port:43366}" ip_ttl="${jgroups.udp.ip_ttl:2}" num_initial_members="3"/>

    <MERGE2 max_interval="30000" min_interval="10000"/>
    <FD_SOCK/>
    <FD timeout="3000" max_tries="3"/>
    <VERIFY_SUSPECT timeout="1500"/>
    <pbcast.NAKACK2 use_mcast_xmit="false" xmit_interval="1000" xmit_table_num_rows="100" xmit_table_msgs_per_row="10000" xmit_table_max_compaction_time="10000" max_msg_batch_size="100"/>

    <UNICAST2 stable_interval="5000" xmit_interval="500" max_bytes="1m" xmit_table_num_rows="20" xmit_table_msgs_per_row="10000" xmit_table_max_compaction_time="10000" max_msg_batch_size="100"/>
    <pbcast.STABLE stability_delay="500" desired_avg_gossip="5000" max_bytes="1m"/>
    <pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>
    <UFC max_credits="200k" min_threshold="0.20"/>
    <MFC max_credits="200k" min_threshold="0.20"/>
    <FRAG2 frag_size="60000"/>
    <RSVP timeout="60000" resend_interval="500" ack_on_delivery="false" />
  <relay.RELAY2 site="CHARLOTTE" config="relay2.xml" relay_multicasts="false" />
</config>

• jgroups-relay.xml

<config>
    <TCP bind_port="7200" loopback="true"  port_range="30" recv_buf_size="20000000" send_buf_size="640000" discard_incompatible_packets="true" max_bundle_size="64000"
        max_bundle_timeout="30" enable_bundling="true" use_send_queues="true" sock_conn_timeout="300" enable_diagnostics="false" bundler_type="old" send_queue_size="0"
        thread_pool.enabled="true" thread_pool.min_threads="1" thread_pool.max_threads="8" thread_pool.keep_alive_time="5000" thread_pool.queue_enabled="false"
        thread_pool.queue_max_size="100"  thread_pool.rejection_policy="Run" oob_thread_pool.enabled="true" oob_thread_pool.min_threads="1" oob_thread_pool.max_threads="8"
        oob_thread_pool.keep_alive_time="5000" oob_thread_pool.queue_enabled="false" oob_thread_pool.queue_max_size="100" oob_thread_pool.rejection_policy="Run"/>
    <TCPPING timeout="3000" initial_hosts="localhost[7200]" port_range="3" ergonomics="false"/>
    <FD_SOCK/>
    <pbcast.NAKACK2 xmit_interval="1000" xmit_table_num_rows="100" xmit_table_msgs_per_row="10000" xmit_table_max_compaction_time="30000" max_msg_batch_size="500" use_mcast_xmit="false" discard_delivered_msgs="true"/>
    <UNICAST2 max_bytes="20M" xmit_table_num_rows="100" xmit_table_msgs_per_row="2000" xmit_table_max_compaction_time="30000" max_msg_batch_size="100"/>
    <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000" max_bytes="8m"/>
    <pbcast.GMS print_local_addr="false" join_timeout="3000" view_bundling="true"/>
    <UFC max_credits="4M" min_threshold="0.1"/>
    <MFC max_credits="4M" min_threshold="0.2"/>
    <FRAG2 frag_size="60000"/>
    <RSVP />
</config>

Denvor Site XMLs:

Used the same set of xmls like CHARLOTTE and just interchanged CHARLOTTE with DENVOR.

Please let me know where the issue is.

Rgds
Srini

[rvansa]

Each site should form its own cluster, and there should be one more cluster formed by all master nodes from each site. You should check JGroups logs if this global cluster was formed (I guess it was not since the other site is unreachable). Check which processes are listening on localhost:7200-7207. I am not sure about default master setttings, but if all nodes try to join the global cluster, it's possible that TCPPING.port_range=3 is not enough - raise that to total number of nodes just to be on the safe side.