14 Replies Latest reply on Mar 16, 2016 9:34 AM by n.dobryukha

    Domain mode: replication between hosts.

    n.dobryukha

      Hey guys!

      I have a similar configuration: Host A is a master and Host B is a slave. Domain Controller is on Host A (domain.xml). Host B gets config remotely.

      domain.svg

      How can I configure cross site replication between master and slave hosts?

      sample.png

        • 1. Re: Domain mode: replication between hosts.
          n.dobryukha

          In the similar case Udit is trying to configure two different domain controllers (two domain.xml with different configs). Is it the only way?

          • 2. Re: Domain mode: replication between hosts.
            udit-mishra-5113a21a

            Hello n.dobryukha,

             

            Cross site replication is something different. In this case, you should not be using that. If you have only two nodes(master and slave, assuming both have a server), you can use a replicated or distributed cache and keep owners="2".

             

            Cross site replication is used ideally in situations where you want to backup data of one cluster to other cluster i.e. across clusters. Within the same cluster, you're not supposed to implement this. In your case, master and slave both are the parts of the same cluster.

            • 3. Re: Domain mode: replication between hosts.
              n.dobryukha

              Hi udit-mishra-5113a21a,

              Could you explain what you mean saying

              If you have only two nodes(master and slave, assuming both have a server)

              ?

              When I launch domain.bat with default domain.xml what do I get? Cluster "master" with two nodes "server-one" & "server-two" or one node "master"? And what do I get when I launch domain.bat on the other machine? Second cluster "slave" winth two other nodes "server-one" and "server-two" or second node "slave"?

              • 4. Re: Domain mode: replication between hosts.
                nadirx

                Nikita,

                looking at your diagram it looks like you've successfully created a 4-node cluster. Looking at the picture, it seems like libraryCache is using distribution with two owners. Adding an entry has therefore created two copies of it: one on master/server-one and one on master/server-two, which is correct. What is not healthy however is the fact that both entries live on the same physical node: lose that node, you lose both copies.

                There are two solutions:

                1. run only one server per physical box, which is probably what you would do in production, unless you need to have multiple servers per large physical node
                2. enable topology hinting, which is a way of telling Infinispan to place backup copies on different physical servers, racks, sites. Look at http://infinispan.org/docs/8.2.x/user_guide/user_guide.html#ServerHinting for information on what this means. In the server jgroups configuration you will need to use the following stanza (you can skip site and rack if you want):
                <transport type="UDP" socket-binding="jgroups-udp"
                     site="${jboss.jgroups.topology.site:s1}"
                     rack="${jboss.jgroups.topology.rack:r1}"
                     machine="${jboss.jgroups.topology.machine:m1}"/>
                
                • 5. Re: Domain mode: replication between hosts.
                  n.dobryukha

                  Tristan,

                  I tried to "run only one server per physical box"

                  nodes.PNG

                  But it wasn't helpful
                  cache.PNG

                  • 6. Re: Domain mode: replication between hosts.
                    nadirx

                    Can we have logs please ? And configuration.

                    • 7. Re: Domain mode: replication between hosts.
                      n.dobryukha

                      Sure

                      This config contains only one domain.xml (on master host). Backup options were temporary disabled or it is needed?

                      • 8. Re: Domain mode: replication between hosts.
                        n.dobryukha

                        Tristan,

                        Could I provide you any additional information?

                        • 9. Re: Domain mode: replication between hosts.
                          nadirx

                          Hi Nikita,

                          are you sure multicast is enabled and available in your network ?

                           

                          2016-03-15 15:08:01,158 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel clustered

                          2016-03-15 15:08:01,171 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel clustered: [master:server-one|0] (1) [master:server-one]

                          2016-03-15 15:08:01,176 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel clustered local address is master:server-one, physical addresses are [10.11.13.220:55200]

                           

                          and

                           

                          2016-03-15 14:08:47,396 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000078: Starting JGroups channel clustered

                          2016-03-15 14:08:47,412 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000094: Received new cluster view for channel clustered: [slave:server-one|0] (1) [slave:server-one]

                          2016-03-15 14:08:47,490 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000079: Channel clustered local address is slave:server-one, physical addresses are [10.17.10.187:55200]

                           

                          You should see the cluster view with two nodes.

                           

                           

                          Tristan

                          • 10. Re: Domain mode: replication between hosts.
                            n.dobryukha

                            Tristan,

                            Thanks for quick answer!

                            Earlier I published a scrennshot

                            nodes.PNG

                            Is that what you mean saying

                            You should see the cluster view with two nodes.

                            ?

                            • 11. Re: Domain mode: replication between hosts.
                              nadirx

                              No, unfortunately the console is behaving wrongly here. Could you please use the CLI:

                              ./bin/ispn-cli.sh

                              > connect

                              > /host=master/server=server-one/subsystem=datagrid-infinispan/cache-container=cluster:read-attribute(name=members)

                              > /host=slave/server=server-one/subsystem=datagrid-infinispan/cache-container=cluster:read-attribute(name=members)

                              • 12. Re: Domain mode: replication between hosts.
                                n.dobryukha

                                Hi Tristan Tarrant,

                                Here is CLI log

                                You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.
                                [disconnected /] connect
                                [domain@eprusarw1154:9990 /] cd /host=master/server=server-one/subsystem=datagrid-infinispan/cache-container=clustered
                                [domain@eprusarw1154:9990 cache-container=clustered] read-attribute members
                                [master:server-one]
                                [domain@eprusarw1154:9990 cache-container=clustered] cd /host=slave/server=server-one/subsystem=datagrid-infinispan/cache-container=clustered
                                [domain@eprusarw1154:9990 cache-container=clustered] read-attribute members
                                [slave:server-one]
                                [domain@eprusarw1154:9990 cache-container=clustered]
                                
                                • 13. Re: Domain mode: replication between hosts.
                                  nadirx

                                  As I suspected: you have two separate "one node" clusters. Is multicast enabled on your network ? If not you should consider alternative PING implementations.

                                  • 14. Re: Domain mode: replication between hosts.
                                    n.dobryukha

                                    Is multicast enabled on your network ?

                                    Host 10.11.13.220

                                    > netsh interface ip show joins
                                    
                                    
                                    Interface 1: Loopback Pseudo-Interface 1
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    2  Yes   239.255.255.250
                                    
                                    
                                    Interface 11: Local Area Connection
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    0  No    224.0.0.1
                                    0                    1  Yes   224.0.0.252
                                    0                    1  Yes   234.99.54.15
                                    0                    1  Yes   239.255.255.250
                                    
                                    
                                    Interface 16: Local Area Connection 2
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    0  Yes   224.0.0.1
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    

                                     

                                    Host 10.17.10.187

                                    > netsh interface ip show joins
                                    
                                    
                                    Interface 1: Loopback Pseudo-Interface 1
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    1  Yes   239.255.255.250
                                    
                                    
                                    Interface 12: Local Area Connection
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    0  No    224.0.0.1
                                    0                    1  Yes   224.0.0.251
                                    0                    1  No    224.0.0.252
                                    0                    1  Yes   234.99.54.15
                                    0                    1  No    239.255.255.250
                                    
                                    
                                    Interface 14: VirtualBox Host-Only Network
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    0  Yes   224.0.0.1
                                    0                    1  Yes   224.0.0.252
                                    0                    1  Yes   239.77.124.213
                                    0                    1  Yes   239.255.255.250
                                    
                                    
                                    Interface 17: Local Area Connection 2
                                    
                                    
                                    Scope       References  Last  Address
                                    ----------  ----------  ----  ---------------------------------
                                    0                    0  Yes   224.0.0.1
                                    
                                    

                                     

                                    If not you should consider alternative PING implementations.

                                    I've changed my domain.xml

                                    ...
                                    <stack name="udp">
                                      <transport type="UDP" socket-binding="jgroups-udp"/>
                                      <!--<protocol type="PING"/>-->
                                      <protocol type="TCPPING">
                                           <property name="initial_hosts">
                                                10.11.13.220[7600],10.17.10.187[7600]
                                           </property>
                                      </protocol>
                                      <protocol type="MERGE3"/>
                                      <protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
                                      <protocol type="FD_ALL"/>
                                      <protocol type="VERIFY_SUSPECT"/>
                                      <protocol type="pbcast.NAKACK2"/>
                                      <protocol type="UNICAST3"/>
                                      <protocol type="pbcast.STABLE"/>
                                      <protocol type="pbcast.GMS"/>
                                      <protocol type="UFC"/>
                                      <protocol type="MFC"/>
                                      <protocol type="FRAG2"/>
                                    </stack>
                                    ...
                                    
                                    
                                    
                                    

                                    But it didn't help.