2 Replies Latest reply on Dec 7, 2016 3:24 AM by Manuel Galgana

    Wildfly 8.2.1 Distributed cache on multiple nodes in different machines

    Manuel Galgana Newbie

      How can I make a distributed cache to work when the nodes are on a different machines?

       

      I'm trying make the cluster-test from "wildfly cookbook" to work.

      I have setup wildfly 8.2.1 on 2 Virtual Machines using standalone-ha.xml configuration.

       

      Below are my configuration changes from the default

       

      Node1

      standalone.conf

      JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone-ha.xml"

      JAVA_OPTS="$JAVA_OPTS -Djboss.node.name=node1 -Djboss.default.multicast.address=230.0.0.10"

       

      standalone-ha.xml

      <interfaces>

              <interface name="management">

                  <inet-address value="${jboss.bind.address.management:node1_IP}"/>

              </interface>

              <interface name="public">

                  <inet-address value="${jboss.bind.address:node1_IP}"/>

              </interface>

              <interface name="unsecure">

                  <inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>

              </interface>

          </interfaces>

       

      Node2

      standalone.conf

      JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone-ha.xml"

      JAVA_OPTS="$JAVA_OPTS -Djboss.node.name=node2 -Djboss.default.multicast.address=230.0.0.10"

       

      standalone-ha.xml

      <interfaces>

              <interface name="management">

                  <inet-address value="${jboss.bind.address.management:node2_IP}"/>

              </interface>

              <interface name="public">

                  <inet-address value="${jboss.bind.address:node2_IP}"/>

              </interface>

              <interface name="unsecure">

                  <inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>

              </interface>

          </interfaces>

       

       

      Below are the content of the cluster-test application from wildfly cookbook

       

      web.xml

      <!DOCTYPE web-app PUBLIC

              "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"

              "http://java.sun.com/dtd/web-app_2_3.dtd" >

       

      <web-app>

          <display-name>cluster-test</display-name>

       

          <distributable/>

       

      </web-app>

       

      index.jsp

      <html>

      <body>

      <h2>WildFly::Cookbook::Cluster Test</h2>

       

      <%

          Object sessAttr = session.getAttribute("visitors");

          int visitors = sessAttr == null ? 0 : Integer.valueOf(sessAttr+"");

          System.out.println("********************************+");

          System.out.println("Visitor(s): " + visitors);

          System.out.println("********************************+");

      %>

       

      <h2>Visitor(s): <%=visitors%></h2>

       

      <%

       

       

      Using the same configuration on a single machine with just port offsetting on the 2nd node everything works well.

       

      Is there anything needed to configure for the distributed cache to work on 2 different machines?

        • 1. Re: Wildfly 8.2.1 Distributed cache on multiple nodes in different machines
          Sebastian Łaskawiec Apprentice

          My lucky guess is that JGroups have binded to the loopback interface - 127.0.0.1. You might check this by running netstat -ntlp.

           

          The easiest fix is to start up Wildfly with your public IP address: ./standalone -c standalone-ha.xml -b /* your IP, e.g. 192.168.1.15 */

          • 2. Re: Wildfly 8.2.1 Distributed cache on multiple nodes in different machines
            Manuel Galgana Newbie

            Hi Guys,

             

                 First, Thank you

             

                 I figured out that the configuration above is actually working. I just have a wrong perception of how the session in the sample code of cluster-test is working at the same time I thought I have the same session when I access that sample application on different browser in different machine in a wildfly container.

             

                 Below are the configuration changes we did to make Distributed cache on multiple nodes in different machines to work (Using our own application with authentication rather than using the cluster test because cluster test can only be use to test caching within the same machine):

             

            Standalone-ha.xml

            Created new cache container

            Note: This cache container is the one we are using in the application (I cannot show it because I'm not familiar how my Teammate created it.)

             

            <cache-container name="jprobe" default-cache="passivation" jndi-name="java:jboss/infinispan/container/jprobe" start="EAGER" module="org.wildfly.clustering.web.infinispan">

            <transport stack="udp" lock-timeout="60000"/>

            <distributed-cache name="passivation" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">

            <file-store passivation="true" purge="false"/>

            </distributed-cache>

            <distributed-cache name="persistent" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">

            <file-store passivation="true" purge="false"/>

            </distributed-cache>

            <distributed-cache name="SessionKeys" mode="ASYNC"/>

            <distributed-cache name="MaxUsers" mode="ASYNC"/>

            </cache-container>

             

            add node-identifier and instance-id (for load balancer)

            <subsystem xmlns="urn:jboss:domain:transactions:2.0">

            <core-environment node-identifier="${jboss.node.name}">

            <process-id>

            <uuid/>

            </process-id>

            </core-environment>

            <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>

            </subsystem>

             

            <subsystem xmlns="urn:jboss:domain:undertow:1.2" instance-id="${jboss.node.name}">

            <buffer-cache name="default"/>

               <server name="default-server">

            <ajp-listener name="ajp" socket-binding="ajp"/>

            <http-listener name="default" socket-binding="http"/>

            <host name="default-host" alias="localhost">

            <location name="/" handler="welcome-content"/>

            <filter-ref name="server-header"/>

            <filter-ref name="x-powered-by-header"/>

            </host>

            </server>

            <servlet-container name="default">

            <jsp-config/>

            <websockets/>

            </servlet-container>

            <handlers>

            <file name="welcome-content" path="${jboss.home.dir}/welcome-content"/>

            </handlers>

            <filters>

            <response-header name="server-header" header-name="Server" header-value="WildFly/8"/>

            <response-header name="x-powered-by-header" header-name="X-Powered-By" header-value="Undertow/1"/>

            </filters>

              </subsystem>

             

            Define Machines IP Address and declare the public inet address as multicast and up

            <interfaces>

               <interface name="management">

            <inet-address value="${jboss.bind.address.management:IP Adress}"/>

               </interface>

               <interface name="public">

            <inet-address value="${jboss.bind.address:IP Adress}"/>

            <multicast/>

               <up/>

               </interface>

               <interface name="unsecure">

            <inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>

               </interface>

               </interfaces>

             

            standalone.conf

            declare node name and multicast address in JAVA_OPTS

            Note: (“jboss.node.name” should be different per server. “jboss.default.multicast.address” should be the same for cluster group, This should be different per environment)

             

            JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone-ha.xml"

            JAVA_OPTS="$JAVA_OPTS -Djboss.node.name=node2 -Djboss.default.multicast.address=230.0.0.5"

             

            Now try to login in your application using the same account (same username and password) on different nodes in different machines.