2 Replies Latest reply on Dec 7, 2016 3:24 AM by pilipino22

Wildfly 8.2.1 Distributed cache on multiple nodes in different machines

pilipino22 Nov 29, 2016 2:13 AM

How can I make a distributed cache to work when the nodes are on a different machines?

I'm trying make the cluster-test from "wildfly cookbook" to work.

I have setup wildfly 8.2.1 on 2 Virtual Machines using standalone-ha.xml configuration.

Below are my configuration changes from the default

Node1

standalone.conf

JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone-ha.xml"

JAVA_OPTS="$JAVA_OPTS -Djboss.node.name=node1 -Djboss.default.multicast.address=230.0.0.10"

standalone-ha.xml

<inet-address value="${jboss.bind.address.management:node1_IP}"/>

</interface>

<inet-address value="${jboss.bind.address:node1_IP}"/>

</interface>

<inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>

</interface>

</interfaces>

Node2

standalone.conf

JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone-ha.xml"

JAVA_OPTS="$JAVA_OPTS -Djboss.node.name=node2 -Djboss.default.multicast.address=230.0.0.10"

standalone-ha.xml

<inet-address value="${jboss.bind.address.management:node2_IP}"/>

</interface>

<inet-address value="${jboss.bind.address:node2_IP}"/>

</interface>

<inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>

</interface>

</interfaces>

Below are the content of the cluster-test application from wildfly cookbook

web.xml

<!DOCTYPE web-app PUBLIC

"-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"

"http://java.sun.com/dtd/web-app_2_3.dtd" >

<web-app>

<display-name>cluster-test</display-name>

</web-app>

index.jsp

<html>

<body>

<h2>WildFly::Cookbook::Cluster Test</h2>

Object sessAttr = session.getAttribute("visitors");

int visitors = sessAttr == null ? 0 : Integer.valueOf(sessAttr+"");

System.out.println("********************************+");

System.out.println("Visitor(s): " + visitors);

System.out.println("********************************+");

<h2>Visitor(s): <%=visitors%></h2>

Using the same configuration on a single machine with just port offsetting on the 2nd node everything works well.

Is there anything needed to configure for the distributed cache to work on 2 different machines?

1. Re: Wildfly 8.2.1 Distributed cache on multiple nodes in different machines

sebastian.laskawiec Dec 6, 2016 10:31 PM (in response to pilipino22)

My lucky guess is that JGroups have binded to the loopback interface - 127.0.0.1. You might check this by running netstat -ntlp.

The easiest fix is to start up Wildfly with your public IP address: ./standalone -c standalone-ha.xml -b /* your IP, e.g. 192.168.1.15 */
Actions
2. Re: Wildfly 8.2.1 Distributed cache on multiple nodes in different machines

pilipino22 Dec 7, 2016 3:24 AM (in response to sebastian.laskawiec)

Hi Guys,

     First, Thank you Sebastian Łaskawiec for taking time on helping me.

     I figured out that the configuration above is actually working. I just have a wrong perception of how the session in the sample code of cluster-test is working at the same time I thought I have the same session when I access that sample application on different browser in different machine in a wildfly container.

     Below are the configuration changes we did to make Distributed cache on multiple nodes in different machines to work (Using our own application with authentication rather than using the cluster test because cluster test can only be use to test caching within the same machine):

Standalone-ha.xml
Created new cache container
Note: This cache container is the one we are using in the application (I cannot show it because I'm not familiar how my Teammate created it.)

<cache-container name="jprobe" default-cache="passivation" jndi-name="java:jboss/infinispan/container/jprobe" start="EAGER" module="org.wildfly.clustering.web.infinispan">
<transport stack="udp" lock-timeout="60000"/>
<distributed-cache name="passivation" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">
<file-store passivation="true" purge="false"/>
</distributed-cache>
<distributed-cache name="persistent" batching="true" mode="ASYNC" owners="2" l1-lifespan="0">
<file-store passivation="true" purge="false"/>
</distributed-cache>
<distributed-cache name="SessionKeys" mode="ASYNC"/>
<distributed-cache name="MaxUsers" mode="ASYNC"/>
</cache-container>

add node-identifier and instance-id (for load balancer)
<subsystem xmlns="urn:jboss:domain:transactions:2.0">
<core-environment node-identifier="${jboss.node.name}">
<process-id>
<uuid/>
</process-id>
</core-environment>
<recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>
</subsystem>

<subsystem xmlns="urn:jboss:domain:undertow:1.2" instance-id="${jboss.node.name}">
<buffer-cache name="default"/>
   <server name="default-server">
<ajp-listener name="ajp" socket-binding="ajp"/>
<http-listener name="default" socket-binding="http"/>
<host name="default-host" alias="localhost">
<location name="/" handler="welcome-content"/>
<filter-ref name="server-header"/>
<filter-ref name="x-powered-by-header"/>
</host>
</server>
<servlet-container name="default">
<jsp-config/>
<websockets/>
</servlet-container>
<handlers>
<file name="welcome-content" path="${jboss.home.dir}/welcome-content"/>
</handlers>
<filters>
<response-header name="server-header" header-name="Server" header-value="WildFly/8"/>
<response-header name="x-powered-by-header" header-name="X-Powered-By" header-value="Undertow/1"/>
</filters>
</subsystem>

Define Machines IP Address and declare the public inet address as multicast and up
<interfaces>
   <interface name="management">
<inet-address value="${jboss.bind.address.management:IP Adress}"/>
   </interface>
   <interface name="public">
<inet-address value="${jboss.bind.address:IP Adress}"/>
<multicast/>
   <up/>
   </interface>
   <interface name="unsecure">
<inet-address value="${jboss.bind.address.unsecure:127.0.0.1}"/>
   </interface>
   </interfaces>

standalone.conf
declare node name and multicast address in JAVA_OPTS
Note: (“jboss.node.name” should be different per server. “jboss.default.multicast.address” should be the same for cluster group, This should be different per environment)

JAVA_OPTS="$JAVA_OPTS -Djboss.server.default.config=standalone-ha.xml"
JAVA_OPTS="$JAVA_OPTS -Djboss.node.name=node2 -Djboss.default.multicast.address=230.0.0.5"

Now try to login in your application using the same account (same username and password) on different nodes in different machines.
Actions

Go to original post