Wildfly cluster in docker?

[cmosher01]

Has anyone managed to get a cluster working in docker yet?

I'm trying to use the standalone-ha.xml on two nodes from the jboss/wildfly docker image, but it seems like the nodes can't see each other, and they never form a cluster. I see this in the server.log on each node:

2014-10-23 20:27:31,677 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web1/web|0] (1) [web1/web]

and

2014-10-23 20:27:30,617 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web2/web|0] (1) [web2/web]

Instead, if I start the docker containers with --net="host" it works as expected:

2014-10-23 19:26:12,183 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web1/web|0] (1) [web1/web]

2014-10-23 19:26:13,461 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,shared=udp) ISPN000094: Received new cluster view: [web1/web|1] (2) [web1/web, web2/web]

and

2014-10-23 19:26:13,501 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web1/web|1] (2) [web1/web, web2/web]

[jugglingcats]

Wouldn't jgroups start on it's default port, eg. 7800 on both nodes, as Docker will give this port as available and translate it to another port externally? Therefore the port reported by jgroups to other nodes in the cluster would be different to the port they actually need to use to communicate with said node.

Am interested to hear if/how you get this working as we want to do something similar ;-)

One obvious approach would be to tell Docker to bind a specific port (same port inside/outside), and modify the jgroups config for each node to use this port, ie. explicitly run container A on 7800, container B on 7801, etc. Eg. docker run -p 7800:7800 ...

I don't think this is the "Docker way" though.

[cmosher01]

Actually I wanted to use UDP with multicast, not TCP.

For UDP, I tested that multicast does indeed work across the two containers:

[cmosher@dtcmosher ~]$ sudo nsenter -p -i -n -u -m -t 8032

[root@49c9c0aacd7a /]# cd /opt/wildfly/modules/system/layers/base/org/jgroups/main

[root@49c9c0aacd7a main]# java -cp jgroups-3.4.3.Final.jar org.jgroups.tests.McastSenderTest

Socket #1=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/fe80:0:0:0:42:acff:fe11:20%65

Socket #2=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/172.17.0.32

Socket #3=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/0:0:0:0:0:0:0:1%1

Socket #4=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/127.0.0.1

> abc

> def

> ghi

and

[cmosher@dtcmosher ~]$ sudo nsenter -p -i -n -u -m -t 8158

[root@927e457a49ea /]# cd /opt/wildfly/modules/system/layers/base/org/jgroups/main/

[root@927e457a49ea main]# java -cp jgroups-3.4.3.Final.jar org.jgroups.tests.McastReceiverTest

Socket=0.0.0.0/0.0.0.0:5555, bind interface=/fe80:0:0:0:42:acff:fe11:21%67

Socket=0.0.0.0/0.0.0.0:5555, bind interface=/172.17.0.33

Socket=0.0.0.0/0.0.0.0:5555, bind interface=/0:0:0:0:0:0:0:1%1

Socket=0.0.0.0/0.0.0.0:5555, bind interface=/127.0.0.1

abc [sender=172.17.0.32:5555]

abc [sender=172.17.0.32:5555]

abc [sender=172.17.0.32:5555]

abc [sender=172.17.0.32:5555]

abc [sender=172.17.0.32:5555]

abc [sender=172.17.0.32:5555]

def [sender=172.17.0.32:5555]

def [sender=172.17.0.32:5555]

def [sender=172.17.0.32:5555]

def [sender=172.17.0.32:5555]

def [sender=172.17.0.32:5555]

def [sender=172.17.0.32:5555]

ghi [sender=172.17.0.32:5555]

ghi [sender=172.17.0.32:5555]

ghi [sender=172.17.0.32:5555]

ghi [sender=172.17.0.32:5555]

ghi [sender=172.17.0.32:5555]

ghi [sender=172.17.0.32:5555]

This leads me to believe it's something within JGroups causing the problem, not something with the docker container setup.

[rhusar]

This blog post should help answer most of the questions:

https://goldmann.pl/blog/2013/10/07/wildfly-cluster-using-docker-on-fedora/

[cmosher01]

I did get clustering working under docker. For those interested, the trick was to specify a specific IP address to standalone.sh, not 0.0.0.0, on the -b option:

$JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=`hostname -I`

[frank_xaero]

Helped me a lot, thank you!

I use CMD in Dockerfile like:

    sh /opt/jboss-eap-6.1/bin/standalone.sh --debug 8787 -c standalone-ha.xml -b $(hostname -i)

[belaban]

I know we're om 2017 now, but here's my recent findings with --net=bridge and --net=host trying to run JGroups in Docker:

GitHub - belaban/jgroups-docker: Dockerfile for a container containing JGroups and a couple of demos