6 Replies Latest reply on Feb 16, 2017 11:43 AM by Bela Ban

    Wildfly cluster in docker?

    Christopher Mosher Newbie

      Has anyone managed to get a cluster working in docker yet?

       

      I'm trying to use the standalone-ha.xml on two nodes from the jboss/wildfly docker image, but it seems like the nodes can't see each other, and they never form a cluster. I see this in the server.log on each node:

       

      2014-10-23 20:27:31,677 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web1/web|0] (1) [web1/web]

      and

      2014-10-23 20:27:30,617 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web2/web|0] (1) [web2/web]

       

      Instead, if I start the docker containers with --net="host" it works as expected:

      2014-10-23 19:26:12,183 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web1/web|0] (1) [web1/web]

      2014-10-23 19:26:13,461 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,shared=udp) ISPN000094: Received new cluster view: [web1/web|1] (2) [web1/web, web2/web]

      and

      2014-10-23 19:26:13,501 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 54) ISPN000094: Received new cluster view: [web1/web|1] (2) [web1/web, web2/web]

        • 1. Re: Wildfly cluster in docker?
          Alfie Kirkpatrick Newbie

          Wouldn't jgroups start on it's default port, eg. 7800 on both nodes, as Docker will give this port as available and translate it to another port externally? Therefore the port reported by jgroups to other nodes in the cluster would be different to the port they actually need to use to communicate with said node.

           

          Am interested to hear if/how you get this working as we want to do something similar ;-)

           

          One obvious approach would be to tell Docker to bind a specific port (same port inside/outside), and modify the jgroups config for each node to use this port, ie. explicitly run container A on 7800, container B on 7801, etc. Eg. docker run -p 7800:7800 ...

           

          I don't think this is the "Docker way" though.

          • 2. Re: Re: Wildfly cluster in docker?
            Christopher Mosher Newbie

            Actually I wanted to use UDP with multicast, not TCP.

             

            For UDP, I tested that multicast does indeed work across the two containers:

             

            [cmosher@dtcmosher ~]$ sudo nsenter -p -i -n -u -m -t 8032

            [root@49c9c0aacd7a /]# cd /opt/wildfly/modules/system/layers/base/org/jgroups/main

            [root@49c9c0aacd7a main]# java -cp jgroups-3.4.3.Final.jar org.jgroups.tests.McastSenderTest

            Socket #1=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/fe80:0:0:0:42:acff:fe11:20%65

            Socket #2=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/172.17.0.32

            Socket #3=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/0:0:0:0:0:0:0:1%1

            Socket #4=0.0.0.0/0.0.0.0:5555, ttl=32, bind interface=/127.0.0.1

            > abc

            > def

            > ghi

             

            and

             

            [cmosher@dtcmosher ~]$ sudo nsenter -p -i -n -u -m -t 8158

            [root@927e457a49ea /]# cd /opt/wildfly/modules/system/layers/base/org/jgroups/main/

            [root@927e457a49ea main]# java -cp jgroups-3.4.3.Final.jar org.jgroups.tests.McastReceiverTest

            Socket=0.0.0.0/0.0.0.0:5555, bind interface=/fe80:0:0:0:42:acff:fe11:21%67

            Socket=0.0.0.0/0.0.0.0:5555, bind interface=/172.17.0.33

            Socket=0.0.0.0/0.0.0.0:5555, bind interface=/0:0:0:0:0:0:0:1%1

            Socket=0.0.0.0/0.0.0.0:5555, bind interface=/127.0.0.1

            abc [sender=172.17.0.32:5555]

            abc [sender=172.17.0.32:5555]

            abc [sender=172.17.0.32:5555]

            abc [sender=172.17.0.32:5555]

            abc [sender=172.17.0.32:5555]

            abc [sender=172.17.0.32:5555]

            def [sender=172.17.0.32:5555]

            def [sender=172.17.0.32:5555]

            def [sender=172.17.0.32:5555]

            def [sender=172.17.0.32:5555]

            def [sender=172.17.0.32:5555]

            def [sender=172.17.0.32:5555]

            ghi [sender=172.17.0.32:5555]

            ghi [sender=172.17.0.32:5555]

            ghi [sender=172.17.0.32:5555]

            ghi [sender=172.17.0.32:5555]

            ghi [sender=172.17.0.32:5555]

            ghi [sender=172.17.0.32:5555]

             

            This leads me to believe it's something within JGroups causing the problem, not something with the docker container setup.

            • 3. Re: Wildfly cluster in docker?
              Radoslav Husar Master

              This blog post should help answer most of the questions:

               

              https://goldmann.pl/blog/2013/10/07/wildfly-cluster-using-docker-on-fedora/

              1 of 1 people found this helpful
              • 4. Re: Wildfly cluster in docker?
                Christopher Mosher Newbie

                I did get clustering working under docker. For those interested, the trick was to specify a specific IP address to standalone.sh, not 0.0.0.0, on the -b option:

                 

                $JBOSS_HOME/bin/standalone.sh -c standalone-ha.xml -b=`hostname -I`

                • 5. Re: Wildfly cluster in docker?
                  Frank Xaero Newbie

                  Helped me a lot, thank you!

                  I use CMD in Dockerfile like:

                      sh /opt/jboss-eap-6.1/bin/standalone.sh --debug 8787 -c standalone-ha.xml -b $(hostname -i)

                  • 6. Re: Wildfly cluster in docker?
                    Bela Ban Master

                    I know we're om 2017 now, but here's my recent findings with --net=bridge and --net=host trying to run JGroups in Docker:

                    GitHub - belaban/jgroups-docker: Dockerfile for a container containing JGroups and a couple of demos