14 Replies Latest reply on Sep 5, 2014 2:17 AM by Alexis Hassler

    Ugly behavior with default-interface and mod_cluster

    Alexis Hassler Apprentice

      Hi,

       

      I’m trying to setting a cluster of WildFly 8.1.0.Final, with mod_cluster as a front end, and all of this in docker containers. I’ve customized the network configuration of my WildFly, in the standalone-ha.xml. I have 2 interfaces : all and public :

          <interfaces>
               <interface name="public">
                 <nic name="eth0"/>
               </interface>
               <interface name="all">
                 <any-address/>
               </interface>
           </interfaces>
      

      I have explicitly declared the interface of all my socket-bindings :

              <socket-binding name="management-http" port="${jboss.management.http.port:9990}" interface="public" />
               <socket-binding name="management-https" port="${jboss.management.https.port:9993}" interface="public" />
               <socket-binding name="ajp" port="${jboss.ajp.port:8009}" interface="public"/>
               <socket-binding name="http" port="${jboss.http.port:8080}" interface="public"/>
               <socket-binding name="https" port="${jboss.https.port:8443}" interface="public"/>
               <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700" interface="public"/>
               <socket-binding name="jgroups-tcp" port="7600" interface="public"/>
               <socket-binding name="jgroups-tcp-fd" port="57600" interface="public"/>
               <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688" interface="public"/>
               <socket-binding name="jgroups-udp-fd" port="54200" interface="public"/>
               <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364" interface="public"/>
               <socket-binding name="txn-recovery-environment" port="4712" interface="public"/>
              <socket-binding name="txn-status-manager" port="4713" interface="public"/>
      

       

      Now, when I declare the default-interface to all, mod_cluster can discover my instance.

      <socket-binding-group name="standard-sockets" default-interface="all" port-offset="${jboss.socket.binding.port-offset:0} »>
      

      But if I declare the default interface to public, mod_cluster cannot discover my instance.

      <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
      

       

      I thought that the default-interface attribute was the default value for the interface attribute of all the socket-bindings. So why does it has an impact whereas I've declare the interface on all the socket-bindings ?

       

      Alexis

        • 1. Re: Ugly behavior with default-interface and mod_cluster
          jaikiran pai Master

          So it's using a wrong interface for the socket binding? Is this just for mod_cluster service or any service relying on the socket binding interface?

          • 2. Re: Ugly behavior with default-interface and mod_cluster
            Alexis Hassler Apprentice

            I really don't know where it come from. I don't see any difference with "netstat -pan".

             

            The problem is only with mod_cluster.

            • 3. Re: Ugly behavior with default-interface and mod_cluster
              Radoslav Husar Master

              Hi Alexis,

               

              Just trying to make sense of this, so the only difference is whether you explicitly or implicitly define interface for the modcluster socket binding? At least it looks like this is trivial to workaround.

               

              Can you try with lastest upstream master as well?

              • 4. Re: Ugly behavior with default-interface and mod_cluster
                Alexis Hassler Apprentice

                No, in both cases the interface of modcluster is explicit. The only difference is on the default-interface.

                 

                I'll give a try to the upstream and come back soon.

                • 5. Re: Ugly behavior with default-interface and mod_cluster
                  Radoslav Husar Master

                  The code actually does get the default interfce (from the SocketBindingManager) if none is specified for that particular binding:

                   

                  The code checks if the interface is configured expliticly, and only then uses the default one.

                      public InetAddress getAddress() {

                          return networkInterface != null ? networkInterface.getAddress() : socketBindings.getDefaultInterfaceAddress();

                      }

                  • 6. Re: Ugly behavior with default-interface and mod_cluster
                    Alexis Hassler Apprentice

                    This piece of code makes sense. But it does not fit to my test. The default interface is probably used somewhere else.

                     

                    Same behaviour with wildfly-9.0.0.Alpha1-SNAPSHOT from master.

                     

                    I can share Docker images if it helps.

                    • 7. Re: Ugly behavior with default-interface and mod_cluster
                      Radoslav Husar Master

                      So I tried to reproduce the problem, but all works as expected.

                       

                      I have set modcluster apache module to advertise on my tunnel interface (tun) by configuring AdvertiseBindAddress. I have configured another interface local (lo). Multicasts from one don't get forwarded to another.

                       

                      Then tried several combinations of setting interface in WildFly 8.1:

                       

                      • default-interface=lo + no mod_cluster socket binding interface => no discovery
                      • default-interface=lo + mod_cluster socket binding interface=tun => discovery OK
                      • default-interface=tun + no mod_cluster socket binding interface => discovery OK
                      • default-interface=tun + mod_cluster socket binding interface=tun => discovery OK

                       

                      All works as expected. Does this test reenact your test?

                      • 8. Re: Ugly behavior with default-interface and mod_cluster
                        Alexis Hassler Apprentice

                        I do not have a tun interface but I'm using eth0. I have lo too (loopback, 127.0.0.1). This is based on the default network configuration of docker.

                        • default-interface=lo + no mod_cluster socket binding interface => no discovery
                        • default-interface=lo + mod_cluster socket binding interface=eth0 => no discovery
                        • default-interface=eth0 + no mod_cluster socket binding interface => no discovery
                        • default-interface=eth0 + mod_cluster socket binding interface=eth0 => no discovery

                         

                        The only way to have discovery is to have eth0+lo as default interface.

                        • default-interface=eth0+lo + no mod_cluster socket binding interface => discovery OK
                        • default-interface=eth0+lo + mod_cluster socket binding interface=eth0 => discovery OK
                        • 9. Re: Ugly behavior with default-interface and mod_cluster
                          Radoslav Husar Master

                          Can you elaborate what you mean by

                          default-interface=eth0+lo

                          ?

                          • 10. Re: Re: Ugly behavior with default-interface and mod_cluster
                            Alexis Hassler Apprentice

                            eth0 + lo means

                             

                                    <interface name="all">
                                      <nic name="eth0"/>
                                      <nic name="lo"/>
                                    </interface>
                            

                            or

                             

                                    <interface name="all">
                                      <any-address/>
                                    </interface>
                            
                            • 11. Re: Re: Ugly behavior with default-interface and mod_cluster
                              Radoslav Husar Master

                              eth0 + lo means

                               

                               

                              1.         <interface name="all"> 
                              2.           <nic name="eth0"/> 
                              3.           <nic name="lo"/> 
                              4.         </interface> 

                               

                              I am afraid this is same as specifying one nic being "lo". I need to double check this, specically talking about org.jboss.as.server.parsing.CommonXml#parseSimpleInterfaceCriterion, but unless you are using nested <any> or <not> the parsing logic seems to just look for the configuration elements and the resulting model will only pick up the last one which overrides any previous values. This looks like a usability bug that it doesn't tell you that this configuration is not doing what you might have expected.

                               

                              Try to look at the resulting model.

                               

                              [standalone@localhost:9990 /] /interface=all:read-resource

                              {

                                  "outcome" => "success",

                                  "result" => {

                                      "any" => undefined,

                                      "any-address" => undefined,

                                      "any-ipv4-address" => undefined,

                                      "any-ipv6-address" => undefined,

                                      "inet-address" => expression "${jboss.bind.address:127.0.0.1}",

                                      "link-local-address" => undefined,

                                      "loopback" => undefined,

                                      "loopback-address" => undefined,

                                      "multicast" => undefined,

                                      "name" => "all",

                                      "nic" => "lo",

                                      "nic-match" => undefined,

                                      "not" => undefined,

                                      "point-to-point" => undefined,

                                      "public-address" => undefined,

                                      "site-local-address" => undefined,

                                      "subnet-match" => undefined,

                                      "up" => undefined,

                                      "virtual" => undefined

                                  }

                              }

                               

                              Nevertheless, this shouldn't be affecting the problem we are talking about here with mod_cluster not picking the right interface.

                              • 12. Re: Re: Ugly behavior with default-interface and mod_cluster
                                Alexis Hassler Apprentice

                                You're totally right. lo or eth0 + lo are the same. Have to replay my tests...

                                 

                                Updated result :

                                • default-interface=lo + no mod_cluster socket binding interface => no discovery
                                • default-interface=lo + mod_cluster socket binding interface=eth0 => discovery OK
                                • default-interface=eth0 + no mod_cluster socket binding interface => no discovery
                                • default-interface=eth0 + mod_cluster socket binding interface=eth0 => no discovery
                                • 13. Re: Re: Ugly behavior with default-interface and mod_cluster
                                  Radoslav Husar Master

                                  Can you post your XML with the socket bindings? It looks like it's the problem when you have multiple <nic>s in your <interface> and thus only the last one is effectively used.

                                  • 14. Re: Re: Re: Ugly behavior with default-interface and mod_cluster
                                    Alexis Hassler Apprentice

                                    My initial config (see first post) didn't have multiple nic. Neither my latest :

                                     

                                        <interfaces>
                                            <interface name="public">
                                                <nic name="eth0"/>
                                            </interface>
                                            <interface name="local">
                                                <loopback/>
                                            </interface>
                                        </interfaces>
                                    
                                    
                                        <socket-binding-group name="standard-sockets" default-interface="local" port-offset="${jboss.socket.binding.port-offset:0}">
                                            <socket-binding name="management-http" interface="public" port="${jboss.management.http.port:9990}"/>
                                            <socket-binding name="management-https" interface="local" port="${jboss.management.https.port:9993}"/>
                                            <socket-binding name="ajp" interface="public" port="${jboss.ajp.port:8009}"/>
                                            <socket-binding name="http" interface="local" port="${jboss.http.port:8080}"/>
                                            <socket-binding name="https" interface="local" port="${jboss.https.port:8443}"/>
                                            <socket-binding name="jgroups-mping" interface="local" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>
                                            <socket-binding name="jgroups-tcp" interface="local" port="7600"/>
                                            <socket-binding name="jgroups-tcp-fd" interface="local" port="57600"/>
                                            <socket-binding name="jgroups-udp" interface="public" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
                                            <socket-binding name="jgroups-udp-fd" interface="public" port="54200"/>
                                            <socket-binding name="modcluster" interface="public" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>
                                            <socket-binding name="txn-recovery-environment" interface="local" port="4712"/>
                                            <socket-binding name="txn-status-manager" interface="local" port="4713"/>
                                            <outbound-socket-binding name="mail-smtp">
                                                <remote-destination host="localhost" port="25"/>
                                            </outbound-socket-binding>
                                        </socket-binding-group>