13 Replies Latest reply on Oct 10, 2014 7:16 AM by Cristian Tetic

    mod_cluster manager does not discover JBoss nodes

    Max Pimm Newbie

      I have followed the instructions at https://docs.jboss.org/author/display/AS71/AS7+Cluster+Howto but am stuck.

       

      I wonder if someone could give me a push in the right direction.

       

      I have a JBoss domain instance and httpd with mod_cluster installed on the same machine (Fedora 17 with IP 192.168.1.103)

       

      I have deployed the cluster-demo app in the instructions in the other server group and the server starts with no errors and the message: Listening to proxy advertisements on 224.0.1.105:23364

       

      Httpd starts without errors and I can access http://192.168.1.103:10001/mod_cluster-manager but it shows an empty list of nodes.

       

      httpd.conf loads the necessary modules and declares the following virtual host

       

      {code:XML}

      <VirtualHost 192.168.1.103:10001>

       

        <Directory />

          Order deny,allow

          Deny from all

          Allow from 192.168.1.

        </Directory>

       

        <Location /mod_cluster-manager>

         SetHandler mod_cluster-manager

         Order deny,allow

         Deny from all

         Allow from 192.168.1.

        </Location>

       

        KeepAliveTimeout 60

        MaxKeepAliveRequests 0

       

        ManagerBalancerName other-server-group

        AdvertiseFrequency 5

        EnableMCPMReceive

       

      </VirtualHost>

      {code}

       

      I'm sure that i'm missing something obvious but i don't know what....

       

      Any help would be greatly appreciated!

        • 2. Re: mod_cluster manager does not discover JBoss nodes
          Max Pimm Newbie

          Thanks Jean. I'm sure that this is a firewall / network related problem and i've got it working with errors now. If i explain my symptoms maybe the problem will be obvious to you.

           

          My machine is a Virtual Box VM with a clean install of Fedora 17.

           

          I have set the firewall zone to "trusted" which i understand allows all network connections.

          I have disactivated SELinux (setenforce 0).

           

          I am new to fedora and think that with this i have disactivated the firewall completely but maybe i'm missing something?

           

          These are the symptoms that i have currently, i hope it makes some sense to you...

           

          Advertise doesn't seem to be working and i don't know how to debug it. However i've tried adding a proxy-list="192.168.1.103:10001" in the definition of modcluster in JBoss domain.xml.

           

          Now when i start the server i get socket connection errors each time it tries to reach the proxy:

           

          IO error sending command CONFIG to proxy lgtdev02/192.168.1.103:10001: java.net.SocketTimeoutException: Read timed out

           

          However, nmap -p 10001 192.168.1.103 gives

          10001/tcp open  scp-config

           

          But, strangely enough, apache does become aware of my node and it shows up in mod_cluster-manager. In the apache logs, which are set to debug i can see that it is initializing workers and connecting via ajp.

           

          However, it shows up with just one line in mod_cluster-manager, without listing any Virual Hosts or Contexts.

           

          Now when i shutdown JBoss, it must be sending more info to the proxy since after shutting down my Virtual Host and Context appear (disabled) in mod_cluster-manager.

           

          Now if i restart JBoss and enable the context from mod_cluster-manager.i can successfully access my node using port 80 and passing through apaches mod_cluster

           

          I've also tried adding another instance of JBoss (on a different virtual machine) to the same domain and the same thing happens. I get periodic errors that the instance can't connect to the proxy however it shows up in mod_cluster-manager without Virtual Hosts but after a restart the Virtual Hosts appear and i can enable the context and start using it via mod_cluster. With both nodes in the cluster i've tried stopping one and the balancer correctly uses the other so i think that the cluster is working as expected.

           

          All seems very strange to me... hope that you can illuminate me.

           

          Thanks

          Max

          • 3. Re: mod_cluster manager does not discover JBoss nodes
            Jean-Frederic Clere Master

            "IO error sending command CONFIG" can happen before httpd is stared or when restarting it.

            Which version of AS are you using (having DISABLED context and restart helping rings like an old bug).

            • 4. Re: mod_cluster manager does not discover JBoss nodes
              Max Pimm Newbie

              Httpd is definitely running when i get the "IO error sending command CONFIG". I can see the calls in apaches access_log.

               

              192.168.1.103 - - [12/Feb/2013:15:44:21 +0100] "CONFIG / HTTP/1.1" 200 - "-" "ClusterListener/1.0"


               

              JBoss version is AS 7.1.1.Final.

               

              I have two network cards configured in this virtual machine... they both have their firewall zones set to trusted but i'm not sure whether that affects anything.

              • 5. Re: mod_cluster manager does not discover JBoss nodes
                Jean-Frederic Clere Master

                then errors are probably due to a problem with your network.

                • 6. Re: mod_cluster manager does not discover JBoss nodes
                  Radoslav Husar Master

                  +1 I haven't seen anything like that before.

                   

                  Try temporarily disabling the firewall completely.

                  • 7. Re: mod_cluster manager does not discover JBoss nodes
                    Max Pimm Newbie

                    I agree... something is wrong with my network setup.

                     

                    I've done

                     

                    service firewalld stop

                    setenforce 0

                     

                    but the problem persists.

                     

                    Is that enough to disable the firewall completely?

                    • 8. Re: mod_cluster manager does not discover JBoss nodes
                      Rob Ferguson Newbie

                      Perhaps try

                       

                      service iptables stop

                      • 9. Re: mod_cluster manager does not discover JBoss nodes
                        fatih hilal Newbie

                        Hi,

                         

                        I encountered exactly same problem.

                        My platform is Windows Server 2008R2SP1, but i also tried Ubuntu and RHEL6.2

                        i am using Jboss AS 7.1.1 Final and Apache2.2.

                         

                        Seperately, i controlled firewall and advertisement status repeatedly.

                        I am not still seeing anything in mod_cluster-manager page except title that written as mod_cluster/1.2.0.Final

                         

                        And my virtualhost configuration.

                         

                        Please Help,

                         

                        Listen 192.168.10.10:6666                    -> I also tried *.6666

                        <VirtualHost 192.168.10.10:6666>               -> I also tried *.6666

                          <Location />
                            Order deny,allow
                            Allow from all
                          </Location>

                          <Location /mod_cluster-manager>
                           SetHandler mod_cluster-manager
                           Order deny,allow
                           Allow from all
                          </Location>

                                                                                                                                      -> I also tried adding these: "AllowDisplay on" and "AdvertiseGroup 192.168.10.10:23364" 

                          KeepAliveTimeout 60
                          MaxKeepAliveRequests 0
                          ServerAdvertise on                                   -> I also tried this: ServerAdvertise on http://192.168.10.10:6666
                          ManagerBalancerName other-server-group              
                        -> I also tried "mycluster" instead of "other-server-group"
                          AdvertiseFrequency 5
                          EnableMCPMReceive

                        </VirtualHost>

                         

                         

                        Best Regards,

                        • 10. Re: mod_cluster manager does not discover JBoss nodes
                          Michal Karm Babacek Apprentice

                          Hmm, looks odd. For instance, AdvertiseGroup 192.168.10.10:23364 does not look like an UDP address.

                          If you want to take an afvantage of advertising feature, you must have matching AdvertiseGroup on httpd side and <socket-binding name="modcluster"... on AS7 side.

                          Furthermore, you must have multicast allowed between boxes (applies even for running on a single box, localhost...).

                           

                          Last but not least, seeing your error_log (on LogLevel debug) would help immensely. BTW: I assume you did try to switch the whole windows firewall off for a while, right?

                           

                          This might help you with debugging:

                          • compile this tiny app: Advertize.java
                          • use it from command line, e.g. like: java Advertize 224.0.1.105 23364
                          • if your httpd advertises, it should show up

                           

                          HTH

                          Let me know...

                          • 11. Re: mod_cluster manager does not discover JBoss nodes
                            fatih hilal Newbie

                            Thanks for your reply,

                             

                            yes, you are right 192.168.10.10:23364 is not an UDP address. It is IP address of my server.

                            I thought I had to write my IP address instead of 224.0.1.105. Now, I fixed.

                             

                            I used Advertize.java. 

                            The following output is written every 5 seconds

                            That is output:

                             

                            received from /192.168.10.10:23364

                            received: HTTP/1.0 200 OK

                            Date: Sat, 23 Feb 2013 12:41:33 GMT

                            Sequence: 164

                            Digest: a836f8810eb2c4c90f0514d2542981d1

                            Server: ae41a358-0821-104d-80c6-f870898a95d2

                            X-Manager-Address: 192.168.10.10:6666

                            X-Manager-Url: /ae41a358-0821-104d-80c6-f870898a95d2

                            X-Manager-Protocol: http

                            X-Manager-Host: jbosscluster01

                             

                             

                            And I don't understand exactly what you mean.

                            So, what should i write as AdvertiseGroup in httpd.conf or instead of "modcluster" on AS7 side?

                            In my domain.xml on AS7 side, that is written

                            <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>

                            Here, is correct port "0"?

                             

                            And, unfortunately, there is not any log in event viewer

                            Last, Whole windows firewall off

                             


                            • 12. Re: mod_cluster manager does not discover JBoss nodes
                              Radoslav Husar Master

                              So, what should i write as AdvertiseGroup in httpd.conf or instead of "modcluster" on AS7 side?

                              Just make sure the ports and the multicast address is the same on the LB and on the AS node.

                              Here, is correct port "0"?

                              Yes, thats correct; multicast-port is what is interesting when configuring autodiscovery.