13 Replies Latest reply on Oct 10, 2014 7:16 AM by cristi.tetic

    mod_cluster manager does not discover JBoss nodes

    maxmil

      I have followed the instructions at https://docs.jboss.org/author/display/AS71/AS7+Cluster+Howto but am stuck.

       

      I wonder if someone could give me a push in the right direction.

       

      I have a JBoss domain instance and httpd with mod_cluster installed on the same machine (Fedora 17 with IP 192.168.1.103)

       

      I have deployed the cluster-demo app in the instructions in the other server group and the server starts with no errors and the message: Listening to proxy advertisements on 224.0.1.105:23364

       

      Httpd starts without errors and I can access http://192.168.1.103:10001/mod_cluster-manager but it shows an empty list of nodes.

       

      httpd.conf loads the necessary modules and declares the following virtual host

       

      {code:XML}

      <VirtualHost 192.168.1.103:10001>

       

        <Directory />

          Order deny,allow

          Deny from all

          Allow from 192.168.1.

        </Directory>

       

        <Location /mod_cluster-manager>

         SetHandler mod_cluster-manager

         Order deny,allow

         Deny from all

         Allow from 192.168.1.

        </Location>

       

        KeepAliveTimeout 60

        MaxKeepAliveRequests 0

       

        ManagerBalancerName other-server-group

        AdvertiseFrequency 5

        EnableMCPMReceive

       

      </VirtualHost>

      {code}

       

      I'm sure that i'm missing something obvious but i don't know what....

       

      Any help would be greatly appreciated!

        • 1. Re: mod_cluster manager does not discover JBoss nodes
          jfclere

          The guess is that advertise is not working (firewall) look to http://docs.jboss.org/mod_cluster/1.2.0/html/faq.html

          • 2. Re: mod_cluster manager does not discover JBoss nodes
            maxmil

            Thanks Jean. I'm sure that this is a firewall / network related problem and i've got it working with errors now. If i explain my symptoms maybe the problem will be obvious to you.

             

            My machine is a Virtual Box VM with a clean install of Fedora 17.

             

            I have set the firewall zone to "trusted" which i understand allows all network connections.

            I have disactivated SELinux (setenforce 0).

             

            I am new to fedora and think that with this i have disactivated the firewall completely but maybe i'm missing something?

             

            These are the symptoms that i have currently, i hope it makes some sense to you...

             

            Advertise doesn't seem to be working and i don't know how to debug it. However i've tried adding a proxy-list="192.168.1.103:10001" in the definition of modcluster in JBoss domain.xml.

             

            Now when i start the server i get socket connection errors each time it tries to reach the proxy:

             

            IO error sending command CONFIG to proxy lgtdev02/192.168.1.103:10001: java.net.SocketTimeoutException: Read timed out

             

            However, nmap -p 10001 192.168.1.103 gives

            10001/tcp open  scp-config

             

            But, strangely enough, apache does become aware of my node and it shows up in mod_cluster-manager. In the apache logs, which are set to debug i can see that it is initializing workers and connecting via ajp.

             

            However, it shows up with just one line in mod_cluster-manager, without listing any Virual Hosts or Contexts.

             

            Now when i shutdown JBoss, it must be sending more info to the proxy since after shutting down my Virtual Host and Context appear (disabled) in mod_cluster-manager.

             

            Now if i restart JBoss and enable the context from mod_cluster-manager.i can successfully access my node using port 80 and passing through apaches mod_cluster

             

            I've also tried adding another instance of JBoss (on a different virtual machine) to the same domain and the same thing happens. I get periodic errors that the instance can't connect to the proxy however it shows up in mod_cluster-manager without Virtual Hosts but after a restart the Virtual Hosts appear and i can enable the context and start using it via mod_cluster. With both nodes in the cluster i've tried stopping one and the balancer correctly uses the other so i think that the cluster is working as expected.

             

            All seems very strange to me... hope that you can illuminate me.

             

            Thanks

            Max

            • 3. Re: mod_cluster manager does not discover JBoss nodes
              jfclere

              "IO error sending command CONFIG" can happen before httpd is stared or when restarting it.

              Which version of AS are you using (having DISABLED context and restart helping rings like an old bug).

              • 4. Re: mod_cluster manager does not discover JBoss nodes
                maxmil

                Httpd is definitely running when i get the "IO error sending command CONFIG". I can see the calls in apaches access_log.

                 

                192.168.1.103 - - [12/Feb/2013:15:44:21 +0100] "CONFIG / HTTP/1.1" 200 - "-" "ClusterListener/1.0"


                 

                JBoss version is AS 7.1.1.Final.

                 

                I have two network cards configured in this virtual machine... they both have their firewall zones set to trusted but i'm not sure whether that affects anything.

                • 5. Re: mod_cluster manager does not discover JBoss nodes
                  jfclere

                  then errors are probably due to a problem with your network.

                  • 6. Re: mod_cluster manager does not discover JBoss nodes
                    rhusar

                    +1 I haven't seen anything like that before.

                     

                    Try temporarily disabling the firewall completely.

                    • 7. Re: mod_cluster manager does not discover JBoss nodes
                      maxmil

                      I agree... something is wrong with my network setup.

                       

                      I've done

                       

                      service firewalld stop

                      setenforce 0

                       

                      but the problem persists.

                       

                      Is that enough to disable the firewall completely?

                      • 8. Re: mod_cluster manager does not discover JBoss nodes
                        rjferguson21

                        Perhaps try

                         

                        service iptables stop

                        • 9. Re: mod_cluster manager does not discover JBoss nodes
                          monte_cristo_kontu_81

                          Hi,

                           

                          I encountered exactly same problem.

                          My platform is Windows Server 2008R2SP1, but i also tried Ubuntu and RHEL6.2

                          i am using Jboss AS 7.1.1 Final and Apache2.2.

                           

                          Seperately, i controlled firewall and advertisement status repeatedly.

                          I am not still seeing anything in mod_cluster-manager page except title that written as mod_cluster/1.2.0.Final

                           

                          And my virtualhost configuration.

                           

                          Please Help,

                           

                          Listen 192.168.10.10:6666                    -> I also tried *.6666

                          <VirtualHost 192.168.10.10:6666>               -> I also tried *.6666

                            <Location />
                              Order deny,allow
                              Allow from all
                            </Location>

                            <Location /mod_cluster-manager>
                             SetHandler mod_cluster-manager
                             Order deny,allow
                             Allow from all
                            </Location>

                                                                                                                                        -> I also tried adding these: "AllowDisplay on" and "AdvertiseGroup 192.168.10.10:23364" 

                            KeepAliveTimeout 60
                            MaxKeepAliveRequests 0
                            ServerAdvertise on                                   -> I also tried this: ServerAdvertise on http://192.168.10.10:6666
                            ManagerBalancerName other-server-group              
                          -> I also tried "mycluster" instead of "other-server-group"
                            AdvertiseFrequency 5
                            EnableMCPMReceive

                          </VirtualHost>

                           

                           

                          Best Regards,

                          • 10. Re: mod_cluster manager does not discover JBoss nodes
                            mbabacek

                            Hmm, looks odd. For instance, AdvertiseGroup 192.168.10.10:23364 does not look like an UDP address.

                            If you want to take an afvantage of advertising feature, you must have matching AdvertiseGroup on httpd side and <socket-binding name="modcluster"... on AS7 side.

                            Furthermore, you must have multicast allowed between boxes (applies even for running on a single box, localhost...).

                             

                            Last but not least, seeing your error_log (on LogLevel debug) would help immensely. BTW: I assume you did try to switch the whole windows firewall off for a while, right?

                             

                            This might help you with debugging:

                            • compile this tiny app: Advertize.java
                            • use it from command line, e.g. like: java Advertize 224.0.1.105 23364
                            • if your httpd advertises, it should show up

                             

                            HTH

                            Let me know...

                            • 11. Re: mod_cluster manager does not discover JBoss nodes
                              monte_cristo_kontu_81

                              Thanks for your reply,

                               

                              yes, you are right 192.168.10.10:23364 is not an UDP address. It is IP address of my server.

                              I thought I had to write my IP address instead of 224.0.1.105. Now, I fixed.

                               

                              I used Advertize.java. 

                              The following output is written every 5 seconds

                              That is output:

                               

                              received from /192.168.10.10:23364

                              received: HTTP/1.0 200 OK

                              Date: Sat, 23 Feb 2013 12:41:33 GMT

                              Sequence: 164

                              Digest: a836f8810eb2c4c90f0514d2542981d1

                              Server: ae41a358-0821-104d-80c6-f870898a95d2

                              X-Manager-Address: 192.168.10.10:6666

                              X-Manager-Url: /ae41a358-0821-104d-80c6-f870898a95d2

                              X-Manager-Protocol: http

                              X-Manager-Host: jbosscluster01

                               

                               

                              And I don't understand exactly what you mean.

                              So, what should i write as AdvertiseGroup in httpd.conf or instead of "modcluster" on AS7 side?

                              In my domain.xml on AS7 side, that is written

                              <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>

                              Here, is correct port "0"?

                               

                              And, unfortunately, there is not any log in event viewer

                              Last, Whole windows firewall off

                               


                              • 12. Re: mod_cluster manager does not discover JBoss nodes
                                rhusar

                                So, what should i write as AdvertiseGroup in httpd.conf or instead of "modcluster" on AS7 side?

                                Just make sure the ports and the multicast address is the same on the LB and on the AS node.

                                Here, is correct port "0"?

                                Yes, thats correct; multicast-port is what is interesting when configuring autodiscovery.

                                • 13. Re: mod_cluster manager does not discover JBoss nodes
                                  cristi.tetic