7 Replies Latest reply on Nov 13, 2012 9:16 AM by fanooos

    Jboss Messaging Clustering Strange behaviore

    fanooos

      I am trying to get a cluster of two jboss nodes working but I met some strange behavior

       

      First of all here is how I installed the two nodes.

       

      The two nodes working in standalone mode with the configuration standalone-full-ha.xml.

       

      Here is the changes I have made in the two configuration files.

       

      1- the interfaces section -> I have changed the IPs in the two files from 127.0.0.1 to the machine IPs

      the first file

      ----------------

      <interfaces>

              <interface name="management">

                  <inet-address value="${jboss.bind.address.management:193.16.1.67}"/>

              </interface>

              <interface name="public">

                  <inet-address value="${jboss.bind.address:193.16.1.67}"/>

              </interface>

              <!-- TODO - only show this if the jacorb subsystem is added  -->

              <interface name="unsecure">

                  <!--

                    ~  Used for IIOP sockets in the standard configuration.

                    ~                  To secure JacORB you need to setup SSL

                    -->

                  <inet-address value="${jboss.bind.address.unsecure:193.16.1.67}"/>

              </interface>

          </interfaces>

       

       

      the second file

      ---------------------

      <interfaces>

              <interface name="management">

                  <inet-address value="${jboss.bind.address.management:193.16.1.93}"/>

              </interface>

              <interface name="public">

                  <inet-address value="${jboss.bind.address:193.16.1.93}"/>

              </interface>

              <!-- TODO - only show this if the jacorb subsystem is added  -->

              <interface name="unsecure">

                  <!--

                    ~  Used for IIOP sockets in the standard configuration.

                    ~                  To secure JacORB you need to setup SSL

                    -->

                  <inet-address value="${jboss.bind.address.unsecure:193.16.1.93}"/>

              </interface>

          </interfaces>

       

      2- For nodes to discover esch others I am using TCP instead of UDP and here is the configuration in the two files

      <subsystem xmlns="urn:jboss:domain:jgroups:1.1" default-stack="tcp">

                  <stack name="udp">

                      ................

                  </stack>

                  <stack name="tcp">

                      <transport type="TCP" socket-binding="jgroups-tcp" diagnostics-socket-binding="jgroups-diagnostics"/>

                          <protocol type="TCPPING">

                          <property name="initial_hosts">193.16.1.67[7600],193.16.1.93[7600]</property>

                          <property name="num_initial_members">2</property>

                          <property name="port_range">0</property>

                          <property name="timeout">2000</property>

                      </protocol>

                      ...................

                  </stack>

              </subsystem>

       

      Now here is the problems I am facing and can not understand what is the wrong with my installation

       

      1- I have developed a little java application that send messages to the first machine (193.16.1.93) and tries to consume the message from the other machine (193.16.1.67) and this works perfectly but when I try to revers the IPs (send to 193.16.1.67 and consum from 193.16.1.93) the sender works fine but the messages does not delivered to the machine 193.16.1.93 (the message return is null) which indicates that the messages is not sent from machine 67 to machine 93 -while the reverts scenario works fine-. what would be the problem here?

       

      2- When restaring jboss on the machine 67, I got a warnning message in the losg of the other node that (there is another node in the same network broadcast the same node id). I understand that this is a normal warnning message in case of node restart and I am assuming that if I reversed the process (restart node 93 and check node 67 logs) the same warnning will appeare in the node 67 log but this does not happend. Also, what would be the problem here ?

       

      3- Node 67 frequently throws the exception Faild To create netty connection SocketTimeOutException.

       

      I am using Jboss 7.1.1-final and the code of the application I use to send and consume the message is attached

       

      I hope getting any help here

       

      Message was edited by: Anas Rabei

        • 1. Re: Jboss Messaging Clustering Strange behaviore
          jbertram

          The first point to make is that HornetQ doesn't use JGroups for clustering so your configuration is invalid.  I suggest you read the HornetQ User Guide on this subject.

           

          Once you straighten out the configuration my guess is that you'll see different behavior.

          • 2. Re: Jboss Messaging Clustering Strange behaviore
            fanooos

            Thanks Justine for your reply.

             

            Please note that this is my first time using Jboss and HornetQ

             

            If HornetQ doesn't use JGroups for clustering or if my configurations are invalid,  how would you explain when I send message to one node in the cluster (193.16.1.93) it is delivered to the second node (193.16.1.67) and I can consume it from the other node ?

             

            By the way, I am following the steps in the book (JBoss AS 7 Configuration, Deployment, and Administration) for configuring the messagin cluster. In this book I found that the only thing I need to configure is to make <clustered>true</clustered> in the hornetq-server subsystem and I left the other settings to default values.

             

            Thanks in advance

            • 3. Re: Jboss Messaging Clustering Strange behaviore
              jbertram

              My guess is that the two nodes are clustering at some level from a HornetQ perspective since you are using standalone-full-ha.xml which makes HornetQ cluster by default.

              • 4. Re: Jboss Messaging Clustering Strange behaviore
                fanooos

                OK, if I am not wrong, this means the configuration is correct. Here we back to the same point.

                • 5. Re: Jboss Messaging Clustering Strange behaviore
                  jbertram

                  Can you attach the server.log files from both nodes?  Make sure they include the server's start-up process so I can verify the nodes are clustering as expected.

                  • 6. Re: Jboss Messaging Clustering Strange behaviore
                    fanooos

                    I have editted the post and attached 4 additional files.

                     

                    The first two files (prefixed 67) are the configuration and server log of the node (193.16.1.67) and the other two files (prefixed 93) are the configuration and server log of the node (193.16.1.39)

                    • 7. Re: Jboss Messaging Clustering Strange behaviore
                      fanooos

                      After some trials I found that the problem is in one of the two machines and here what I have done to reach to this result.

                       

                      The two machines I was tring on were 193.16.1.93 and 193.16.1.67. I have added another machine 193.16.1.146 and make a cluster of the two machines 93 and 146.

                       

                      With the cluster of these two machines I got the same scenario.

                       

                      So I tried to exclude the machine 93 and make a cluster of the other two machines (67, and 146).

                       

                      When I made the cluster of these two machines, every thing worked properly. I tried the same application, the message is sent to the first machine and consumed from the other machine. Then I switched the IPs and again the messages is delivered correctly to the two machines.

                       

                      So I suspected that there is a problem in the machine 93. I checked the firewall and it was off.

                       

                      What would be the problem with this machine ? Is there any thing (rather than the firewall) can prevent the other machine to connect to this machine ?

                       

                      Thanks in advance