9 Replies Latest reply on Apr 10, 2007 7:36 PM by alpheratz-jb

    Configuration for redundant ethernets

    alpheratz-jb

      I have two multihomed boxes.

      Each box is running two instances of JBoss 4.0.5GA, configured using
      the ServiceBindingManager. This all works very nicely as I have it at the moment.

      Now, Each box has 4 Ethernet NICs.

      I intend to use two NICs for HTTP service traffic and two for intra-cluster housekeeping.

      The question is: how do I configure this?

      * For the configuration of the service interfaces...
      I am currently allowing each instance to bind to 0.0.0.0 -- ie all interfaces.

      This is OK but it would be nice to be able to say "bind to ONLY interfaces A and B"
      instead of "bind to all interfaces."

      Is there a way to do this?

      * For the configuration of the intra-cluster housekeepting traffic...
      I am looking at ...deploy/cluster-service.xml and can see that I can establish
      the <UDP bind_addr attrribute. Presumably, this tells the JBoss instance to
      listen on this address for intra-cluster housekeeping 'chatter' and of course,
      the address used will correspond to a given interface.

      Can I configure a list of addresses in bind_addr? Ie:

      <UDP bind_addr="192.168.10.1,192.168.11.1" ...
      


      This would enable me to loose an individual link but not loose the cluster.

      I have had a look through the Wiki and this forum, but I can't see anybody dealing
      with redundancy, as I have to.

      Thoughts/suggestions gratefully accepted.

      Cheers

      Alph

        • 1. Re: Configuration for redundant ethernets
          belaban

          You can define a UDP.receive_interfaces="192.168.10.1,192.168.11.1" property. Alternatively, you can also list the interfaces by device name rather than assigned IP address, e.g.
          receive_interfaces="eth0,eth1".
          This was introduced in JGroups 2.3 (not sure), so you may have to upgrade JGroups.

          • 2. Re: Configuration for redundant ethernets
            alpheratz-jb

            Thank you...fast reply, very impressive.

            Do you think that I will be able to pick up the version shipped with 4.2.0CR1 and put it into 4.0.5GA?

            C:\DEVTOOLS\jboss-4.2.0.CR1\server\all\lib>java -classpath jgroups.jar org.jgroups.Version
            
            Version: 2.4.1
            CVS: $Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $
            History: (see doc/history.txt for details)
            


            Any other dependencies to look out for?

            Cheers,

            Alph

            • 3. Re: Configuration for redundant ethernets
              belaban

              Yes, that will work. However, you need to set the system property:
              Use of JGroups 2.4 or later with these JBoss AS versions requires starting JBoss with -Djgroups.marshalling.compatible=true

              as described in http://labs.jboss.com/portal/jbosscache/compatibility/index.html

              • 4. Re: Configuration for redundant ethernets

                Bela - the compatibility matrix indicates that you don't need the -Djgroups.marshalling.compatible=true property setting if you're running JGroups 2.4.x with JBossAS 4.0.5.GA. It does note that earlier versions of JBossAS require the setting.

                We're planning to use JGroups 2.4.1-SP1 with JBossAS 4.0.5.GA so we're interested in whether this setting is required.

                Jerry

                • 5. Re: Configuration for redundant ethernets
                  belaban

                  You're right, sorry for the misinformation

                  • 6. Re: Configuration for redundant ethernets
                    brian.stansberry

                    Yeah, we noticed the issue in between 4.0.5.CR1 and 4.0.5.GA and built the fix into GA.

                    • 7. Re: Configuration for redundant ethernets
                      alpheratz-jb

                      Thanks guys (all of you!).

                      I picked up the 4.2.0CR1 version and so far it seems to be "a thing of beauty."

                      Cheers,

                      Alph

                      • 8. Re: Configuration for redundant ethernets
                        alpheratz-jb

                        Well, I have found a wrinkle...

                        If I run $JBOSS_HOME/probe.sh, I don't get a listing of the cluster members. The application doesn't show any errors, it just behaves/exits normally, just as it would if there were no cluster members.

                        Previously probe.sh gave me a listing of all the members in the cluster.

                        If I check a server log file, I see:

                        2007-04-10 11:07:02,262 WARN [org.jgroups.protocols.UDP] packet from
                        147.209.225.205:37242 has different version (25705) from ours (241).
                        This may cause problems
                        2007-04-10 11:07:02,263 ERROR [org.jgroups.protocols.UDP] failed unmarshalling message java.io.EOFException
                         at java.io.DataInputStream.readInt(DataInputStream.java:358)
                         at org.jgroups.protocols.TP.bufferToList(TP.java:1020)
                         at org.jgroups.protocols.TP.handleIncomingPacket(TP.java:827)
                         at org.jgroups.protocols.TP.access$400(TP.java:45)
                         at
                        org.jgroups.protocols.TP$IncomingPacketHandler.run(TP.java:1296)
                         at java.lang.Thread.run(Thread.java:595)
                        2007-04-10 11:10:58,178 WARN [org.jgroups.protocols.UDP] packet from
                        147.209.225.205:37262 has different version (25705) from ours (241).
                        This may cause problems
                        


                        I am going to try setting the compatibility flag -Djgroups.marshalling.compatible=true on the command line in probe.sh.

                        My guess is that this is what happens within JBoss 4.0.5GA which "built the fix into GA."?

                        I did have to hack probe.sh a little: to ensure that it picked up the version of jgroups.jar from one of the cluster member's server/lib directories, rather than using the one in default/all (forget which it was now...typing at home).

                        Will report back.

                        Cheers,

                        Alph

                        • 9. Re: Configuration for redundant ethernets
                          alpheratz-jb

                          Setting compatibility mode worked...I think.

                          I now get:

                          [root@bob1 bin]# ./probe.sh
                          
                          -- send probe on /224.0.0.75:7500
                          
                          
                          #1 (332 bytes): 147.209.225.205:37848 (TCEPartition)
                          local_addr=147.209.225.205:37848
                          group_name=TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37848|3] [147.209.225.205:37848, 147.209.225.205:37853, 147.209.225.208:33260, 147.209.225.208:33266]
                          group_addr=230.0.0.88:45566
                          
                          
                          #2 (346 bytes): 147.209.225.205:37846 (Tomcat-TCEPartition)
                          local_addr=147.209.225.205:37846
                          group_name=Tomcat-TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37846|3] [147.209.225.205:37846, 147.209.225.205:37851, 147.209.225.208:33258, 147.209.225.208:33264]
                          group_addr=230.0.0.88:45577
                          
                          
                          #3 (332 bytes): 147.209.225.205:37853 (TCEPartition)
                          local_addr=147.209.225.205:37853
                          group_name=TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37848|3] [147.209.225.205:37848, 147.209.225.205:37853, 147.209.225.208:33260, 147.209.225.208:33266]
                          group_addr=230.0.0.88:45566
                          
                          
                          #4 (346 bytes): 147.209.225.205:37851 (Tomcat-TCEPartition)
                          local_addr=147.209.225.205:37851
                          group_name=Tomcat-TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37846|3] [147.209.225.205:37846, 147.209.225.205:37851, 147.209.225.208:33258, 147.209.225.208:33264]
                          group_addr=230.0.0.88:45577
                          
                          
                          #5 (332 bytes): 147.209.225.208:33266 (TCEPartition)
                          local_addr=147.209.225.208:33266
                          group_name=TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37848|3] [147.209.225.205:37848, 147.209.225.205:37853, 147.209.225.208:33260, 147.209.225.208:33266]
                          group_addr=230.0.0.88:45566
                          
                          
                          #6 (332 bytes): 147.209.225.208:33260 (TCEPartition)
                          local_addr=147.209.225.208:33260
                          group_name=TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37848|3] [147.209.225.205:37848, 147.209.225.205:37853, 147.209.225.208:33260, 147.209.225.208:33266]
                          group_addr=230.0.0.88:45566
                          
                          
                          #7 (346 bytes): 147.209.225.208:33264 (Tomcat-TCEPartition)
                          local_addr=147.209.225.208:33264
                          group_name=Tomcat-TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37846|3] [147.209.225.205:37846, 147.209.225.205:37851, 147.209.225.208:33258, 147.209.225.208:33264]
                          group_addr=230.0.0.88:45577
                          
                          
                          #8 (346 bytes): 147.209.225.208:33258 (Tomcat-TCEPartition)
                          local_addr=147.209.225.208:33258
                          group_name=Tomcat-TCEPartition
                          version=2.4.1, cvs="$Id: Version.java,v 1.42.2.1 2006/12/04 13:57:06 belaban Exp $"
                          view: [147.209.225.205:37846|3] [147.209.225.205:37846, 147.209.225.205:37851, 147.209.225.208:33258, 147.209.225.208:33264]
                          group_addr=230.0.0.88:45577
                          
                          
                          
                          [root@bob1 bin]#
                          


                          My partition is 'TCEPartition'/45566. I can now see entries for 'Tomcat-TCEPartition'/45577 as well.

                          Is this correct? it is certainly different.

                          Also, consider this:

                          [root@bob1 bin]# ./probe.sh -addr 230.0.0.88 -port 45566
                          
                          -- send probe on /230.0.0.88:45566
                          
                          
                          
                          [root@bob1 bin]#
                          


                          in my log, I see:

                          2007-04-11 09:33:00,434 WARN [org.jgroups.protocols.UDP] packet from 147.209.225.205:42856 has different version (20821) from ours (241). This may cause problems
                          2007-04-11 09:33:00,435 ERROR [org.jgroups.protocols.UDP] failed unmarshalling message
                          java.io.EOFException
                           at java.io.DataInputStream.readBoolean(DataInputStream.java:222)
                           at org.jgroups.util.Util.readAddress(Util.java:472)
                           at org.jgroups.protocols.TP.bufferToList(TP.java:1021)
                           at org.jgroups.protocols.TP.handleIncomingPacket(TP.java:827)
                           at org.jgroups.protocols.TP.access$400(TP.java:45)
                           at org.jgroups.protocols.TP$IncomingPacketHandler.run(TP.java:1296)
                           at java.lang.Thread.run(Thread.java:595)
                          


                          Not what I expected...and not what WAS happening

                          FYI, my probe.sh is:

                          #!/bin/sh
                          ### ====================================================================== ###
                          ## ##
                          ## JGroups Cluster Discovery Script ##
                          ## ##
                          ### ====================================================================== ###
                          
                          # Discovers all UDP-based members running on a certain mcast address (use -help for help)
                          # Probe [-help] [-addr <addr>] [-port <port>] [-ttl <ttl>] [-timeout <timeout>]
                          
                          # CLASSPATH=.:../lib/commons-logging.jar:../server/all/lib/jgroups.jar:$CLASSPATH
                          
                          JAVA_HOME="/opt/jrockit-R27.1.0-jdk1.5.0_08"
                          JBOSS_HOME="/opt/jboss-4.0.5.GA"
                          
                          CLASSPATH=.:$JBOSS_HOME/lib/commons-logging.jar:$JBOSS_HOME/server/ports-02/lib/jgroups.jar:$CLASSPATH
                          
                          # OS specific support (must be 'true' or 'false').
                          cygwin=false;
                          case "`uname`" in
                           CYGWIN*)
                           cygwin=true
                           ;;
                          esac
                          
                          if [ $cygwin = "true" ]; then
                           CP=`cygpath -wp $CLASSPATH`
                          else
                           CP=$CLASSPATH
                          fi
                          
                          $JAVA_HOME/bin/java -Djgroups.marshalling.compatible=true -cp $CP org.jgroups.tests.Probe $*
                          


                          The actual cluster seems OK...we are just talking about the behaviour of probe.sh (aren't we?).

                          Cheers,

                          Alph