1 Reply Latest reply on Aug 10, 2002 9:04 AM by slaboure

    Possible to use a "backplane" to connect the servers??

    joao.clemente


      This message was originally posted in JBoss-user mailing list and I'm moving the thread to the forum as Sacha asked.
      Here goes the full message and my reply:

      >[JBoss-user] Clustering: Possible to use a "backplane" to connect the servers??
      >
      >
      >> I am thinking if it's possible to use a 2nd network card to implement a
      >> interconnection backplane between the clustered servers.
      >
      >> Let me "ascii-art" this :
      >>
      >>
      >> ||
      >> Load Balancer
      >> ||
      >> --------SWITCH--------
      >> | | | |
      >> | | | |
      >> JB1 JB2 JB4 JB5
      >> | | | |
      >> --HUB-- --HUB--
      >>
      >>
      >> In this example JB{1,2} are a cluster, and JB{3,4} are a second cluster.
      >> The replication messages would be contained in the backplane between the
      >> elements of each cluster and would not overload the other cluster.
      >> The switch between the LoadBalancer and the JB instances would redirect
      >> the requests to the servers, not depending on the saturation of the
      >> backplanes.
      >>
      >> Well, with this I'm asking 2 questions:
      >>
      >> 1 - Can JBoss use a topology like this?
      >
      >yes: in the JavaGroups properties, you can indicate which network interface
      >to use.
      >
      >> 2 - Someone else thinks that this (topology) can be usefull/more
      >> efficient?
      >
      >(Could you explain me your topology first? how you use it? That's
      >interesting. )

      Well, I was thinking that when you start needing a very large number of machines because of processing power limitations, you wont improve the cluster performance linearly because each machine memory gets loaded with replicated session they dont use (unless a fail occurs, wich is something unlikely to happen the most of the time).
      Therefore, it makes sense to split the machines in smaller clusters of 2-3 machines. A bigger number of machines in each cluster will provide a bigger fault tolerance (more machines can fail without compromising the service). IMO, to provide HA and maximize resource usage, a 2 node cluster is a good number and 3 is acceptable.
      If I got it right, this is what JBossClutering refers as "partitions" and I'll use this term now on:

      For what I'm discussing here, it's not important wether the partition have or not the same services or access the same data. I'm only trying to maximize the cluster performance as a whole. So, let me explain the lower-level detail that I was worrying about. I'll rever to the elements discribed in the "ascii-art" above:

      A request (R1) gets to the LB and is directed to JB1.
      R1 goes switched jo JB1 without loading other JB's network interfaces.
      JB1 creates session1 (S1) that is replicated at partition1 through multicast. This replication is done using the partition1 bacplane and machines in partition2 will not see this message (and therefore will not introduce processor or network load at JB3 and JB4).

      What are the alternatives?
      - Using the same backplane for both partitions will introduce unnecessary network and processor load.
      - Using a switch instead of a hub in the backplane is of no use because a multicast is a broadcast at mac-level and so you would get no improvement
      - Not using a backplane,or connecting the LB to the backplane(s): The LB would get the replication messages that he doesn't need.
      - Using a hub instead of a switch between LB and JB: Every JB would get a request that is only meant for one of them. This could only be usefull if there was a takeover feature (active replication) + hearbeat or something like that. JBoss does not have such an option, the failover is controlled at LB or client side.


      Sacha, tring to answer your question on how to use it: The load balancer algorithm is the solution for a good workload division, and for a correct failover. As I've stated before, I think this topology is usefull for both the cenarios where you need to maximize the number of users you can have simultaneosly and where you want minimize latency (have a faster reply):
      - If you have the same services/data reached from particion1 and partition2, you'll have a greater resource (memory usage) profit: supose each machine has 100 Mb to hold sessions. In my topology with 2 partition x 2 nodes/partition, each machine will hold "usefull" 50 Mb of sessions, what means that the cluster has 200Mb of usefull memory. If you had a single 4-node partition, each machine would hold 25Mb "usefull" memory, giving you a total of 100 Mb usefull cluster memory.

      >First, it depends on your switch. If it has enough processing power and a
      >good backplane, you don't need a second set of interfaces. In this case, you
      >must either configure your switch to not forward multicast (of a particular
      >port) from one side to the other => you need a switch working a the OSI
      >layer 4 (=> a router) or you need to change the JavaGroups properties to use
      >TCP instead of multicast. Only membership discovery paquets will then travel
      >from one side to the other => low traffic.

      You'll have the incoming requests packets from the LB competing with replication packets between the JB instances. Even if you use TCP instead of multicast: They are crossing the same physical lines.
      Anyway, I want to understand your cenario better. You are saying the backplane and the connection from LB to JBoss instances can be in the same physical network, right?


      >If you have a "slow switch" (or a very big inter-node communication), it
      >will certainly help to split the load yes.
      >
      >Cheers,
      >
      > Sacha
      >
      >P.S.: could you please use the clustering online forums? So we have a
      >unified access for archives. Thank you.

      I've moved this thread here. I take this chance to say 2 things: I've found the forum a worse way of placing messages, because it's a bit heavy/slow and sometimes not pratical. I think other users think the same, and that is one reason that makes the mailing list more responsive than the forum.
      I would like to ask if there is any toughts of having some sort of nntp mapping to the forums or mailing list. I'm not sure how dificcult would be to get this running. That way, you could acess the forum either from a web interface or a news reader, wich would allow the offline use of the forum.


      Hope this message explains what I was thinking to provide or improve by using this topology. If not, I'll be happy to deeply explore any tought.

      Thank you
      Joao Clemente
      INESC - Portugal

        • 1. Re: Possible to use a "backplane" to connect the servers??
          slaboure

          > Well, I was thinking that when you start needing a
          > very large number of machines because of processing
          > power limitations, you wont improve the cluster
          > performance linearly because each machine memory gets
          ...
          > one of them. This could only be usefull if there was
          > a takeover feature (active replication) + hearbeat or
          > something like that. JBoss does not have such an
          > option, the failover is controlled at LB or client
          > side.

          OK, for this, there is also another solution which is already implemented but not currently used (too complicated for now): sub-partitions. You can have, let's say a 10 node cluster, and have each node only replicate its state to 1 other node in the cluster, called a brother node. If the node dies, the sub-partitions topology (couples) are automatically recomputed and state exchange between new couples. This is already in the codebase but not used.

          so yes, sharing the whole state in a big cluster cannot scale indefinitvely.

          > >First, it depends on your switch. If it has
          > enough processing power and a
          > >good backplane, you don't need a second set of
          > interfaces. In this case, you
          > >must either configure your switch to not forward
          > multicast (of a particular
          > >port) from one side to the other => you need a
          > switch working a the OSI
          > >layer 4 (=> a router) or you need to change
          > the JavaGroups properties to use
          > >TCP instead of multicast. Only membership
          > discovery paquets will then travel
          > >from one side to the other => low traffic.
          >
          > You'll have the incoming requests packets from the LB
          > competing with replication packets between the JB
          > instances. Even if you use TCP instead of multicast:
          > They are crossing the same physical lines.
          > Anyway, I want to understand your cenario better. You
          > are saying the backplane and the connection from LB
          > to JBoss instances can be in the same physical
          > network, right?

          yes, they are crossing the same line, and what? If your switch backplane is not the cluster bottleneck, then you are fine.

          > I've moved this thread here. I take this chance to
          > say 2 things: I've found the forum a worse way of
          > placing messages, because it's a bit heavy/slow and
          > sometimes not pratical. I think other users think the

          yes, that's right.

          > same, and that is one reason that makes the mailing
          > list more responsive than the forum.
          > I would like to ask if there is any toughts of having
          > some sort of nntp mapping to the forums or mailing
          > list. I'm not sure how dificcult would be to get this
          > running. That way, you could acess the forum either
          > from a web interface or a news reader, wich would
          > allow the offline use of the forum.

          This idea is in the air for months.... If it should be done? Yes. "When?" is an un-authorized question ;)