4 Replies Latest reply on Mar 11, 2014 3:15 PM by Michal Karm Babacek

    mod_cluster load balancing problem

    Tirthankar Bhadra Newbie

      Hi,

      I am running with 4 servers(1101, 1102, 1103 and 1104) in a cluster (JBoss EAP 5.1.1).These servers are LB with Apache and mod_cluster. The problem I am facing is whenever I start 1102, all the request starts going into this box. If I stop the 1102 instance then the loads are equally getting distributed. The boxes are VM boxes, and we tried rebuilding the box 1102 from scratch as well.

       

      Need your help in debugging the cause of this issue. We are using introscope to monitor these servers.

        • 1. Re: mod_cluster load balancing problem
          Radoslav Husar Master

          It is very likely that brining a new mode, most of the new requests are going to go to that node, since this is the new node with no load.

           

          Look at the mod_cluster management console and look at the loads -- what does it say (load field)? Is 1102 really the one with highest factor?

           

          What load-balancing factor provider are you using? (cpu, mem, etc..).

          • 2. Re: mod_cluster load balancing problem
            Michal Karm Babacek Apprentice

            +1 To Rado's answer.

             

            I would add another one regarding this statement:

            all the request starts going into this box

            Do you use stickySession? e.g. something like:

            <property name="stickySession">true</property>
            <property name="stickySessionForce">false</property>
            <property name="stickySessionRemove">false</property>
            
            

            If it is so, requests that belongs to sessions created prior to server 1102 joining the balancer should indeed keep going to "their" respective servers.

            • 3. Re: mod_cluster load balancing problem
              Kuldeep Sharma Novice

              Please share the logs and the mod_cluster screen shot during the issue.

              That can help to troubleshoot further.

              • 4. Re: mod_cluster load balancing problem
                Michal Karm Babacek Apprentice

                ...just a one more piece of info: One must give the system some time, in matter of the amount of new sessions created, to settle and pick other nodes.

                 

                An example from an actual environment:

                You have 3 nodes with the following Load values:

                • Node jboss-6, Load: 20
                • Node jboss-6-2, Load: 90
                • Node jboss-6-3, Load: 1

                Yes, this means that jboss-6-2 is almost not loaded at all whereas jboss-6-3 is desperately overloaded.

                Now, I send 1001 requests, each representing a new session (the client is forgetting cookies). The distribution of the requests will be as follows:

                • Node jboss-6 served 181 requests
                • Node jboss-6-2 served 811 requests
                • Node jboss-6-3 served 9 requests

                So, generally, yes, the least loaded box received by far the greatest amount of requests, but it did not receive them all. Furthermore, and this concerns your case, for some time from the start, it was jboss-6 who was getting requests.

                This whole magic is in place in order to prevent congestion.