6 Replies Latest reply on May 3, 2004 3:58 PM by _avi_

    HTTP session clustering in jboss 3.2.1 causing memory proble

    _avi_ Newbie

      Hi All,
      I have 3 servers in production which are being load balanced. Two of these have http session clustering enabled (all of the jboss instances are running on individual windows 2000 servers).
      Following is the config setting from my jboss-service.xml -



      6048
      1000
      300
      TCP(start_port=7801;bind_addr=192.168.xx.xx):TCPPING(initial_hosts=192.168.xx.xx[7801];port_range=10;timeout=30000;num_initial_members=1):FD(shun=true;timeout=60000;max_tries=10):VERIFY_SUSPECT(timeout=1000000):pbcast.STABLE(desired_avg_gossip=10000):pbcast.NAKACK(gc_lag=100;retransmit_timeout=300,600,1200,2400,4800;max_xmit_size=65536):pbcast.GMS(join_timeout=10000;join_retry_timeout=100000;shun=true;print_local_addr=true):pbcast.STATE_TRANSFER

      DefaultSubCluster
      20000
      5000
      GET_NONE



      I have setup jboss as a windows service and from the windows task manager I keep monitoring the memory usage. The mem usage of jboss keeps increasing steadily and when it reaches 428 MB roughly, jboss almost hangs up and my users face trouble connecting to the application - it becomes painfully slow. I have to pull out the servers (that have clustering enabled) from the LB and recycle them and then enable them in the LB.
      Does anybody have any ideas about the memory issue?
      What am I missing? Is something missing in the config settings above? Has anybody faced a similar problem before?

      I would highly appreciate any thoughts on this.

      Thanks in advance.

      Regards,
      Avi

        • 1. Re: HTTP session clustering in jboss 3.2.1 causing memory pr
          Bela Ban Master

          You don't have a very aggressive STABLE layer, and no flow control. I suggest you start out with fc.xml (from JGroups distro), and replace the bottom 2 protocols, minus UNICAST.
          Bela

          • 2. Re: HTTP session clustering in jboss 3.2.1 causing memory pr
            _avi_ Newbie

            Thanks for your response Bela.

            Based on what you wrote, I duly downloaded the latest jgroups package and am still doing further research on how to setup jboss 3.2.1 correctly with fc.xml.
            While doing my research, I found the following post (by you) as a reply to a similar clustering related memory issue by one Joao Clemente -
            http://www.junlu.com/msg/35648.html

            I am pasting a snippet :

            "I just realized what I said is pointless - you would have to use JGroups
            2.1 or higher and I would have to make changes to the JBoss Clustering
            in order for you to use the new optimized flow control for JGroups.

            I will make these changes once I join JBoss (Oct 6th):
            - Update JGroups to the latest stable version
            - Change package names from org.javagroups to org.jgroups
            - Update the X-service.xml files affected (cluster-service.xml) and
            - Provide an example of an efficient high-throughput protocol stack for
            JGroups in cluster-service.xml
            - Plus: documentation of how to use the newer stack

            This will be relevant for high-throughput high load situations, but I 'm
            not sure if this is applicable to you. You mentioned you only send
            around 30-40msgs/sec right ?

            Joao Clemente wrote:

            > Bela, could you point me to some documentation or provide a little better
            > explanation of what to do?
            > I 'm a bit lost here... : I downloaded javagroups 2.1.1, found a fc.xml
            > file
            > inside (btw, there 's a flow-control.xml file aswell, that seems to be
            > a copy
            > of fc.xml), checked what was inside my cluster-service.xml and the only
            > thing I could think of was to copy the values of "mcast_addr ",
            > "mcast_port ",
            > ..., "ip_ttl "
            > from fc.xml to cluster-service.xml... but I 'm not sure if that is what you
            > were suggesting me... "

            Is there any documentation available on this? I am sure lots of developers would be anxious to read about the correct way of using flow control. Also, my problem is I cannot upgrade jboss. I will have to continue to use 3.2.1. Another thing about my setup is, that I have removed cluster-service.xml from \deploy. I have made my web app distributable and enabled the TCP protocol stack in jboss-service.xml.

            I will dig for further information but if you can point to any documentation it would be immensely helpful.

            Thanks once again.

            Regards,
            Avi

            • 3. Re: HTTP session clustering in jboss 3.2.1 causing memory pr
              Bela Ban Master

              I plan to add a flow-control based configuration to JBoss soon. However, this requires some stuff in JGroups that's fairly new. This will be available in JBoss 3.2.4 (or 3.2.5), but you will most probably be able to just take the config file and jgroups.jar from 3.2.4 and copy it to 3.2.1.
              Bela

              • 4. Re: HTTP session clustering in jboss 3.2.1 causing memory pr
                _avi_ Newbie

                Thanks Bela.
                It will be great if you can post up a link or let us know once it's ready.
                I checked from sourceforge, release 3.2.4 RC1 and 4.0 seem to be available for download. Should I be able to use the config files from them?

                Regards,
                Avi

                • 5. Re: HTTP session clustering in jboss 3.2.1 causing memory pr
                  Bela Ban Master

                  No, not yet.

                  I have just completed converting JGroups' XML files, so you can now use them inside your cluster-service.xml. This requires the latest JGroups, and I should have a new version in jboss-head and jboss-3.2 up and running by Wednesday.
                  This will then be part of 3.2.4RC2.
                  Bela

                  • 6. Re: HTTP session clustering in jboss 3.2.1 causing memory pr
                    _avi_ Newbie

                    Hi Bela,
                    Are you through with the updates to 3.2.4 RC2?
                    I checked from sourceforge again and 324 RC1 seems to be the last update and no other 324 updates since 2004-03-11 21:00.
                    Regards,
                    Avi