11 Replies Latest reply on Mar 5, 2006 10:34 PM by Susitha Bandara

    Http session replication not happening in clustered environm

    vivek oswal Newbie

      Hi all,
      We are setting up a cluster of servers, the specifications are as follows:

      OS: Windows 2000 Professional
      JDK: jdk1.3.1_04
      JBoss version: jboss-3.2.1_tomcat-4.1.24 (on 2 nodes viz server1 and server2.)
      Apache: 2.0.45(installed on server1)

      We have included

      <distributable/>
      element in the web.xml of our application
      similarly we have included entries for all the entity beans and session beans in jboss.xml's of all the components.

      We have added the jvmRoute attribute in the {JBOSS_HOME}\server\all\deploy\jbossweb-tomcat.sar\META-INF\jboss-service.xml file
      of both the nodes
      server1
      <Engine jvmRoute="server1" name="MainEngine" defaultHost="localhost">
      
      server2 <Engine jvmRoute="server2" name="MainEngine" defaultHost="localhost">

      We have included the following tag in the {JBOSS_HOME}\server\all\deploy\wsc.ear\wsc.war\jboss-web.xml file of both the nodes.
      <replication-config>
      <replication-trigger>SET_AND_GET</replication-trigger>
      <replication-type>SYNC</replication-type>
      </replication-config>

      APACHE mod_jk2 specific Configuration is as follows:
      Following lines are added at the end of {apache_home}\conf\httpd.conf file
      # Include mod_jk's specific configuration file
      Include conf/mod-jk2.conf
      # Load mod_jk module
      LoadModule jk2_module modules/mod_jk2.so

      Following are the content of {apache_home}\conf\workers2.properties file.

      [shm]
      info=Scoreboard. Requried for reconfiguration and status with multiprocess servers.
      file=anon
      size=1048576

      [lb:lb]
      stickySession=1
      debug=1
      [channel.socket:server1:8009]
      port=8009
      host=server1
      tomcatId=server1
      [channel.socket:server2:8009]
      port=8009
      host=server2
      tomcatId=server2
      [ajp13:server1:8009]
      channel=channel.socket:server1:8009
      group=lb
      [ajp13:server2:8009]
      channel=channel.socket:server2:8009
      group=lb
      [uri:/*]
      group=lb

      [status:]
      info=Status worker, displays runtime information

      Following are the content of {apache_home}\conf\mod-jk2.conf file
      <IfModule mod_jk2.c>
      JkSet config.file C:/Program Files/Apache Group/Apache2/conf/workers2.properties
      </IfModule>
      After making the above configurations the load balancing behaviour was achieved but the session replication does'nt seem to be happening.
      We started both the nodes and observed that the cluster of 2 nodes is formed. Then we made a request which went to server1. We shut down the jboss-tomcat at server1 and made a request with the same client, we expected that server2 would take the charge (session will be replicated on server2) but it does not happen and we get a session expired message.

      We would appreciate any help and suggestions on this.

      Thanks and regards.
      popnet.



        • 1. Re: Http session replication not happening in clustered envi
          vivek oswal Newbie

          Putting some more info in the above post...there are times when we see the following error in the JBOSS_HOME/server/all/log/server.log file:

          2004-08-03 16:55:25,328 DEBUG [org.jboss.ha.httpsession.server.ClusteredHTTPSessionService] getHttpSession called for session: cIROaQ-rCKQpZmYvGPLR+w**.ankit
          2004-08-03 16:55:25,328 DEBUG [org.jboss.web.catalina.EmbeddedCatalinaService41] Loading a session out of the clustered store failed
          javax.ejb.EJBException: Exception in setHttpSession: javax.ejb.FinderException: cIROaQ-rCKQpZmYvGPLR+w**.ankit does not exist
          at org.jboss.ha.httpsession.server.ClusteredHTTPSessionService.getHttpSession(ClusteredHTTPSessionService.java:141)
          at java.lang.reflect.Method.invoke(Native Method)
          at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
          at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:549)
          at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:177)
          at $Proxy211.getHttpSession(Unknown Source)
          at org.jboss.web.catalina.session.ClusterManager.loadSession(ClusterManager.java:582)
          at org.jboss.web.catalina.session.ClusterManager.findSession(ClusterManager.java:260)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:173)
          at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
          at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
          at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
          at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
          at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:509)
          at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
          at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
          at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
          at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
          at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
          at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
          at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223)
          at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:261)
          at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:360)
          at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:604)
          at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:562)
          at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:679)
          at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
          at java.lang.Thread.run(Thread.java:479)


          Does this indicate something?

          Regards,
          Vivek

          • 2. Re: Http session replication not happening in clustered envi
            nikita tovstoles Newbie

            your session id string appears garbled. i'm currently having the same problem and i wonder whether that contributes to broken session replication. Seems to me, the session id is garbled because jboss (tomcat?) uses incorrect locale. I'm pretty confident that once you get the session id to appear correctly, your clustering problems will resolve. Let me know if you figure out how to fix that, i will do the same if i get there first

            cheers,

            -nikita

            • 3. Re: Http session replication not happening in clustered envi
              Matt Vincent Newbie

              We're getting a similar Exception in setHttpSession for JBoss 3.2.5.

              I don't think that the garbled session ID string has much to do with it. The session ID should be "garbled". This is from ClusteredHTTPSessionService.java

              This code was written by Kevin Kelley <kelley@ruralnet.net>
              and adapted by Thomas Peuss <jboss@peuss.de>
              @param data The bytes you want to encode
              @return the encoded String
              */
              protected String encode(byte[] data)
              {
              char[] out = new char[((data.length + 2) / 3) * 4];
              char[] alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-*".toCharArray();
              //
              // 3 bytes encode to 4 chars. Output is always an even
              // multiple of 4 characters.
              //
              for (int i=0, index=0; i<data.length; i+=3, index+=4) {
              boolean quad = false;
              boolean trip = false;
              int val = (0xFF & (int) data);
              val <<= 8;
              if ((i+1) < data.length) {
              val |= (0xFF & (int) data[i+1]);
              trip = true;
              }
              val <<= 8;
              if ((i+2) < data.length) {
              val |= (0xFF & (int) data[i+2]);
              quad = true;
              }
              out[index+3] = alphabet[(quad? (val & 0x3F): 64)];
              val >>= 6;
              out[index+2] = alphabet[(trip? (val & 0x3F): 64)];
              val >>= 6;
              out[index+1] = alphabet[val & 0x3F];
              val >>= 6;
              out[index+0] = alphabet[val & 0x3F];
              }
              return new String(out);
              }

              • 4. Re: Http session replication not happening in clustered envi
                marina const Newbie

                Hi,

                I also have this error and the session is not working. Could someone advise ? Thanks.

                org.jboss.web.tomcat.tc5.TomcatDeployer Looking for session with id=07by25pCarQpjKT-hUm+sw**.worker1
                org.jboss.web.tomcat.tc5.TomcatDeployer Found

                org.jboss.web.tomcat.tc5.TomcatDeployer Replicating session with id 07by25pCarQpjKT-hUm+sw**.worker1

                org.jboss.ha.httpsession.server.ClusteredHTTPSessionService setHttpSession called for session: 07by25pCarQpjKT-hUm+sw**.worker1

                org.jboss.web.tomcat.tc5.session.ClusteredSession Are current session 07by25pCarQpjKT-hUm+sw**.worker1 attributes modified? : true

                org.jboss.ha.framework.server.HAPartitionImpl dests=[cphzt20:10104 (additional data: 17 bytes)], method_call=DistributedState._set(CMPClusteredInMemoryPersistenceManager-ClusteredHTTPSession, 07by25pCarQpjKT-hUm+sw**.worker1, [B@1cfd2c6), mode=2, timeout=60000

                org.jboss.ha.framework.server.HAPartitionImpl real_dests=[cphzt20:10104 (additional data: 17 bytes)]

                org.jboss.ha.framework.server.HAPartitionImpl responses: [sender=cphzt20:10104 (additional data: 17 bytes), retval=null, received=true, suspected=false]

                org.jboss.web.tomcat.tc5.TomcatDeployer Looking for session with id=sm0QM5I9aaSDODG0dp1XQA**.worker2

                org.jboss.web.tomcat.tc5.TomcatDeployer JvmRoute id is :sm0QM5I9aaSDODG0dp1XQA**.worker1

                org.jboss.ha.httpsession.server.ClusteredHTTPSessionService getHttpSession called for session: sm0QM5I9aaSDODG0dp1XQA**.worker2

                org.jboss.web.tomcat.tc5.TomcatDeployer Loading a session out of the clustered store failed
                javax.ejb.EJBException: Exception in setHttpSession: ; CausedByException is:
                sm0QM5I9aaSDODG0dp1XQA**.worker2 does not exist
                at org.jboss.ha.httpsession.server.ClusteredHTTPSessionService.getHttpSession(ClusteredHTTPSessionService.java:145)
                at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.i

                • 5. Re: Http session replication not happening in clustered envi
                  Ben Wang Master

                  My advice is to upgrade to 3.2.7 and up. Reason being that we have a new http session replication layer that is much more maintainable. It has been out for almost 1.5 years.

                  -Ben

                  • 6. Re: Http session replication not happening in clustered envi
                    marina const Newbie

                    Hi Ben,

                    We upgrade to 3.2.7 as you told me. The problem is now I cannot get the sticky session work anymore and I give up of the cluster because the site is using sessions. Is there any documentation I can follow specifically on how to make sticky session works on 3.2.7 ?

                    Thanks.

                    • 7. Re: Http session replication not happening in clustered envi
                      Ben Wang Master

                      Follow this Wiki trail should get you what you need on sticky session. Let me know if there still is a problem.

                      -Ben

                      • 9. Re: Http session replication not happening in clustered envi
                        marina const Newbie


                        I have found it, the first message. Thank you!

                        • 10. Re: Http session replication not happening in clustered envi
                          marina const Newbie

                          Hi Ben,

                          I want to use sticky session, not session replication on jboss 3.2.7.

                          I was able to configure well jboss 3.2.5 but now something goes wrong.

                          Now the server keeps the session as far as I are on the same page, so I can refresh many times and the session id and server are the same.
                          I have made a test.jsp.

                          Then navigate into the site and check the session id and the server from time to time. I can see the request is round robin between the servers. I do not understand why this is happening when navigate and not when I am on the same page and refresh again an again.

                          What I have done is I have changed jboss-service.xml as I found out here: http://wiki.jboss.org/wiki/Wiki.jsp?page=ClusteringFAQ.

                          "Since 3.2.6, HTTP session replication is based on JBossCache and more precisely the TomcatClusteringCacheservice.
                          In 3.2.6, 3.2.7, 4.0.1 and 4.0.1SP1, the dependency on this service is missing in Tomcat's jboss-service.xml.

                          To fix it, please edit /all/jbossweb-tomcat5x.sar/META-INF/jboss-service.xml and enable the <depends optional-attribute-name="CacheName<http://wiki.jboss.org/wiki/Edit.jsp?page=CacheName>"> element, update the MBean ObjectName <http://wiki.jboss.org/wiki/Wiki.jsp?page=ObjectName> as well as the element itself.

                          The resulting code should look as follows:
                          <!--
                          Configuration for HTTP Session Clustering using JBossCache
                          -->
                          jboss.cache:service=TomcatClusteringCache
                          instead of
                          <!--
                          Configuration for HTTP Session Clustering using JBossCache
                          -->
                          <!--
                          <depends optional-attribute-name="CacheName">jboss.cache:service=TreeCache
                          -->
                          "

                          • 11. Re: Http session replication not happening in clustered envi
                            Susitha Bandara Newbie

                            How did you check the load balancing? What were you using in your application (stateless session bean, Stateful etc.) Let me know in detail.
                            -Prabath