1 Reply Latest reply on Aug 23, 2002 10:45 AM by romalley

    Tomcat ajp Connector Sticky Load Balance Policy thwarted

    romalley

      The Tomcat isapi_redirect.dll uses a Load Balance Policy that looks for a token in the session cookie. That token must be created by Tomcat when configured to do so. Here's a message I posted to the bottom of another thread but I am re-posting as a separate topic in hopes that Sacha will see it here:
      ----------------------------------

      Sacha

      1) For the Jakarta Tomcat redirector dll plugin, there is a simple piece of round robin load balancing that Tomcat must do. I think this redirector does it's round robin work by looking at a portion of the session cookie. To control this piece of the cookie, there is field called 'jvmRoute' that Tomcat places at the end of the session ID. The Tomcat configuration approach is to put the jvmRoute attribute into the Engine entry in the tomcat4-service.xml file like this:





      Then, for the load balance redirector within the worker.properties file, you set up a worker with the same name like this:

      worker.serverOne.port=8009
      worker.serverOne.host=localhost
      worker.serverOne.type=ajp13
      worker.serverOne.cachesize=20
      worker.serverOne.lbfactor=2

      Then, you can see in my example the session.getId() method returns a value like this:

      7F9086AFF3E0E6DDE974452847CBB3BB.serverOne

      Sticky load balancing works -- all requests for this session are forwarded to the worker called serverOne.

      But when I use the jbossha-httpsession.sar, the session does NOT have the jvmRoute in it, instead the session looks like this with clustering turned on:

      dc9A8mxS88aHU8Z8C3sXfA**

      So, the IIS redirector does not know what to do because it does not find the .serverOne at the end of the session. It no longer does sticky load balancing.


      2) I was somewhat incorrect in the way I reported it's behavior. What it actually does, is to round robin every request when clustering is turned on. I think this is not so bad as long as our application doesn't modify the session for every request. I might leave it alone and leave the session replication so it replicates immediately (the default I think). We'll load test and see if there is a performance hit.

      3) Ideally, the jbossha-httpsession would continue to append the 'jvmRoute' of the 1st server hit to the end of the session id and I think sticky load balancing for the Tomcat redirector would be fixed.

      Bob O'Malley

        • 1. Re: Tomcat ajp Connector Sticky Load Balance Policy thwarted
          romalley

          And here's another message I posted in that other thread. Basically, if I can't use sticky load balancing then I get a long error trace. Could I be doing something wrong which causes the error trace?


          ------------------------------------------

          Sacha,

          I posted a detailed message but it did not make it to the forum (yet -- I still hope it does).

          In summary, With Tomcat, there is a 'relationship' when using sticky load balancing between the Tomcat isapi_redirect.dll plugin, and the attribute called 'jvmRoute' set up in the Tomcat server configuration.

          Call it 'proprietary load balancing'. You set up Tomcat with an Engine that has the 'jvmRoute' attribute in it (Jboss treats this as part of the 'config' Element for Tomcat). This jvmRoute name must match exactly the the name of a 'worker' you set up in your redirector.dll configuration.

          When you do, each session id has the jvmRoute appended to it like this, where serverOne is my jvmRoute:

          D1100F27231B789676CE555AF590DFED.serverOne

          So, the isapi_redirect.dll load balancer just looks at the end of the sessionid and decides where to send the request. Simple.


          I think this has been lost in the jbossha-httpsession.sar implementation. It might be Okay not to use sticky load balancing except that now I routinely get long exception traces in my console after a session expires. I think the server that is sharing the session does not know that the session has been removed. Here is the exception trace:

          13:14:59,082 INFO [EmbeddedCatalinaServiceSX] javax.ejb.EJBException: Exception in setHttpSession: javax.ejb.FinderException: IlQ2aHC56M6P-F+KkFWOSA** does not exist
          13:14:59,082 ERROR [LogInterceptor] TransactionRolledbackException, causedBy:
          java.lang.NullPointerException
          at java.io.ByteArrayInputStream.(ByteArrayInputStream.java:90)
          at org.jboss.ejb.plugins.CMPClusteredInMemoryPersistenceManager.loadEntity(CMPClusteredInMemoryPersistenceManager.java:352)
          at org.jboss.ejb.plugins.CMPPersistenceManager.loadEntity(CMPPersistenceManager.java:410)
          at org.jboss.ejb.plugins.EntitySynchronizationInterceptor.invoke(EntitySynchronizationInterceptor.java:262)
          at org.jboss.ejb.plugins.EntityInstanceInterceptor.invoke(EntityInstanceInterceptor.java:152)
          at org.jboss.ejb.plugins.EntityLockInterceptor.invoke(EntityLockInterceptor.java:107)
          at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:96)
          at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:228)
          at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:61)
          at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:183)
          at org.jboss.ejb.EntityContainer.invoke(EntityContainer.java:493)
          at org.jboss.ejb.Container.invoke(Container.java:705)
          at org.jboss.ejb.EntityContainer.invoke(EntityContainer.java:1055)
          at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:491)
          at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:98)
          at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:102)
          at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:73)
          at org.jboss.proxy.ejb.HomeInterceptor.invoke(HomeInterceptor.java:189)
          at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:76)
          at $Proxy13.remove(Unknown Source)
          at org.jboss.ha.httpsession.server.ClusteredHTTPSessionService.removeHttpSession(ClusteredHTTPSessionService.java:153)
          at java.lang.reflect.Method.invoke(Native Method)
          at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
          at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:491)
          at org.jboss.util.jmx.MBeanProxy.invoke(MBeanProxy.java:174)
          at $Proxy26.removeHttpSession(Unknown Source)
          at org.jboss.web.catalina.session.ClusterManager.removeSession(ClusterManager.java:401)
          at org.jboss.web.catalina.session.ClusterManager.remove(ClusterManager.java:201)
          at org.jboss.web.catalina.session.ClusteredSession.expire(ClusteredSession.java:565)
          at org.jboss.web.catalina.session.ClusteredSessionCleanup.processExpires(ClusteredSessionCleanup.java:102)
          at org.jboss.web.catalina.session.ClusteredSessionCleanup.run(ClusteredSessionCleanup.java:167)
          at java.lang.Thread.run(Thread.java:484)
          13:14:59,082 INFO [EmbeddedCatalinaServiceSX] Removing a session out of the clustered store failedjavax.ejb.EJBException: Exception in removeHttpSession: javax.transaction.TransactionRolledbackException: Load failed; CausedByException is: null; nested exception is:
          javax.ejb.EJBException: Load failed; CausedByException is:
          null


          Bob O'Malley