6 Replies Latest reply on Sep 28, 2009 3:35 AM by jmpdesign

    Failover trouble mod_jk

    jmpdesign Newbie

      Hi everybody,

      My environment is the following :

      - Two machines Node1 and Node2 in a JBoss partition.
      - On each node I have the same web application marked as distributable.

      When the 2 nodes are alive, the balancing and sticky session work fine.

      When I turn off one node (for example Node2) I have http 503 errors (service unavailable) during 10-15 seconds because JK always address request to this node (Node2).
      After this delay JK change its redirection and address requests to Node1 and everything works.
      If I take a look on JBoss Node2 during shutting down process I see this error:

      java.lang.RuntimeException: DistributedCacheManager is null.
      at org.jboss.web.tomcat.service.session.ClusteredSession.establishDistri
      butedCacheManager(ClusteredSession.java:1773)


      I have checked jvmRoute parameters etc ..

      Is anybody has an idea why I have this trouble?

      Thanks,

      Regards.

        • 1. Re: Failover trouble mod_jk
          jmpdesign Newbie


          I am testing this features with JBoss 5.1.0 GA

          • 2. Re: Failover trouble mod_jk
            jmpdesign Newbie


            To be more precise, if I unplugged the network cable of my node2, failover processing works well and it is transparent for my client.

            It seems that shutdown.sh kill brutally JBoss, is it possible ?

            Thanks,

            Regards.

            • 3. Re: Failover trouble mod_jk
              Brian Stansberry Master

              This sounds like https://jira.jboss.org/jira/browse/JBAS-7123 . If you post the full stack trace of your "java.lang.RuntimeException: DistributedCacheManager is null.
              at org.jboss.web.tomcat.service.session.ClusteredSession.establishDistri
              butedCacheManager(ClusteredSession.java:1773) " I can confirm.

              • 4. Re: Failover trouble mod_jk
                jmpdesign Newbie

                Hi,

                Thank you very much Brian, actually it sounds like https://jira.jboss.org/jira/browse/JBAS-7123.
                Below this is my full stack trace.
                Have you got an idea when this issue will be fixed or is there a work around to shutdown properly JBoss 5.1.0GA instance in clustering environment?


                2009-09-25 09:33:22,527 ERROR [org.apache.catalina.connector.CoyoteAdapter] (ajp-CPO-ARC2%2F172.18.210.33-8009-2) Une exception ou une erreur s'est produite dans le conteneur durant le traitement de la requ�te
                java.lang.RuntimeException: DistributedCacheManager is null.
                 at org.jboss.web.tomcat.service.session.ClusteredSession.establishDistributedCacheManager(ClusteredSession.java:1773)
                 at org.jboss.web.tomcat.service.session.ClusteredSession.setManager(ClusteredSession.java:507)
                 at org.jboss.web.tomcat.service.session.ClusteredSession.<init>(ClusteredSession.java:399)
                 at org.jboss.web.tomcat.service.session.AttributeBasedClusteredSession.<init>(AttributeBasedClusteredSession.java:65)
                 at org.jboss.web.tomcat.service.session.JBossCacheManager.createEmptyClusteredSession(JBossCacheManager.java:1715)
                 at org.jboss.web.tomcat.service.session.JBossCacheManager.loadSession(JBossCacheManager.java:1805)
                 at org.jboss.web.tomcat.service.session.JBossCacheManager.findSession(JBossCacheManager.java:489)
                 at org.apache.catalina.connector.Request.doGetSession(Request.java:2320)
                 at org.apache.catalina.connector.Request.getSession(Request.java:2094)
                 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:132)
                 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
                 at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
                 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
                 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
                 at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
                 at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
                 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
                 at java.lang.Thread.run(Thread.java:619)



                Thanks,
                Regards.



                • 5. Re: Failover trouble mod_jk
                  Brian Stansberry Master

                  Thanks for the stack trace; yeah that's the JBAS-7123 condition.

                  It's already fixed in SVN and will be fixed in the 5.2 which should be out late this year. It will also be fixed in JBoss Enterprise Application Platform 5.

                  As to a workaround for AS 5.1.0, the only one I can see is to add this to your startup command:

                  -Dorg.apache.catalina.STRICT_SERVLET_COMPLIANCE=false

                  That will prevent the org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:132)
                  call that is leading to the problem.

                  That has a number of effects on the behavior of JBoss Web though, many of which I don't claim to understand. This doc gives a good overview of the effect of that flag:

                  http://tomcat.apache.org/tomcat-6.0-doc/config/systemprops.html

                  The "every request that is associated with a session will cause the session's last accessed time to be updated regardless of whether or not the request explicity accesses the session. (SRV.7.6) " bit on that page is the one specifically relevant to this issue.

                  • 6. Re: Failover trouble mod_jk
                    jmpdesign Newbie


                    Thank you Brian for your explanation.
                    So I think I am waiting 5.2 released.

                    Regards.