Failover trouble mod_jk

[jmpdesign]

Hi everybody,

My environment is the following :

- Two machines Node1 and Node2 in a JBoss partition.
- On each node I have the same web application marked as distributable.

When the 2 nodes are alive, the balancing and sticky session work fine.

When I turn off one node (for example Node2) I have http 503 errors (service unavailable) during 10-15 seconds because JK always address request to this node (Node2).
After this delay JK change its redirection and address requests to Node1 and everything works.
If I take a look on JBoss Node2 during shutting down process I see this error:

java.lang.RuntimeException: DistributedCacheManager is null.
at org.jboss.web.tomcat.service.session.ClusteredSession.establishDistri
butedCacheManager(ClusteredSession.java:1773)


I have checked jvmRoute parameters etc ..

Is anybody has an idea why I have this trouble?

Thanks,

Regards.

[jmpdesign]

I am testing this features with JBoss 5.1.0 GA

[jmpdesign]

To be more precise, if I unplugged the network cable of my node2, failover processing works well and it is transparent for my client.

It seems that shutdown.sh kill brutally JBoss, is it possible ?

Thanks,

Regards.

[brian.stansberry]

This sounds like https://jira.jboss.org/jira/browse/JBAS-7123 . If you post the full stack trace of your "java.lang.RuntimeException: DistributedCacheManager is null.
at org.jboss.web.tomcat.service.session.ClusteredSession.establishDistri
butedCacheManager(ClusteredSession.java:1773) " I can confirm.

[jmpdesign]

Hi,

Thank you very much Brian, actually it sounds like https://jira.jboss.org/jira/browse/JBAS-7123.
Below this is my full stack trace.
Have you got an idea when this issue will be fixed or is there a work around to shutdown properly JBoss 5.1.0GA instance in clustering environment?

2009-09-25 09:33:22,527 ERROR [org.apache.catalina.connector.CoyoteAdapter] (ajp-CPO-ARC2%2F172.18.210.33-8009-2) Une exception ou une erreur s'est produite dans le conteneur durant le traitement de la requ�te
java.lang.RuntimeException: DistributedCacheManager is null.
 at org.jboss.web.tomcat.service.session.ClusteredSession.establishDistributedCacheManager(ClusteredSession.java:1773)
 at org.jboss.web.tomcat.service.session.ClusteredSession.setManager(ClusteredSession.java:507)
 at org.jboss.web.tomcat.service.session.ClusteredSession.<init>(ClusteredSession.java:399)
 at org.jboss.web.tomcat.service.session.AttributeBasedClusteredSession.<init>(AttributeBasedClusteredSession.java:65)
 at org.jboss.web.tomcat.service.session.JBossCacheManager.createEmptyClusteredSession(JBossCacheManager.java:1715)
 at org.jboss.web.tomcat.service.session.JBossCacheManager.loadSession(JBossCacheManager.java:1805)
 at org.jboss.web.tomcat.service.session.JBossCacheManager.findSession(JBossCacheManager.java:489)
 at org.apache.catalina.connector.Request.doGetSession(Request.java:2320)
 at org.apache.catalina.connector.Request.getSession(Request.java:2094)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:132)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
 at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
 at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
 at java.lang.Thread.run(Thread.java:619)

Thanks,
Regards.

[brian.stansberry]

Thanks for the stack trace; yeah that's the JBAS-7123 condition.

It's already fixed in SVN and will be fixed in the 5.2 which should be out late this year. It will also be fixed in JBoss Enterprise Application Platform 5.

As to a workaround for AS 5.1.0, the only one I can see is to add this to your startup command:

-Dorg.apache.catalina.STRICT_SERVLET_COMPLIANCE=false

That will prevent the org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:132)
call that is leading to the problem.

That has a number of effects on the behavior of JBoss Web though, many of which I don't claim to understand. This doc gives a good overview of the effect of that flag:

http://tomcat.apache.org/tomcat-6.0-doc/config/systemprops.html

The "every request that is associated with a session will cause the session's last accessed time to be updated regardless of whether or not the request explicity accesses the session. (SRV.7.6) " bit on that page is the one specifically relevant to this issue.

[jmpdesign]

Thank you Brian for your explanation.
So I think I am waiting 5.2 released.

Regards.