Seam clustering problems

[pbrewer_uk]

I've got my seam 2.2.0 app to deploy using apache, mod_cluster and two jboss 5.1.0 instances running on my development machine. But I still have a few outstanding issues that I hope someone can help me with or point me in the right direction. Any help at all is much appreciated.

Issue 1 - Facelets

Facelets flushes the component when one server fails over to another server that has not yet loaded a component. This causes the component to not render. I don't know how to fix or workaround this problem. (Same issue is mentioned here - Old nabble link - point 1 of the second issue). Log extract:

17:09:28,755 WARN  [SimpleConnectionManager] A problem has been detected with the connection to remote client 5c4o123-agoqg1-gcoxicsf-1-gcoy253z-be, j
msClientID=null. It is possible the client has exited without closing its connection(s) or the network has failed. All associated connection resources
 will be cleaned up.
17:09:29,992 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,992 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,993 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,993 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,993 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,993 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,994 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,994 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,994 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26
17:09:29,994 INFO  [facelet] Facelet[/WEB-INF/includes/noScript.xhtml] was modified @ 17:09:29, flushing component applied @ 17:00:26

Issue 2 - @Logger

@Logger
private transient Log log ;

The log variable is null after fail-over occurs. I assumed that the log would be injected by the BijectionInterceptor, but as you can see from the log below, the bijection interceptor is being called but log is not populated. So is the only correct way of doing this to declare a final static variable, as below:

private static final Log LOG = Logging.getLog(MemberWorkflowSelector.class);

Log extract:

17:10:42,987 SEVERE [viewhandler] Error Rendering View[/viewMember.xhtml]
javax.faces.FacesException: javax.el.ELException: /WEB-INF/includes/myStack.xhtml @15,72 rendered="#{not empty workflowSelector.memberTaskList}": Error reading 'memberTaskList' on type uk.co.iblocks.midas.web.MemberWorkflowSelector_$$_javassist_seam_9
        at javax.faces.component.UIComponentBase.isRendered(UIComponentBase.java:393)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxComponent(AjaxChildrenRenderer.java:111)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxChildren(AjaxChildrenRenderer.java:83)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxComponent(AjaxChildrenRenderer.java:157)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxChildren(AjaxChildrenRenderer.java:83)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxComponent(AjaxChildrenRenderer.java:157)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxChildren(AjaxChildrenRenderer.java:83)
        at org.ajax4jsf.renderkit.AjaxChildrenRenderer.encodeAjaxComponent(AjaxChildrenRenderer.java:157)
        at org.ajax4jsf.renderkit.AjaxContainerRenderer.encodeAjax(AjaxContainerRenderer.java:123)
        at org.ajax4jsf.component.AjaxViewRoot.encodeAjax(AjaxViewRoot.java:677)
        at org.ajax4jsf.component.AjaxViewRoot.encodeChildren(AjaxViewRoot.java:548)
        at javax.faces.component.UIComponent.encodeAll(UIComponent.java:930)
        at com.sun.facelets.FaceletViewHandler.renderView(FaceletViewHandler.java:592)
        at org.ajax4jsf.application.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:100)
        at org.ajax4jsf.application.AjaxViewHandler.renderView(AjaxViewHandler.java:176)
        at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:110)
        at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100)
        at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
        at javax.faces.webapp.FacesServlet.service(FacesServlet.java:266)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
        at org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:90)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at uk.co.iblocks.iflow.web.HttpSessionChecker$HttpSessionCheckerFilter.doFilter(HttpSessionChecker.java:156)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
        at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:178)
        at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290)
        at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:368)
        at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:495)
        at org.jboss.seam.web.Ajax4jsfFilter.doFilter(Ajax4jsfFilter.java:56)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:60)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:393)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
        at org.jboss.web.tomcat.service.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:135)
        at org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:94)
        at org.jboss.web.tomcat.service.session.JvmRouteValve.invoke(JvmRouteValve.java:88)
        at org.jboss.web.tomcat.service.session.LockingValve.invoke(LockingValve.java:62)
        at org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.event(CatalinaContext.java:285)
        at org.jboss.modcluster.catalina.CatalinaContext$RequestListenerValve.invoke(CatalinaContext.java:261)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Thread.java:619)
Caused by: javax.el.ELException: /WEB-INF/includes/myStack.xhtml @15,72 rendered="#{not empty workflowSelector.memberTaskList}": Error reading 'memberTaskList' on type uk.co.iblocks.midas.web.MemberWorkflowSelector_$$_javassist_seam_9
        at com.sun.facelets.el.TagValueExpression.getValue(TagValueExpression.java:76)
        at javax.faces.component.UIComponentBase.isRendered(UIComponentBase.java:390)
        ... 71 more
Caused by: java.lang.NullPointerException
        at uk.co.iblocks.midas.web.MemberWorkflowSelector.refreshMemberTaskList(MemberWorkflowSelector.java:139)
        at uk.co.iblocks.midas.web.MemberWorkflowSelector.getMemberTaskList(MemberWorkflowSelector.java:146)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.jboss.seam.util.Reflections.invoke(Reflections.java:22)
        at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56)
        at org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:77)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
        at org.jboss.seam.persistence.ManagedEntityInterceptor.aroundInvoke(ManagedEntityInterceptor.java:48)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
        at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
        at org.jboss.seam.core.ConversationInterceptor.aroundInvoke(ConversationInterceptor.java:65)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
        at org.jboss.seam.transaction.TransactionInterceptor$1.work(TransactionInterceptor.java:97)
        at org.jboss.seam.util.Work.workInTransaction(Work.java:47)
        at org.jboss.seam.transaction.TransactionInterceptor.aroundInvoke(TransactionInterceptor.java:91)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
        at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44)
        at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
        at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)
        at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185)
        at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103)
        at uk.co.iblocks.midas.web.MemberWorkflowSelector_$$_javassist_seam_9.getMemberTaskList(MemberWorkflowSelector_$$_javassist_seam_9.java)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at javax.el.BeanELResolver.getValue(BeanELResolver.java:62)
        at javax.el.CompositeELResolver.getValue(CompositeELResolver.java:54)
        at com.sun.faces.el.FacesCompositeELResolver.getValue(FacesCompositeELResolver.java:72)
        at org.jboss.el.parser.AstPropertySuffix.getValue(AstPropertySuffix.java:53)
        at org.jboss.el.parser.AstValue.getValue(AstValue.java:67)
        at org.jboss.el.parser.AstEmpty.getValue(AstEmpty.java:29)
        at org.jboss.el.parser.AstNot.getValue(AstNot.java:26)
        at org.jboss.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:186)
        at com.sun.facelets.el.TagValueExpression.getValue(TagValueExpression.java:71)
        ... 72 more

Issue 3 - JBoss AS clustering

JBPAPP-4195 - Apparently fixed, but no idea how to get the fixes. Currently using the workaround of @BypassInterceptors on hashCode and toString methods of all components that use bijection.

[kapitanpetko]

Peter Brewer wrote on Aug 10, 2010 13:10:


Issue 1 - Facelets
Facelets flushes the component when one server fails over to another server that has not yet loaded a component. This causes the component to not render. I don't know how to fix or workaround this problem.

Never seen this one. Turn off debug mode?

Issue 2 - @Logger

@Logger
private transient Log log ;

The log variable is null after fail-over occurs. I assumed that the log would be injected by the BijectionInterceptor, but as you can see from the log below, the bijection interceptor is being called but log is not populated. So is the only correct way of doing this to declare a final static variable, as below:

Never seen this one either, but the workaround is simple -- don't inject the Log, use Logger.getInstance() or whatever.

Issue 3 - JBoss AS clustering
JBPAPP-4195 - Apparently fixed, but no idea how to get the fixes. Currently using the workaround of @BypassInterceptors on hashCode and toString methods of all components that use bijection.

The fix is in the AS code, so you have to get the newest snapshot version or build from source. It's EAP 5.1, so it might not be in the community version (yet?).

Not relevant, but let me say this again out loud (again...): formatting for this forum SUCKS BIGTIME!

[pbrewer_uk]

Hi Nikolay, thank you for the quick response - this is much appreciated.

Issue 1 - Facelets

Never seen this one. Turn off debug mode?

I think debug mode is already off as my web.xml has

  <context-param>
    <param-name>facelets.DEVELOPMENT</param-name>
    <param-value>false</param-value>
  </context-param>

and my components.xml has

<core:init debug="false" jndi-pattern="midas/#{ejbName}/local" distributable="true">

So I'm still stumped on this one.

Issue 2 - @Logger

don't inject the Log, use Logger.getInstance() or whatever

I agree that will certainly work. I guess I was kind of curious as to how the log was injected if not via the BijectionInterceptor. Since yesterday, I have done a little more digging into the LogImpl class and it appears that this class is fully serializable - so in theory removing the transient keyword should work (I'll test it in a moment):

@Logger
private Log log ;

Issue 3 - JBoss AS Clustering

The fix is in the AS code, so you have to get the newest snapshot version or build from source. It's EAP 5.1, so it might not be in the community version (yet?).

I'm really only interested in getting the jboss-ha-server-api-1.1.3.GA jars (and dependencies) so that I can modify my copy of JBoss AS 5.1.0.GA but I've no idea where to get a snapshot or the source from. Anyone know the url for either of these?

Thanks again for the hints and tips!

[pbrewer_uk]

Out of curiousity, does anyone actually use seam in a clustered environment?

Issue 1

Facelets is still broken on failover - anyone got any suggestions?

Issue 2

Solved - one working way of obtaining a logger is to use the following code and let seam's LogImpl take of the rest.

@Logger
private Log log;

Issue 3

The hashCode method is still being called outside of seam's scope, despite building and deploying the jboss-ha-server-api-1.1.3.Final jar (I've made this available for download) or you can build from SVN

[kapitanpetko]

Peter Brewer wrote on Aug 12, 2010 06:10:


Out of curiousity, does anyone actually use seam in a clustered environment?

We do, but not with 5.1 (yet).

You might want to ask on the JBoss AS forum, since your remaining issues are AS issues, rather than Seam ones.

HTH

[pbrewer_uk]

Hi Nikolay,

Thanks for your reply, is there any technical reason why you haven't moved to JBoss 5.1 yet?

I'm suprised you haven't encountered the Facelets issue when clustering on JBoss 4, as I'm pretty sure its independent of the AS (and its version). To reproduce, you just need to failover to a page that hasn't been accessed on the second server.

Steps to reproduce:

1. Start node1 and node2


2. Access home.seam on node1


3. Shutdown node1


4. Click a link on home.seam that reRenders a portion of the page. (This request will be served by node2 and you should see the flushing components messages on the AS console).

The key is that the second server has not yet accessed a particular facelet that the first server has already loaded. When the page is restored on node2, Facelets checks the Applied Token timestamp (from its serialized state in the cluster) and it sees that it (node2) has created the same facelet more recently. So it thinks the Facelet has been updated and performs a flush.

The process of flushing the facelet causes the page to not refresh properly (certainly for ajax requests, maybe for full page requests too).

So there are two problems:

• The ApplyToken needs to take account of JVM change over (maybe using an uuid? Or don't use the create time to check for changes e.g. MD5 hash of facelet file?)


• Flushing should not cause the page to fail to reRender

Any hints or comments gratefully recieved.

Thanks, Pete.

[pbrewer_uk]

Solutions to clustering issues

This is a summary of the issues I've encountered while clustering Seam 2.2.0.GA on JBoss 5.1.0.GA. I hope it is of help to others...

Issue 1 - Facelets failover error

In web.xml, set the refresh to -1 as below (setting the development flag to false is not enough to prevent refresh checking).

<context-param>
  <param-name>facelets.REFRESH_PERIOD</param-name>
  <param-value>-1</param-value><!--  set to 2 in dev mode -->
</context-param>

Issue 2 - @Logger variable is null

Do not mark @Logger variables as transient, instead just define them as below:

@Logger
private Log log ;

Issue 3 - JBoss session replication breaks

@In(required=true) causes replication errors. See JBPAPP-4195 - Workaround is to override the toString() and hashCode() methods in seam components and annotate them with @BypassInterceptors

Issue 4 - <core:init distributable=true ...> breaks nested conversations

JBSEAM-4375 - Use the patch provided