Clustered WebBeans| JBoss.org Content Archive (Read Only)

1. Re: Clustered WebBeans

meetoblivion Jul 1, 2009 2:53 PM (in response to meetoblivion)

So it looks like off the bat, webbeans doesn't serialize something

javax.servlet.ServletException: Servlet execution threw an exception
     org.jboss.webbeans.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:113)
     org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
root cause

java.lang.reflect.InvocationTargetException
     sun.reflect.GeneratedMethodAccessor539.invoke(Unknown Source)
     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     java.lang.reflect.Method.invoke(Method.java:597)
     org.jboss.webbeans.bean.proxy.ClientProxyMethodHandler.invoke(ClientProxyMethodHandler.java:97)
     com.coat.web.images.server.ejb.ImagePathBuilder_$$_javassist_129.parseErezRequestInfo(ImagePathBuilder_$$_javassist_129.java)
     com.coat.web.images.server.web.ImageProxyServlet.service(Unknown Source)
     javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
     org.jboss.webbeans.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:113)
     org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
root cause

java.lang.IllegalArgumentException: setAttribute: Non-serializable attribute
     org.jboss.web.tomcat.service.session.ClusteredSession.setAttribute(ClusteredSession.java:856)
     org.apache.catalina.session.StandardSessionFacade.setAttribute(StandardSessionFacade.java:130)
     org.jboss.webbeans.servlet.HttpRequestSessionBeanStore.setAttribute(HttpRequestSessionBeanStore.java:90)
     org.jboss.webbeans.context.beanstore.AbstractAttributeBackedBeanStore.put(AbstractAttributeBackedBeanStore.java:133)
     org.jboss.webbeans.context.AbstractMapContext.get(AbstractMapContext.java:101)
     org.jboss.webbeans.bean.proxy.ClientProxyMethodHandler.getProxiedInstance(ClientProxyMethodHandler.java:117)
     org.jboss.webbeans.bean.proxy.ClientProxyMethodHandler.invoke(ClientProxyMethodHandler.java:96)
     com.coat.web.images.db.dao.IImagePropertyDAO_$$_javassist_130.findByPropertyValue(IImagePropertyDAO_$$_javassist_130.java)
     com.coat.web.images.server.ejb.ImagePathBuilder.findImageFromSku1(Unknown Source)
     com.coat.web.images.server.ejb.ImagePathBuilder.parseErezRequestInfo(Unknown Source)
     sun.reflect.GeneratedMethodAccessor539.invoke(Unknown Source)
     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     java.lang.reflect.Method.invoke(Method.java:597)
     org.jboss.webbeans.bean.proxy.ClientProxyMethodHandler.invoke(ClientProxyMethodHandler.java:97)
     com.coat.web.images.server.ejb.ImagePathBuilder_$$_javassist_129.parseErezRequestInfo(ImagePathBuilder_$$_javassist_129.java)
     com.coat.web.images.server.web.ImageProxyServlet.service(Unknown Source)
     javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
     org.jboss.webbeans.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:113)
     org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)

2. Re: Clustered WebBeans

pmuir Jul 1, 2009 4:40 PM (in response to meetoblivion)

John, we definitely need clustering support for the 1.0.0.GA release. I am intending to address this in the CR series, but any help you guys can provide would be excellent!

Either, report your progress here, or open a JIRA with changes needed. If you get stuck, it would help a lot if you could attach a patch to an issue which includes any changes you have worked out to get an example clustering...

3. Re: Clustered WebBeans

meetoblivion Jul 1, 2009 4:58 PM (in response to meetoblivion)

Pete,

So right now, we're having a hard time even getting jboss as 5.1 clustered in the first place :-(

The docs http://www.jboss.org/file-access/default/members/jbossas/freezone/docs/Clustering_Guide/5/html_single/index.html#clustering-http-state are out of date compared to the file names and paths found in 5.1. You wouldn't happen to know of where these might live now, for 5.1, would you? I just posted the same thing on the forums.

I will say this, it looks like clustered ejb's are not working at all. I only get the above exception when I try to cluster an ejb. Both servers are currently configured for cluster but it appears that the session state doesn't replicate. I'm not sure if this is a webbeans issue or not. It seems to work fine with clustered POJOs but I didn't configure any @Clustered or @CacheConfig on these objects.

4. Re: Clustered WebBeans

pmuir Jul 1, 2009 5:54 PM (in response to meetoblivion)

You'll need to attach a debugger and find out what it is that isn't serializaing properly.

https://svn.jboss.org/repos/jbossas/projects/docs/community/5/Clustering_Guide/en-US/Clustering_Guide_HTTP.xml is the most up to date docs that exist.

5. Re: Clustered WebBeans

meetoblivion Jul 1, 2009 6:17 PM (in response to meetoblivion)

I was able to get to that using the anonsvn. It's a bit hard to read, is there a rendered version of it somewhere?

6. Re: Clustered WebBeans

pmuir Jul 1, 2009 6:25 PM (in response to meetoblivion)

FYI

[17:20]  <bstansberry> pmuir: you are best off looking at http://www.jboss.org/community/wiki/ConfigurationChangesforClusteredWebApplicationsinAS5 and http://www.jboss.org/community/wiki/DistributableHttpSessionPassivation
[17:24]  <pmuir> bstansberry: thanks

7. Re: Clustered WebBeans

meetoblivion Jul 1, 2009 8:46 PM (in response to meetoblivion)

Pete,

We added the custom config as mentioned in the guides. probably missed something but this is what we get during load of the app. The custom cache config didn't work, got an unkown cache config exception. We changed to both

standard-session-cache Standard cache used for web sessions
field-granularity-session-cache Standard cache used for FIELD granularity web sessions

and it still didn't replicate.

Any ideas?

8. Re: Clustered WebBeans

meetoblivion Jul 1, 2009 9:45 PM (in response to meetoblivion)

So I just got something really weird.

15:27:09,913 ERROR [AttributeBasedJBossCacheService] IOException occurred unmarshalling value java.io.InvalidClassException: org.jboss.webbeans.conversation.ConversationTerminator at javassist.util.proxy.SerializedProxy.readResolve(SerializedProxy.java:63) at sun.reflect.GeneratedMethodAccessor358.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at java.io.ObjectStreamClass.invokeReadResolve(ObjectStreamClass.java:1061) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1762) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947) at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351) at org.jboss.ha.framework.server.SimpleCachableMarshalledValue.get(SimpleCachableMarshalledValue.java:94) at org.jboss.web.tomcat.service.session.distributedcache.impl.jbc.AbstractJBossCacheService.getUnMarshalledValue(AbstractJBossCacheService.java:660) at org.jboss.web.tomcat.service.session.distributedcache.impl.jbc.AttributeBasedJBossCacheService.getSessionAttributes(AttributeBasedJBossCacheService.java:166) at org.jboss.web.tomcat.service.session.distributedcache.impl.jbc.AbstractJBossCacheService.getDistributableSessionData(AbstractJBossCacheService.java:581) at org.jboss.web.tomcat.service.session.distributedcache.impl.jbc.AbstractJBossCacheService.getSessionData(AbstractJBossCacheService.java:364) at org.jboss.web.tomcat.service.session.JBossCacheManager.loadSession(JBossCacheManager.java:1832) at org.jboss.web.tomcat.service.session.JBossCacheManager.findSession(JBossCacheManager.java:489) at org.jboss.web.tomcat.service.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:124) at org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:94) at org.jboss.web.tomcat.service.session.LockingValve.invoke(LockingValve.java:62) at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436) at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619) 15:27:09,914 ERROR [CoyoteAdapter] An exception or error occurred in the container during the request processing java.lang.NullPointerException at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881) at java.util.concurrent.ConcurrentHashMap.putAll(ConcurrentHashMap.java:909) at org.jboss.web.tomcat.service.session.ClusteredSession.populateAttributes(ClusteredSession.java:1661) at org.jboss.web.tomcat.service.session.ClusteredSession.update(ClusteredSession.java:1120) at org.jboss.web.tomcat.service.session.JBossCacheManager.loadSession(JBossCacheManager.java:1835) at org.jboss.web.tomcat.service.session.JBossCacheManager.findSession(JBossCacheManager.java:489) at org.jboss.web.tomcat.service.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:124) at org.jboss.web.tomcat.service.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:94) at org.jboss.web.tomcat.service.session.LockingValve.invoke(LockingValve.java:62) at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126) at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436) at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447) at java.lang.Thread.run(Thread.java:619)

9. Re: Clustered WebBeans

pmuir Jul 2, 2009 1:51 AM (in response to meetoblivion)

Sorry John, I'm not a clustering expert. Take this to the JBoss clustering forum, and discuss there. Feel free to post a link here - and ping me if you need my input on WB specific stuff.

10. Re: Clustered WebBeans

meetoblivion Jul 2, 2009 2:51 AM (in response to meetoblivion)

Hi Pete

Unfortunately, the jboss forums are typically dead to me. I've actually posted requests for newer clustering docs in both the AS5 and Clustering forums, with no such luck in either. In general, they both seem to be a bit on the light side as far as people watching them go.

so just looking at the source of the cache service, this appears to be the broken cog on the AS side

   private Object getUnMarshalledValue(Object obj)
   {
      if (!(obj instanceof SimpleCachableMarshalledValue))
            return obj;
         
      // Swap in/out the tcl for this web app. Needed only for un marshalling.
      ClassLoader prevTCL = Thread.currentThread().getContextClassLoader();
      Thread.currentThread().setContextClassLoader(webAppClassLoader_);
      try
      {
         SimpleCachableMarshalledValue mv = (SimpleCachableMarshalledValue) obj;
         mv.setObjectStreamSource(SessionSerializationFactory.getObjectStreamSource());
         return mv.get();
      }
      catch (IOException e)
      {
         log_.error("IOException occurred unmarshalling value ", e);
         return null;
      }
      catch (ClassNotFoundException e)
      {
         log_.error("ClassNotFoundException occurred unmarshalling value ", e);
         return null;
      }
      finally
      {
         Thread.currentThread().setContextClassLoader(prevTCL);
      }
   }

where get is calling:

 84     public synchronized Serializable get() throws IOException, ClassNotFoundException
 85     {
 86        if (raw == null)
 87        {
 88           if (serializedForm != null)
 89           {
 90              ByteArrayInputStream bais = new ByteArrayInputStream(serializedForm);
 91              ObjectInput mvis = getObjectStreamSource().getObjectInput(bais);
 92              try
 93              {
 94                 raw =  (Serializable) mvis.readObject();
 95                 serializedForm = null;
 96              }
 97              finally
 98              {
 99                 mvis.close();
100              }
101           }         
102        }
103        return raw;
104     }

Which to me sounds weird, as ConversationTerminator isn't serializable. Perhaps, the implementation of it is, but the interface isn't so java's not quite sure how to handle it?

11. Re: Clustered WebBeans

pmuir Jul 2, 2009 12:09 PM (in response to meetoblivion)

Right, there is every chance something is broken in Web Beans, but serialization error reporting is bad, so perhaps it's not showing clearly.

You've probably run into https://jira.jboss.org/jira/browse/WBRI-140 - try the updated Javassist.

12. Re: Clustered WebBeans

meetoblivion Jul 2, 2009 4:22 PM (in response to meetoblivion)

Ok, that clears up part of this. Yes, I was in fact seeing WBRI-140. I upgraded to the javassist attached to the jira, and I can see replication occurring correctly.

2009-07-02 09:58:44,325 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,twebapps,172.17.11.16:32837) resuming message garbage collection
2009-07-02 09:58:44,325 DEBUG [org.jgroups.protocols.pbcast.GMS] (ViewHandler,twebapps,172.17.11.16:32837) 172.17.11.16:32837 sending RESUME event 
2009-07-02 09:58:44,326 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (ViewHandler,twebapps,172.17.11.16:32837) Received RESUME at 172.17.11.16:32837, sent STOP_FLUSH to all
2009-07-02 09:58:44,327 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-11,172.17.11.16:32837) At 172.17.11.16:32837 received STOP_FLUSH, unblocking FLUSH.down() and sending UNBLOCK up
2009-07-02 09:58:44,327 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.twebapps] (Incoming-11,172.17.11.16:32837) Unblock processed at 172.17.11.16:1399
2009-07-02 09:58:44,339 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,twebapps-HAPartitionCache,172.17.11.16:32837) resuming message garbage collection
2009-07-02 09:58:44,450 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-14,172.17.11.16:32837) At 172.17.11.16:32837 received STOP_FLUSH, unblocking FLUSH.down() and sending UNBLOCK up
2009-07-02 09:58:49,333 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.twebapps] (Incoming-15,172.17.11.16:32837) Block processed at 172.17.11.16:1399
2009-07-02 09:58:49,336 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-15,172.17.11.16:32837) Received START_FLUSH at 172.17.11.16:32837 responded with FLUSH_COMPLETED to 172.17.11.17:32890
2009-07-02 09:58:49,339 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (OOB-8,172.17.11.16:32837) Received FLUSH_RECONCILE at 172.17.11.16:32837 passing digest to NAKACK 172.17.11.17:32890: [0 : 1 (0)], 172.17.11.16:32837: [0 : 11 (0)]
2009-07-02 09:58:49,340 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (OOB-8,172.17.11.16:32837) Returned from FLUSH_RECONCILE at 172.17.11.16:32837 Sending RECONCILE_OK to 172.17.11.17:32890, thread Thread[OOB-8,172.17.11.16:32837,10,Thread Pools]
2009-07-02 09:58:49,375 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.twebapps] (Incoming-16,172.17.11.16:32837) getState called.
2009-07-02 09:58:49,375 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.twebapps] (Incoming-16,172.17.11.16:32837) getState for HASessionStateTransfer
2009-07-02 09:58:49,375 DEBUG [org.jboss.ha.hasessionstate.server.HASessionStateImpl./HASessionState/Default] (Incoming-16,172.17.11.16:32837) Building and returning state of HASessionState
2009-07-02 09:58:49,384 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.twebapps] (Incoming-16,172.17.11.16:32837) getState for DistributedReplicantManager
2009-07-02 09:58:49,456 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-17,172.17.11.16:32837) At 172.17.11.16:32837 received STOP_FLUSH, unblocking FLUSH.down() and sending UNBLOCK up
2009-07-02 09:58:49,456 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.twebapps] (Incoming-17,172.17.11.16:32837) Unblock processed at 172.17.11.16:1399
2009-07-02 09:58:51,133 DEBUG [com.coat.web.images.server.web.SSOServlet] (http-172.17.11.16-8380-1) username: jament
2009-07-02 09:58:51,692 DEBUG [com.coat.web.images.server.ejb.LoginBean] (http-172.17.11.16-8380-1) iResult is: 1
2009-07-02 09:58:51,693 DEBUG [com.coat.web.images.server.ejb.LoginBean] (http-172.17.11.16-8380-1) Successful call.
2009-07-02 09:58:51,694 DEBUG [com.coat.web.images.server.web.SSOServlet] (http-172.17.11.16-8380-1) Successful login.
2009-07-02 09:58:54,582 DEBUG [org.jboss.ha.framework.server.HARMIServerImpl$RefreshProxiesHATarget] (AsynchKeyChangeHandler Thread) replicantsChanged 'HAJNDI' to 2 (intra-view id: -843339459)
2009-07-02 09:58:54,663 DEBUG [org.jboss.ha.singleton.HASingletonSupport$HASingletonService] (AsynchKeyChangeHandler Thread) partitionTopologyChanged, isElectedNewMaster=true, isMasterNode=true, viewID=-843339459
2009-07-02 09:59:04,308 DEBUG [org.jgroups.protocols.pbcast.GMS] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) new=[172.17.11.17:32890], suspected=[], leaving=[], new view: [172.17.11.16:32837|1] [172.17.11.16:32837, 172.17.11.17:32890]
2009-07-02 09:59:04,309 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) Received Event[type=SUSPEND, arg=[172.17.11.16:32837, 172.17.11.17:32890]] at 172.17.11.16:32837. Running FLUSH...
2009-07-02 09:59:04,309 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) Flush coordinator 172.17.11.16:32837 is starting FLUSH with participants [172.17.11.16:32837]
2009-07-02 09:59:04,310 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-8,172.17.11.16:32837) Received START_FLUSH at 172.17.11.16:32837 responded with FLUSH_COMPLETED to 172.17.11.16:32837
2009-07-02 09:59:04,310 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-9,172.17.11.16:32837) At 172.17.11.16:32837 FLUSH_COMPLETED from 172.17.11.16:32837,completed true,flushMembers [172.17.11.16:32837],flushCompleted [172.17.11.16:32837]
2009-07-02 09:59:04,310 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-9,172.17.11.16:32837) All FLUSH_COMPLETED received at 172.17.11.16:32837
2009-07-02 09:59:04,310 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) suspending message garbage collection
2009-07-02 09:59:04,310 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) resume task started, max_suspend_time=33000
2009-07-02 09:59:04,311 DEBUG [org.jgroups.protocols.pbcast.GMS] (Incoming-11,172.17.11.16:32837) view=[172.17.11.16:32837|1] [172.17.11.16:32837, 172.17.11.17:32890]
2009-07-02 09:59:04,311 DEBUG [org.jgroups.protocols.pbcast.GMS] (Incoming-11,172.17.11.16:32837) [local_addr=172.17.11.16:32837] view is [172.17.11.16:32837|1] [172.17.11.16:32837, 172.17.11.17:32890]
2009-07-02 09:59:04,336 DEBUG [org.jgroups.protocols.FD_SOCK] (Timer-4,172.17.11.16:32837) VIEW_CHANGE received: [172.17.11.16:32837, 172.17.11.17:32890]
2009-07-02 09:59:04,336 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-11,172.17.11.16:32837) org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipListener@71e2b09b got new view [172.17.11.16:32837|1] [172.17.11.16:32837, 172.17.11.17:32890], old view is [172.17.11.16:32837|0] [172.17.11.16:32837]
2009-07-02 09:59:04,336 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-11,172.17.11.16:32837) I am (172.17.11.16:32837)
2009-07-02 09:59:04,336 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-11,172.17.11.16:32837) New Members : 1 ([172.17.11.17:32890])
2009-07-02 09:59:04,337 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (Incoming-11,172.17.11.16:32837) All Members : 2 ([172.17.11.16:32837, 172.17.11.17:32890])
2009-07-02 09:59:04,337 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-11,172.17.11.16:32837) Installing view at  172.17.11.16:32837 view is [172.17.11.16:32837|1] [172.17.11.16:32837, 172.17.11.17:32890]
2009-07-02 09:59:04,337 DEBUG [org.jgroups.protocols.FD_SOCK] (FD_SOCK pinger,MessagingPostOffice-CTRL,172.17.11.16:32837) first member; cache is empty
2009-07-02 09:59:04,339 DEBUG [org.jgroups.protocols.FD_SOCK] (FD_SOCK pinger,MessagingPostOffice-CTRL,172.17.11.16:32837) determinePingDest()=172.17.11.17:32890
2009-07-02 09:59:04,341 DEBUG [org.jgroups.protocols.FD_SOCK] (FD_SOCK pinger,MessagingPostOffice-CTRL,172.17.11.16:32837) ping_dest=172.17.11.17:32890, ping_sock=Socket[addr=/172.17.11.17,port=40857,localport=36305], cache={172.17.11.17:32890=172.17.11.17:40857, 172.17.11.16:32837=172.17.11.16:37931}
2009-07-02 09:59:04,348 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) resuming message garbage collection
2009-07-02 09:59:04,349 DEBUG [org.jgroups.protocols.pbcast.GMS] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) 172.17.11.16:32837 sending RESUME event
2009-07-02 09:59:04,349 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (ViewHandler,MessagingPostOffice-CTRL,172.17.11.16:32837) Received RESUME at 172.17.11.16:32837, sent STOP_FLUSH to all
2009-07-02 09:59:04,349 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-13,172.17.11.16:32837) At 172.17.11.16:32837 received STOP_FLUSH, unblocking FLUSH.down() and sending UNBLOCK up
2009-07-02 09:59:09,355 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-15,172.17.11.16:32837) Received START_FLUSH at 172.17.11.16:32837 responded with FLUSH_COMPLETED to 172.17.11.17:32890
2009-07-02 09:59:09,357 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (OOB-13,172.17.11.16:32837) Received FLUSH_RECONCILE at 172.17.11.16:32837 passing digest to NAKACK 172.17.11.17:32890: [0 : 1 (0)], 172.17.11.16:32837: [0 : 5 (0)]
2009-07-02 09:59:09,357 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (OOB-13,172.17.11.16:32837) Returned from FLUSH_RECONCILE at 172.17.11.16:32837 Sending RECONCILE_OK to 172.17.11.17:32890, thread Thread[OOB-13,172.17.11.16:32837,5,Thread Pools]
2009-07-02 09:59:10,779 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-4,172.17.11.16:32837) At 172.17.11.16:32837 received STOP_FLUSH, unblocking FLUSH.down() and sending UNBLOCK up
2009-07-02 09:59:10,873 DEBUG [org.jgroups.protocols.pbcast.GMS] (ViewHandler,MessagingPostOffice-DATA,172.17.11.16:7900) new=[172.17.11.17:7900], suspected=[], leaving=[], new view: [172.17.11.16:7900|1] [172.17.11.16:7900, 172.17.11.17:7900]
2009-07-02 09:59:10,873 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,MessagingPostOffice-DATA,172.17.11.16:7900) suspending message garbage collection
2009-07-02 09:59:10,873 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,MessagingPostOffice-DATA,172.17.11.16:7900) resume task started, max_suspend_time=33000
2009-07-02 09:59:10,883 DEBUG [org.jgroups.protocols.pbcast.GMS] (Incoming-1,172.17.11.16:7900) view=[172.17.11.16:7900|1] [172.17.11.16:7900, 172.17.11.17:7900]
2009-07-02 09:59:10,883 DEBUG [org.jgroups.protocols.pbcast.GMS] (Incoming-1,172.17.11.16:7900) [local_addr=172.17.11.16:7900] view is [172.17.11.16:7900|1] [172.17.11.16:7900, 172.17.11.17:7900]
2009-07-02 09:59:10,887 DEBUG [org.jgroups.protocols.FD_SOCK] (Timer-4,172.17.11.16:7900) VIEW_CHANGE received: [172.17.11.16:7900, 172.17.11.17:7900]
2009-07-02 09:59:10,889 DEBUG [org.jgroups.protocols.FD_SOCK] (FD_SOCK pinger,MessagingPostOffice-DATA,172.17.11.16:7900) first member; cache is empty
2009-07-02 09:59:10,890 DEBUG [org.jgroups.protocols.FD_SOCK] (FD_SOCK pinger,MessagingPostOffice-DATA,172.17.11.16:7900) determinePingDest()=172.17.11.17:7900
2009-07-02 09:59:10,892 DEBUG [org.jgroups.protocols.FD_SOCK] (FD_SOCK pinger,MessagingPostOffice-DATA,172.17.11.16:7900) ping_dest=172.17.11.17:7900, ping_sock=Socket[addr=/172.17.11.17,port=49042,localport=47572], cache={172.17.11.17:7900=172.17.11.17:49042, 172.17.11.16:7900=172.17.11.16:50496}
2009-07-02 09:59:10,930 DEBUG [org.jgroups.protocols.pbcast.STABLE] (ViewHandler,MessagingPostOffice-DATA,172.17.11.16:7900) resuming message garbage collection
2009-07-02 09:59:30,219 DEBUG [org.jgroups.protocols.pbcast.GMS] (ViewHandler,twebapps-FieldSessionCache,172.17.11.16:32837) new=[172.17.11.17:32890], suspected=[], leaving=[], new view: [172.17.11.16:32837|1] [172.17.11.16:32837, 172.17.11.17:32890]
2009-07-02 09:59:30,219 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (ViewHandler,twebapps-FieldSessionCache,172.17.11.16:32837) Received Event[type=SUSPEND, arg=[172.17.11.16:32837, 172.17.11.17:32890]] at 172.17.11.16:32837. Running FLUSH...
2009-07-02 09:59:30,220 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (ViewHandler,twebapps-FieldSessionCache,172.17.11.16:32837) Flush coordinator 172.17.11.16:32837 is starting FLUSH with participants [172.17.11.16:32837]
2009-07-02 09:59:30,220 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-9,172.17.11.16:32837) Received START_FLUSH at 172.17.11.16:32837 responded with FLUSH_COMPLETED to 172.17.11.16:32837
2009-07-02 09:59:30,220 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-11,172.17.11.16:32837) At 172.17.11.16:32837 FLUSH_COMPLETED from 172.17.11.16:32837,completed true,flushMembers [172.17.11.16:32837],flushCompleted [172.17.11.16:32837]
2009-07-02 09:59:30,221 DEBUG [org.jgroups.protocols.pbcast.FLUSH] (Incoming-11,172.17.11.16:32837) All FLUSH_COMPLETED received at 172.17.11.16:32837

However, it appears that attempting to access the replicated session object on the second server results in a new instance being created, rather than using the existing one, which makes me think that WebBeans can't find the correct HTTP session.

13. Re: Clustered WebBeans

meetoblivion Jul 3, 2009 5:37 AM (in response to meetoblivion)

so after a day of trying this out, i think i see a possible catch that's causing my woes.

I'm not 100% clear on it as of now, but I'm inclined to believe that the process in which webbeans keeps session references around for the client never become candidates for replication. as a result, when you attempt to reference a distributed EJB, it can't locate its dependencies (if it has a persistence context, or dependent beans). this seems like a bug as i believe injection should be occurring during reactivation and cleared during passivation.

i've also seen really odd behavior when a POJO has a persistencecontext injected. let's say i have a persistence unit that's configured with no transaction-type, i get very odd behavior. when i call entitymanager.persist / entitymanager.merge, nothing happens. this is independent of clustering. when i give these same classes a @Stateful, they work correctly.

14. Re: Clustered WebBeans

pmuir Jul 3, 2009 10:36 AM (in response to meetoblivion)

John Ament wrote on Jul 03, 2009 05:37:

so after a day of trying this out, i think i see a possible catch that's causing my woes.

I'm not 100% clear on it as of now, but I'm inclined to believe that the process in which webbeans keeps session references around for the client never become candidates for replication. as a result, when you attempt to reference a distributed EJB, it can't locate its dependencies (if it has a persistence context, or dependent beans). this seems like a bug as i believe injection should be occurring during reactivation and cleared during passivation.

Well, no, injection doesn't happen at reactivation, the dependent objects and proxies should be serialized along with the session bean and then deserialized at the other end. I know there may be problems with bean identity being different on different nodes. If you can describe in more detail what you are seeing?

i've also seen really odd behavior when a POJO has a persistencecontext injected. let's say i have a persistence unit that's configured with no transaction-type, i get very odd behavior. when i call entitymanager.persist / entitymanager.merge, nothing happens. this is independent of clustering. when i give these same classes a @Stateful, they work correctly.

This sounds like a TX management problem - remember that there is no TX started or stopped - you need to control the TX manaually in POJOs.