11 Replies Latest reply on Aug 29, 2012 8:01 PM by johanvermeij

    AS 7.1.1, simple clustered deployment and ClassNotFound exception

    bwallis42

      I have a simple little app with a bit of faces/richfaces for the UI, a single stateful session bean for a service layer and a couple of JPA beans for persistence backed by hibernate. It is based off the project called "multi"  generated from jboss tools when you select new->project and JBoss Central->Java EE Project.

       

      It successfully deploys and runs in AS 7.1.1.Final running in Domain mode using the other-server-group, server-three which I have configured to run full-ha profile with the full-ha-sockets.

       

      Now I am trying to run a second AS 7.1.1.Final server on a second machine as part of a cluster setup (I have two OpenSUSE 12.1 linux VMs). I have installed and configured mod-cluster with apache and configured things by following the instructions at https://docs.jboss.org/author/display/AS71/AS7+Cluster+Howto

       

      I seem to have come across a problem. It is very similar to the problem described in ticket AS7-2356 and other linked tickets. This problem was marked as fixed in 7.1.0.Beta1.

       

      I start jboss on the master server (running the domain controller process) with no errors. Then I start jboss on the slave server and as it proceeds to deploy the ear file for my application it gets the following exception as it attempts to setup communications for the clustering with the master server

       

      {code}

      [Server:server-three] 11:50:52,514 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000078: Starting JGroups Channel

      [Server:server-three] 11:50:52,519 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000094: Received new cluster view: [master:server-three/hibernate|1] [master:server-three/hibernate, slave:server-three/hibernate]

      [Server:server-three] 11:50:52,523 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-4) ISPN000079: Cache local address is slave:server-three/hibernate, physical addresses are [172.16.95.132:55450]

      [Server:server-three] 11:50:52,556 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-4) ISPN000031: MBeans were successfully registered to the platform mbean server.

      [Server:server-three] 11:50:52,562 WARN  [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-20,null) Problems unmarshalling remote command from byte buffer: java.lang.ClassNotFoundException: org.infinispan.util.ImmutableListCopy from [Module "deployment.multi.ear.multi-ejb-0.0.1-SNAPSHOT.jar:main" from Service Module Loader]

      [Server:server-three]   at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:423)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)

      [Server:server-three]   at java.lang.Class.forName0(Native Method) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.lang.Class.forName(Class.java:264) [rt.jar:1.6.0_22]

      [Server:server-three]   at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135)

      [Server:server-three]   at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:934)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1204)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

      [Server:server-three]   at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

      [Server:server-three]   at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:118)

      [Server:server-three]   at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:164)

      [Server:server-three]   at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:68)

      [Server:server-three]   at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:401)

      [Server:server-three]   at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:290)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

      [Server:server-three]   at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

      [Server:server-three]   at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:148)

      [Server:server-three]   at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:110)

      [Server:server-three]   at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:84)

      [Server:server-three]   at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)

      [Server:server-three]   at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:196)

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:456)

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:363)

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238)

      [Server:server-three]   at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543)

      [Server:server-three]   at org.jboss.as.clustering.jgroups.ClassLoaderAwareUpHandler.up(ClassLoaderAwareUpHandler.java:56)

      [Server:server-three]   at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)

      [Server:server-three]   at org.jboss.as.clustering.jgroups.MuxChannel$ClassLoaderAwareMuxUpHandler.up(MuxChannel.java:64)

      [Server:server-three]   at org.jgroups.JChannel.up(JChannel.java:716)

      [Server:server-three]   at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026)

      [Server:server-three]   at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)

      [Server:server-three]   at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)

      [Server:server-three]   at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)

      [Server:server-three]   at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881)

      [Server:server-three]   at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)

      [Server:server-three]   at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:759)

      [Server:server-three]   at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:365)

      [Server:server-three]   at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:595)

      [Server:server-three]   at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)

      [Server:server-three]   at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140)

      [Server:server-three]   at org.jgroups.protocols.FD.up(FD.java:273)

      [Server:server-three]   at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282)

      [Server:server-three]   at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)

      [Server:server-three]   at org.jgroups.protocols.Discovery.up(Discovery.java:355)

      [Server:server-three]   at org.jgroups.stack.Protocol.up(Protocol.java:358)

      [Server:server-three]   at org.jgroups.protocols.TP.passMessageUp(TP.java:1174)

      [Server:server-three]   at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722)

      [Server:server-three]   at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704)

      [Server:server-three]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.lang.Thread.run(Thread.java:679) [rt.jar:1.6.0_22]

      {code}

       

      at the same time the master server gets the following errors.

       

      {code}

      [Server:server-three] 11:50:50,336 INFO  [org.jboss.as.clustering.impl.CoreGroupCommunicationService.lifecycle.web] (Incoming-1,null) JBAS010247: New cluster view for partition web (id: 1, delta: 1, merge: false) : [master:server-three/web, slave:server-three/web]

      [Server:server-three] 11:50:50,337 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,null) ISPN000094: Received new cluster view: [master:server-three/web|1] [master:server-three/web, slave:server-three/web]

      [Server:server-three] 11:50:50,385 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-3,null) ISPN000094: Received new cluster view: [master:server-three/hibernate|1] [master:server-three/hibernate, slave:server-three/hibernate]

      [Server:server-three] 11:50:52,644 SEVERE [org.jgroups.blocks.RequestCorrelator] (OOB-19,null) failed unmarshalling buffer into return value: java.lang.ClassNotFoundException: org.infinispan.CacheException from [Module "deployment.multi.ear.multi-ejb-0.0.1-SNAPSHOT.jar:main" from Service Module Loader]

      [Server:server-three]   at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:423)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)

      [Server:server-three]   at java.lang.Class.forName0(Native Method) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.lang.Class.forName(Class.java:264) [rt.jar:1.6.0_22]

      [Server:server-three]   at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135)

      [Server:server-three]   at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:892)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1204)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

      [Server:server-three]   at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

      [Server:server-three]   at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:66)

      [Server:server-three]   at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:58)

      [Server:server-three]   at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:401)

      [Server:server-three]   at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:290)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

      [Server:server-three]   at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

      [Server:server-three]   at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:148)

      [Server:server-three]   at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:110)

      [Server:server-three]   at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:84)

      [Server:server-three]   at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:376)

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238)

      [Server:server-three]   at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543)

      [Server:server-three]   at org.jboss.as.clustering.jgroups.ClassLoaderAwareUpHandler.up(ClassLoaderAwareUpHandler.java:56)

      [Server:server-three]   at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130)

      [Server:server-three]   at org.jboss.as.clustering.jgroups.MuxChannel$ClassLoaderAwareMuxUpHandler.up(MuxChannel.java:64)

      [Server:server-three]   at org.jgroups.JChannel.up(JChannel.java:716)

      [Server:server-three]   at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026)

      [Server:server-three]   at org.jgroups.protocols.FRAG2.up(FRAG2.java:181)

      [Server:server-three]   at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)

      [Server:server-three]   at org.jgroups.protocols.FlowControl.up(FlowControl.java:400)

      [Server:server-three]   at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881)

      [Server:server-three]   at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)

      [Server:server-three]   at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:759)

      [Server:server-three]   at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:365)

      [Server:server-three]   at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:595)

      [Server:server-three]   at org.jgroups.protocols.BARRIER.up(BARRIER.java:102)

      [Server:server-three]   at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140)

      [Server:server-three]   at org.jgroups.protocols.FD.up(FD.java:273)

      [Server:server-three]   at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282)

      [Server:server-three]   at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)

      [Server:server-three]   at org.jgroups.protocols.Discovery.up(Discovery.java:355)

      [Server:server-three]   at org.jgroups.stack.Protocol.up(Protocol.java:358)

      [Server:server-three]   at org.jgroups.protocols.TP.passMessageUp(TP.java:1174)

      [Server:server-three]   at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722)

      [Server:server-three]   at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704)

      [Server:server-three]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.lang.Thread.run(Thread.java:679) [rt.jar:1.6.0_22]

      [Server:server-three]

      [Server:server-three] 11:50:52,673 ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-1,master:server-three/hibernate) ISPN000172: Failed to prepare view CacheView{viewId=2, members=[master:server-three/hibernate, slave:server-three/hibernate]} for cache  multi.ear/multi-ejb-0.0.1-SNAPSHOT.jar#primary.org.hibernate.cache.spi.UpdateTimestampsCache, rolling back to view CacheView{viewId=1, members=[master:server-three/hibernate]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.ClassNotFoundException: org.infinispan.CacheException from [Module "deployment.multi.ear.multi-ejb-0.0.1-SNAPSHOT.jar:main" from Service Module Loader]

      [Server:server-three]   at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.util.concurrent.FutureTask.get(FutureTask.java:119) [rt.jar:1.6.0_22]

      [Server:server-three]   at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:319) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.lang.Thread.run(Thread.java:679) [rt.jar:1.6.0_22]

      [Server:server-three] Caused by: org.infinispan.CacheException: java.lang.ClassNotFoundException: org.infinispan.CacheException from [Module "deployment.multi.ear.multi-ejb-0.0.1-SNAPSHOT.jar:main" from Service Module Loader]

      [Server:server-three]   at org.infinispan.util.Util.rewrapAsCacheException(Util.java:524) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:168) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:478) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.cacheviews.CacheViewsManagerImpl$2.call(CacheViewsManagerImpl.java:303) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.cacheviews.CacheViewsManagerImpl$2.call(CacheViewsManagerImpl.java:300) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   ... 5 more

      [Server:server-three] Caused by: java.lang.ClassNotFoundException: org.infinispan.CacheException from [Module "deployment.multi.ear.multi-ejb-0.0.1-SNAPSHOT.jar:main" from Service Module Loader]

      [Server:server-three]   at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:423)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)

      [Server:server-three]   at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)

      [Server:server-three]   at java.lang.Class.forName0(Native Method) [rt.jar:1.6.0_22]

      [Server:server-three]   at java.lang.Class.forName(Class.java:264) [rt.jar:1.6.0_22]

      [Server:server-three]   at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135) [jboss-marshalling-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116) [jboss-marshalling-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:892) [jboss-marshalling-river-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1204) [jboss-marshalling-river-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272) [jboss-marshalling-river-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:66) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.remoting.responses.ExceptionResponse$Externalizer.readObject(ExceptionResponse.java:58) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:401) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:290) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351) [jboss-marshalling-river-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209) [jboss-marshalling-river-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37) [jboss-marshalling-1.3.11.GA.jar:1.3.11.GA]

      [Server:server-three]   at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:148) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:110) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:84) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50) [infinispan-core-5.1.2.FINAL.jar:5.1.2.FINAL]

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:376) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jboss.as.clustering.jgroups.ClassLoaderAwareUpHandler.up(ClassLoaderAwareUpHandler.java:56)

      [Server:server-three]   at org.jgroups.blocks.mux.MuxUpHandler.up(MuxUpHandler.java:130) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jboss.as.clustering.jgroups.MuxChannel$ClassLoaderAwareMuxUpHandler.up(MuxChannel.java:64)

      [Server:server-three]   at org.jgroups.JChannel.up(JChannel.java:716) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:759) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:365) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:595) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.BARRIER.up(BARRIER.java:102) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.Discovery.up(Discovery.java:355) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.stack.Protocol.up(Protocol.java:358) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.TP.passMessageUp(TP.java:1174) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704) [jgroups-3.0.6.Final.jar:3.0.6.Final]

      [Server:server-three]   ... 3 more

      {code}

       

      I can supply more details and configuration but the errors seem to indicate that the fix noted in the ticket referenced above is not applied to the release 7.1.1.FINAL

        • 1. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
          traian20

          I have a configuration similar to this one using JBoss 7.1.1, and it works for me(on Windows).

          So I guess the bug is fixed. Does your war have the <distributable/> tag in web.xml?

          Make sure the <server-group> uses the <socket-binding-group ref="ha-sockets"/>

          • 2. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
            bwallis42

            Thanks, good to hear that it is working for someone.

             

            I do have the distributable tag and am using the ha-sockets.

             

            I just had a thought though, these are new OpenSUSE installations and are running the OpenJDK JDK 1.6.0_22, I wonder if that could be the problem. I will download a "proper" jdk and give that a go.

            • 3. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
              rhusar

              Thanks, good to hear that it is working for someone.

              It surely works in standalone mode with HA profile (there might be some issues in the domain).

              • 4. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                bwallis42

                I've changed to running a pair of standalone servers configured for clustered operation.

                 

                I start up jboss in standalone mode using standalone-full-ha.xml on the first server (hostname "master") and it starts OK and I can access the deployed application.

                 

                I start up jboss in standalone mode on the second server (hostname "slave") and when it deploys the ejb jar file I get the following error when it attempts to setup the clustered ejb I get the following:

                 

                {code}

                10:51:01,013 INFO  [org.hibernate.cache.spi.UpdateTimestampsCache] (MSC service thread 1-1) HHH000250: Starting update timestamps cache at region: multi.ear/multi-ejb-0.0.1-SNAPSHOT.jar#primary.org.hibernate.cache.spi.UpdateTimestampsCache

                10:51:01,016 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups Channel

                10:51:01,017 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000094: Received new cluster view: [master/hibernate|3] [master/hibernate, slave/hibernate]

                10:51:01,017 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Cache local address is slave/hibernate, physical addresses are [172.16.95.132:55200]

                10:51:01,026 INFO  [org.infinispan.jmx.CacheJmxRegistration] (MSC service thread 1-1) ISPN000031: MBeans were successfully registered to the platform mbean server.

                10:51:01,032 WARN  [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-15,null) Problems unmarshalling remote command from byte buffer: java.lang.ClassNotFoundException: org.infinispan.util.ImmutableListCopy from [Module "deployment.multi.ear.multi-ejb-0.0.1-SNAPSHOT.jar:main" from Service Module Loader]

                {code}

                 

                and the rest of the stacktrace is as before.

                 

                My application is deployed as an ear file containing a war file and an ejb jar file. There is a single statefull session bean in the ejb jar.

                 

                The exception seems to point straight back to  AS7-2206 which is supposed to be fixed.

                 

                I'm going to try and simplify my test application some more to see if I can isolate what is causing this.

                • 5. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                  bwallis42

                  OK. I can now deploy and run in the cluster without the error(s). I had hibernate second level cache turned on. Turning it off makes the errors go away.

                   

                  Now to look for a reason for this. Probably a misconfiguration on my part.

                  • 6. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                    smarlow

                    If the problem with the second level cache recreates with the latest AS7 nightly build (if this is something you can try).  Please show us the contents of your persistence.xml.

                    1 of 1 people found this helpful
                    • 7. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                      bwallis42

                      I've downloaded the latest nightly build and tried my app with the hibernate 2nd level caching turned back on and it works without the errors.

                       

                      I'll go back to using a domain setup instead of the standalone one and see how that goes.

                       

                      Thanks for the suggestion.

                      • 8. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                        johanvermeij

                        I have been struggling with a similar problem for a while now, see: https://community.jboss.org/thread/198659

                         

                        Please give me a bit of xml context for the <distributable\> tag and also how to turn of hbernate 2nd level cache.

                         

                        I only know of 2 config files for 7.1.1 clustering: domain.xml and host.xml (the master has both, the slave only has host.xml)

                         

                        What is web.xml ?

                        • 9. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                          johanvermeij

                          Let me answer this because it may be useful to others, I just found out.

                           

                          JBoss 7.1 is configured by using domain.xml and host.xml

                           

                          web.xml and persistence.xml are configured separately outside the above xml's

                          • 10. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                            bwallis42

                            Hi Johan,

                              Correct, web.xml and persistence.xml are part of your deployed application, not part of the configuration of the appserver. The appserver config is in either domain/host.xml or in standalone.xml if you are running in standalone mode.

                            • 11. Re: AS 7.1.1, simple clustered deployment and ClassNotFound exception
                              johanvermeij

                              I had a chance to try with EAP 6 and all is ok.