2 Replies Latest reply on May 22, 2013 9:57 AM by sojourner48

    java.util.ConcurrentModificationException occurs during session replication

    sojourner48

      I have been using Vaadin without any issues in a single standalone JBoss AS 7.1.3 environment. When running two JBoss nodes in a cluster with session replication enabled, a ConcurrentModificationException (see stacktrace below) occurs when the session replication occurs. There was only one user on the system at the time, and there are no add-ons as part of the project that would be starting a separate thread. The stacktrace below occurred on a page that contains a table. However, the problem can be replicated on any other page as well.

      I've had a headache for days trying to figure this out. Does anyone have any ideas or suggestions as to the cause of this? Any assistance would be greatly appreciated.

      12:28:53,939 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (ajp-/10.52.1.124:8009-4) ISPN000136: Execution error: org.infinispan.CacheException: java.lang.RuntimeException: Failure to marshal argument(s)
        at org.infinispan.util.Util.rewrapAsCacheException(Util.java:526) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:172) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:489) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:161) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:183) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:240) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:227) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:222) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:201) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.ReplicationInterceptor.broadcastPrepare(ReplicationInterceptor.java:136) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:128) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:93) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:120) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:131) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:106) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.StateTransferLockInterceptor.handleWithRetries(StateTransferLockInterceptor.java:213) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.StateTransferLockInterceptor.visitPrepareCommand(StateTransferLockInterceptor.java:85) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:132) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:91) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:86) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:133) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:174) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:81) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:285) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:334) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:91) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:102) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.as.clustering.web.impl.TransactionBatchingManager.endBatch(TransactionBatchingManager.java:75)
        at org.jboss.as.web.session.DistributableSessionManager.processSessionRepl(DistributableSessionManager.java:1515) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.session.DistributableSessionManager.storeSession(DistributableSessionManager.java:857) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.session.InstantSnapshotManager.snapshot(InstantSnapshotManager.java:47) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:142) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:99) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:92) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:64) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:165) [jboss-as-web-7.1.3.Final.jar:7.1.3.Final]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.17.Final.jar:]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.17.Final.jar:]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.17.Final.jar:]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:372) [jbossweb-7.0.17.Final.jar:]
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) [jbossweb-7.0.17.Final.jar:]
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:453) [jbossweb-7.0.17.Final.jar:]
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:931) [jbossweb-7.0.17.Final.jar:]
        at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09]
      Caused by: java.lang.RuntimeException: Failure to marshal argument(s)
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:257) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:274) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:165) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        ... 72 more
      Caused by: java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:894) [rt.jar:1.7.0_09]
        at java.util.HashMap$KeyIterator.next(HashMap.java:928) [rt.jar:1.7.0_09]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:604)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:679)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:585)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:679)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverObjectOutputStream.defaultWriteObject(RiverObjectOutputStream.java:159)
        at com.vaadin.event.ListenerMethod.writeObject(ListenerMethod.java:90) [vaadin-6.8.6.jar:6.8.6]
        at sun.reflect.GeneratedMethodAccessor98.invoke(Unknown Source) [:1.7.0_09]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_09]
        at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_09]
        at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:188) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1007)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:585)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:680)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:585)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:679)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1063)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:1019)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:998)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:885)
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:680)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.as.clustering.SimpleMarshalledValue.getBytes(SimpleMarshalledValue.java:85)
        at org.jboss.as.clustering.SimpleMarshalledValue.writeExternal(SimpleMarshalledValue.java:175)
        at org.jboss.as.clustering.infinispan.io.ExternalizableExternalizer.writeObject(ExternalizableExternalizer.java:46)
        at org.jboss.as.clustering.infinispan.io.ExternalizableExternalizer.writeObject(ExternalizableExternalizer.java:36)
        at org.infinispan.marshall.jboss.ExternalizerTable$ForeignExternalizerAdapter.writeObject(ExternalizerTable.java:450) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.infinispan.atomic.PutOperation$Externalizer.writeObject(PutOperation.java:80) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.atomic.PutOperation$Externalizer.writeObject(PutOperation.java:76) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:401) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.infinispan.marshall.MarshallUtil.marshallCollection(MarshallUtil.java:48) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.LinkedListExternalizer.writeObject(LinkedListExternalizer.java:50) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.LinkedListExternalizer.writeObject(LinkedListExternalizer.java:45) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:401) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.infinispan.atomic.AtomicHashMapDelta$Externalizer.writeObject(AtomicHashMapDelta.java:110) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.atomic.AtomicHashMapDelta$Externalizer.writeObject(AtomicHashMapDelta.java:106) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:401) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:83) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:70) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeObject(ReplicableCommandExternalizer.java:58) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:401) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.infinispan.marshall.exts.ReplicableCommandExternalizer.writeCommandParameters(ReplicableCommandExternalizer.java:85) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.marshallParameters(CacheRpcCommandExternalizer.java:124) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:111) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.writeObject(CacheRpcCommandExternalizer.java:66) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.writeObject(ExternalizerTable.java:401) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:145)
        at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:62) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:119) [jboss-marshalling-1.3.15.GA.jar:1.3.15.GA]
        at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectToObjectStream(AbstractJBossMarshaller.java:96) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.VersionAwareMarshaller.objectToBuffer(VersionAwareMarshaller.java:90) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.AbstractMarshaller.objectToBuffer(AbstractMarshaller.java:64) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.marshall.AbstractDelegatingMarshaller.objectToBuffer(AbstractDelegatingMarshaller.java:109) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectToBuffer(MarshallerAdapter.java:45) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.marshallCall(CommandAwareRpcDispatcher.java:255) [infinispan-core-5.1.7.Final.jar:5.1.7.Final]
        ... 74 more
      Caused by: an exception which occurred:
        in field changedSelectedKeys
        in field componentToExpandRatio
        in field root
        in field components
        in field content
        in field componentToExpandRatio
        in field parent
        in field target
        in field listenerList
        in field eventRouter
        in field parent
        in field parent
        in field idPaintableMap
        in field repaintRequestListeners
        in field mainWindow
        in field applicationToAjaxAppMgrMap
        in object java.util.HashMap@38046b36
        in object org.jboss.as.clustering.SimpleMarshalledValue@38046b36
        in object org.infinispan.atomic.PutOperation@4a3da462
        in object java.util.LinkedList@1d3ef9f9
        in object org.infinispan.atomic.AtomicHashMapDelta@3829a747
        in object org.infinispan.commands.write.PutKeyValueCommand@b1b7df3a
        in object org.infinispan.commands.tx.PrepareCommand@706ff234