0 Replies Latest reply on Jan 26, 2015 7:27 AM by Ahmed Khan

    jboss-7.1.1-final in domain mode with 1 master / 1 slave throws CNFE when an app object instance is retrieved from cache

    Ahmed Khan Newbie

      I have set up JBoss 7.1.1-Final in domain mode with 1 master & 1 slave instances. Apache HTTPD is at the front end providing fail over / load balancing. Session replication is set up & working fine.

      Using Spring 3, I have injected one of the cache defined in domain.xml into a class provided by me. The purpose is to disallow concurrent log-in through the same user ID. For JBoss instances with sticky sessions, Spring provided SessionRegistryImpl works fine. But in cluster mode with non sticky sessions, a custom class ClusterAwareSessionRegistryImpl is implemented which is same as SessionRegistryImpl but instead of using local Maps it uses the injected cache with replication. Putting application specific class objects into the cache works and no exception is thrown. But when the cache is accessed for the objects on another jboss node then deserialzation results in ClassNotFoundException. From the exception it looks like that the application specific classes are not visible to JBoss classloader. Searched & tried a lot but I am unable to make this work. Setting cache with the classloader didn't work or maybe I missed something i.e. org.infinispan.Cache.getAdvancedCache().with(this.getClass().ClassLoader());

      Kindly point me in the right direction.

       

      12:20:17,895 INFO [org.springframework.security.core.session.SessionRegistryImpl] (ajp--10.0.1.182-8259-3) get sess info: 3Z0vJyPsKxzGVOLfk5ntxelx.slave:server-three-slave
      12:20:17,897 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (ajp--10.0.1.182-8259-3) ISPN000136: Execution error: org.infinispan.CacheException: Unable to unmarshall value
        at org.infinispan.marshall.MarshalledValue.deserialize0(MarshalledValue.java:159)
        at org.infinispan.marshall.MarshalledValue.deserialize(MarshalledValue.java:143)
        at org.infinispan.marshall.MarshalledValue.get(MarshalledValue.java:224)
        at org.infinispan.interceptors.MarshalledValueInterceptor.processRetVal(MarshalledValueInterceptor.java:326)
        at org.infinispan.interceptors.MarshalledValueInterceptor.visitGetKeyValueCommand(MarshalledValueInterceptor.java:205)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:189)
        at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:184)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:93)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:130)
        at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:89)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.interceptors.IsMarshallableInterceptor.visitGetKeyValueCommand(IsMarshallableInterceptor.java:94)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:86)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345)
        at org.infinispan.CacheImpl.get(CacheImpl.java:272)
        at org.infinispan.CacheImpl.get(CacheImpl.java:264)
        at org.infinispan.AbstractDelegatingCache.get(AbstractDelegatingCache.java:255)
        at com.ClusterAwareSessionRegistryImpl.sessionIds(ClusterAwareSessionRegistryImpl.java:160) [classes:]
        at com.ClusterAwareSessionRegistryImpl.getSessionInformation(ClusterAwareSessionRegistryImpl.java:198) [classes:]
        at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:98) [spring-security-web-3.1.0.RELEASE.jar:3.1.0.RELEASE]
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) [spring-security-web-3.1.0.RELEASE.jar:3.1.0.RELEASE]
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173) [spring-security-web-3.1.0.RELEASE.jar:3.1.0.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [org.springframework.web-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) [org.springframework.web-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]
        at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)
        at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)
        at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) [jbossweb-7.0.13.Final.jar:]
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445) [jbossweb-7.0.13.Final.jar:]
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
        at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
      Caused by: java.lang.ClassNotFoundException: com.common.model.UserModel from [Module "org.jboss.as.clustering.infinispan:main" from local module loader @6322e802 (roots: /opt/jboss-as-7.1.1.Final-slave/modules)]
        at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
        at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)
        at java.lang.Class.forName0(Native Method) [rt.jar:1.7.0_45]
        at java.lang.Class.forName(Class.java:270) [rt.jar:1.7.0_45]
        at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135)
        at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:892)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1204)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadCollectionObject(RiverUnmarshaller.java:180)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObjectArray(RiverUnmarshaller.java:1502)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadArray(RiverUnmarshaller.java:1541)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:326)
        at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:153)
        at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:139)
        at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:57)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364) [rt.jar:1.7.0_45]
        at java.util.concurrent.ConcurrentHashMap.readObject(ConcurrentHashMap.java:1574) [rt.jar:1.7.0_45]
        at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source) [:1.7.0_45]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
        at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:213)
        at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1574)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1235)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
        at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
        at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:148)
        at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:110)
        at org.infinispan.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:90)
        at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:79)
        at org.infinispan.marshall.MarshalledValue.deserialize0(MarshalledValue.java:154)
        ... 60 more

      12:20:17,937 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/i8Microbank].[action]] (ajp--10.0.1.182-8259-3) Servlet.service() for servlet action threw exception: org.infinispan.CacheException: Unable to unmarshall value
        at org.infinispan.marshall.MarshalledValue.deserialize0(MarshalledValue.java:159)
        at org.infinispan.marshall.MarshalledValue.deserialize(MarshalledValue.java:143)
        at org.infinispan.marshall.MarshalledValue.get(MarshalledValue.java:224)
        at org.infinispan.interceptors.MarshalledValueInterceptor.processRetVal(MarshalledValueInterceptor.java:326)
        at org.infinispan.interceptors.MarshalledValueInterceptor.visitGetKeyValueCommand(MarshalledValueInterceptor.java:205)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:189)
        at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:184)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:93)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:130)
        at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:89)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.interceptors.IsMarshallableInterceptor.visitGetKeyValueCommand(IsMarshallableInterceptor.java:94)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
        at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:86)
        at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:95)
        at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:60)
        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345)
        at org.infinispan.CacheImpl.get(CacheImpl.java:272)
        at org.infinispan.CacheImpl.get(CacheImpl.java:264)
        at org.infinispan.AbstractDelegatingCache.get(AbstractDelegatingCache.java:255)
        at com.ClusterAwareSessionRegistryImpl.sessionIds(ClusterAwareSessionRegistryImpl.java:160) [classes:]
        at com.ClusterAwareSessionRegistryImpl.getSessionInformation(ClusterAwareSessionRegistryImpl.java:198) [classes:]
        at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:98) [spring-security-web-3.1.0.RELEASE.jar:3.1.0.RELEASE]
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) [spring-security-web-3.1.0.RELEASE.jar:3.1.0.RELEASE]
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173) [spring-security-web-3.1.0.RELEASE.jar:3.1.0.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [org.springframework.web-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) [org.springframework.web-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]
        at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:67)
        at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:48)
        at org.jboss.as.web.session.ClusteredSessionValve.handleRequest(ClusteredSessionValve.java:125) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.session.ClusteredSessionValve.invoke(ClusteredSessionValve.java:91) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.session.JvmRouteValve.invoke(JvmRouteValve.java:88) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.session.LockingValve.invoke(LockingValve.java:56) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final-RECOMPILE.jar:7.1.1.Final]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505) [jbossweb-7.0.13.Final.jar:]
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445) [jbossweb-7.0.13.Final.jar:]
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
        at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
      Caused by: java.lang.ClassNotFoundException: com.common.model.UserModel from [Module "org.jboss.as.clustering.infinispan:main" from local module loader @6322e802 (roots: /opt/jboss-as-7.1.1.Final-slave/modules)]
        at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)
        at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)
        at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)
        at java.lang.Class.forName0(Native Method) [rt.jar:1.7.0_45]
        at java.lang.Class.forName(Class.java:270) [rt.jar:1.7.0_45]
        at org.jboss.marshalling.AbstractClassResolver.loadClass(AbstractClassResolver.java:135)
        at org.jboss.marshalling.AbstractClassResolver.resolveClass(AbstractClassResolver.java:116)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:892)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1204)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadCollectionObject(RiverUnmarshaller.java:180)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObjectArray(RiverUnmarshaller.java:1502)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadArray(RiverUnmarshaller.java:1541)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:326)
        at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:153)
        at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:139)
        at org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:57)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:364) [rt.jar:1.7.0_45]
        at java.util.concurrent.ConcurrentHashMap.readObject(ConcurrentHashMap.java:1574) [rt.jar:1.7.0_45]
        at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source) [:1.7.0_45]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
        at org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:213)
        at org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1574)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1235)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)
        at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)
        at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)
        at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:148)
        at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:110)
        at org.infinispan.marshall.AbstractMarshaller.objectFromByteBuffer(AbstractMarshaller.java:90)
        at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:79)
        at org.infinispan.marshall.MarshalledValue.deserialize0(MarshalledValue.java:154)
        ... 60 more