4 Replies Latest reply on Aug 12, 2013 3:10 AM by Galder Zamarreño

    Preloading Exception when using infinispan as RemoteCacheStore for JBoss As 7.2.0

    Zhilong Lei Newbie

      Hi

       

       

      I got a problem while I was trying to use standalone infinispan data grid as a remote cache store of Jboss AS(7.2.0 Final) in order to avoid web session replication.

      Here's my ideal architecture:

       

      无标题.png

       

      My nodes infomation:

      Jboss  As Node address:172.17.254.230

      Jboss As Version: 7.2.0.Final

      Infinispan Node1 address:172.17.254.231

      Infinispan Node2 address:172.17.254.232

      Infinispan Data Grid version: 5.3.0.Final

      and also my Infinispan Standalone Server version: 5.2.1.Final

       

      When I put some data into cache infinispan data grid:

      infinispan@/home/infinispan/infinispan-server-5.3.0.Final/bin>ispn-cli.sh -c remoting://172.17.254.232:9999
      Mar 1, 2013 2:43:31 PM org.xnio.Xnio <clinit>
      INFO: XNIO Version 3.0.7.GA
      Mar 1, 2013 2:43:31 PM org.xnio.nio.NioXnio <clinit>
      INFO: XNIO NIO Implementation Version 3.0.7.GA
      Mar 1, 2013 2:43:31 PM org.jboss.remoting3.EndpointImpl <clinit>
      INFO: JBoss Remoting version 3.2.14.GA
      [remoting://172.17.254.232:9999/clustered/]> cache usertable 
      [remoting://172.17.254.232:9999/clustered/usertable]> put key1 aaa
      [remoting://172.17.254.232:9999/clustered/usertable]> put key2 bbb
      [remoting://172.17.254.232:9999/clustered/usertable]> put key3 ccc
      [remoting://172.17.254.232:9999/clustered/usertable]> get key1
      aaa
      [remoting://172.17.254.232:9999/clustered/usertable]> get key2
      bbb
      [remoting://172.17.254.232:9999/clustered/usertable]> get key3
      ccc
      [remoting://172.17.254.232:9999/clustered/usertable]> 
      

       

       

       

      and then I started a Jboss AS instance, got an error while preloading cache data from remote cache store.

      it seems that the exception was found on infinispan  and  passed to jboss as.

       

       

      here's my Jboss AS log:

      =========================================================================
      
      
        JBoss Bootstrap Environment
      
      
        JBOSS_HOME: /home/infinispan/jbossas/jboss-as-7.2.0.Final
      
      
        JAVA: /usr/java/jdk1.6.0_45/bin/java
      
      
        JAVA_OPTS:  -server -XX:+UseCompressedOops -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true
      
      
      =========================================================================
      
      
      13:12:01,060 INFO  [org.jboss.modules] (main) JBoss Modules version 1.2.0.CR1
      13:12:01,391 INFO  [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA
      13:12:01,491 INFO  [org.jboss.as] (MSC service thread 1-1) JBAS015899: JBoss AS 7.2.0.Final "Janus" starting
      13:12:04,521 INFO  [org.xnio] (MSC service thread 1-3) XNIO Version 3.0.7.GA
      13:12:04,549 INFO  [org.xnio.nio] (MSC service thread 1-3) XNIO NIO Implementation Version 3.0.7.GA
      13:12:04,551 INFO  [org.jboss.as.server] (Controller Boot Thread) JBAS015888: Creating http management service using socket-binding (management-http)
      13:12:04,589 INFO  [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 3.2.14.GA
      13:12:04,703 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 36) JBAS010280: Activating Infinispan subsystem.
      13:12:04,706 INFO  [org.jboss.as.jacorb] (ServerService Thread Pool -- 37) JBAS016300: Activating JacORB Subsystem
      13:12:04,740 INFO  [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 42) JBAS010260: Activating JGroups subsystem.
      13:12:04,804 INFO  [org.jboss.as.security] (ServerService Thread Pool -- 54) JBAS013171: Activating Security Subsystem
      13:12:04,848 INFO  [org.jboss.as.naming] (ServerService Thread Pool -- 49) JBAS011800: Activating Naming Subsystem
      13:12:04,861 INFO  [org.jboss.as.jsf] (ServerService Thread Pool -- 45) JBAS012605: Activated the following JSF Implementations: [main, 1.2]
      13:12:04,951 INFO  [org.jboss.as.webservices] (ServerService Thread Pool -- 58) JBAS015537: Activating WebServices Extension
      13:12:05,083 INFO  [org.jboss.as.security] (MSC service thread 1-3) JBAS013170: Current PicketBox version=4.0.15.Final
      13:12:05,085 INFO  [org.jboss.as.connector.logging] (MSC service thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.15.Final)
      13:12:05,135 INFO  [org.jboss.as.naming] (MSC service thread 1-1) JBAS011802: Starting Naming Service
      13:12:05,492 INFO  [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 32) JBAS010403: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3)
      13:12:05,620 INFO  [org.jboss.jaxr] (MSC service thread 1-1) JBAS014000: Started JAXR subsystem, binding JAXR connection factory into JNDI as: java:jboss/jaxr/ConnectionFactory
      13:12:05,642 INFO  [org.jboss.as.mail.extension] (MSC service thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
      13:12:06,112 INFO  [org.jboss.ws.common.management] (MSC service thread 1-3) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.1.3.Final
      13:12:06,296 INFO  [org.apache.coyote.http11] (MSC service thread 1-4) JBWEB003001: Coyote HTTP/1.1 initializing on : http-/172.17.254.230:8080
      13:12:06,298 INFO  [org.apache.coyote.ajp] (MSC service thread 1-2) JBWEB003046: Starting Coyote AJP/1.3 on ajp-/172.17.254.230:8009
      13:12:06,327 INFO  [org.apache.coyote.http11] (MSC service thread 1-4) JBWEB003000: Coyote HTTP/1.1 starting on: http-/172.17.254.230:8080
      13:12:06,661 INFO  [org.jboss.modcluster.ModClusterService] (ServerService Thread Pool -- 60) Initializing mod_cluster 1.2.3.Final
      13:12:06,792 INFO  [org.jboss.modcluster.advertise.impl.AdvertiseListenerImpl] (ServerService Thread Pool -- 60) Listening to proxy advertisements on 224.0.1.105:23,364
      13:12:07,009 INFO  [org.jboss.as.jacorb] (MSC service thread 1-1) JBAS016330: CORBA ORB Service started
      13:12:07,037 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 36) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
      13:12:07,072 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 36) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
      13:12:07,764 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-3) JBAS015012: Started FileSystemDeploymentService for directory /home/infinispan/jbossas/jboss-as-7.2.0.Final/standalone/deployments
      13:12:07,816 INFO  [org.jboss.as.jacorb] (MSC service thread 1-4) JBAS016328: CORBA Naming Service started
      13:12:07,893 INFO  [org.jboss.as.remoting] (MSC service thread 1-4) JBAS017100: Listening on 172.17.254.230:4447
      13:12:07,893 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on 172.17.254.230:9999
      13:12:08,070 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]
      13:12:08,845 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
      13:12:08,848 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-3) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
      13:12:08,859 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-2) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
      13:12:08,865 INFO  [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (MSC service thread 1-3) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated.
      13:12:08,949 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 60) ISPN000078: Starting JGroups Channel
      13:12:08,978 INFO  [stdout] (ServerService Thread Pool -- 60) 
      13:12:08,978 INFO  [stdout] (ServerService Thread Pool -- 60) -------------------------------------------------------------------
      13:12:08,979 INFO  [stdout] (ServerService Thread Pool -- 60) GMS: address=slave07/web, cluster=web, physical address=172.17.254.230:55200
      13:12:08,979 INFO  [stdout] (ServerService Thread Pool -- 60) -------------------------------------------------------------------
      13:12:11,017 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 60) ISPN000094: Received new cluster view: [slave07/web|0] [slave07/web]
      13:12:11,235 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (ServerService Thread Pool -- 60) ISPN000079: Cache local address is slave07/web, physical addresses are [172.17.254.230:55200]
      13:12:11,263 INFO  [org.infinispan.factories.GlobalComponentRegistry] (ServerService Thread Pool -- 60) ISPN000128: Infinispan version: Infinispan 'Delirium' 5.2.1.Final
      13:12:11,381 INFO  [org.infinispan.factories.TransactionManagerFactory] (ServerService Thread Pool -- 60) ISPN000161: Using a batchMode transaction manager
      13:12:11,628 INFO  [org.infinispan.client.hotrod.RemoteCacheManager] (ServerService Thread Pool -- 60) ISPN004021: Infinispan version: Infinispan 'Delirium' 5.2.1.Final
      13:12:11,632 INFO  [org.infinispan.client.hotrod.RemoteCacheManager] (ServerService Thread Pool -- 60) ISPN004021: Infinispan version: Infinispan 'Delirium' 5.2.1.Final
      13:12:11,855 INFO  [org.infinispan.client.hotrod.impl.protocol.Codec12] (ServerService Thread Pool -- 60) ISPN004006: /172.17.254.231:11222 sent new topology view (id=2) containing 2 addresses: [/172.17.254.232:11222, /172.17.254.231:11222]
      13:12:11,910 INFO  [org.infinispan.jmx.CacheJmxRegistration] (ServerService Thread Pool -- 60) ISPN000031: MBeans were successfully registered to the platform MBean server.
      13:12:11,928 WARN  [org.infinispan.client.hotrod.impl.protocol.Codec12] (ServerService Thread Pool -- 60) ISPN004005: Error received from the server: java.lang.ClassCastException: java.lang.String cannot be cast to [B
      13:12:11,932 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 60) MSC00001: Failed to start service jboss.infinispan.web.repl: org.jboss.msc.service.StartException in service jboss.infinispan.web.repl: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.preload() on object of type CacheLoaderManagerImpl
              at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:87) [jboss-as-clustering-common-7.2.0.Final.jar:7.2.0.Final]
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [rt.jar:1.6.0_45]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [rt.jar:1.6.0_45]
              at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45]
              at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.0.Final.jar:2.1.0.Final]
      Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.preload() on object of type CacheLoaderManagerImpl
              at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:205)
              at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:883)
              at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:654)
              at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:643)
              at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:546)
              at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:199)
              at org.infinispan.CacheImpl.start(CacheImpl.java:559)
              at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:686)
              at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:649)
              at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:545)
              at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:559)
              at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:109)
              at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:100)
              at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
              at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:82) [jboss-as-clustering-common-7.2.0.Final.jar:7.2.0.Final]
              ... 4 more
      Caused by: org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[3] returned server error (status=0x85): java.lang.ClassCastException: java.lang.String cannot be cast to [B
              at org.infinispan.client.hotrod.impl.protocol.Codec10.checkForErrorsInResponseStatus(Codec10.java:157)
              at org.infinispan.client.hotrod.impl.protocol.Codec10.readHeader(Codec10.java:110)
              at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:78)
              at org.infinispan.client.hotrod.impl.operations.BulkGetOperation.executeOperation(BulkGetOperation.java:60)
              at org.infinispan.client.hotrod.impl.operations.BulkGetOperation.executeOperation(BulkGetOperation.java:41)
              at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:68)
              at org.infinispan.client.hotrod.impl.RemoteCacheImpl.getBulk(RemoteCacheImpl.java:371)
              at org.infinispan.client.hotrod.impl.RemoteCacheImpl.getBulk(RemoteCacheImpl.java:363)
              at org.infinispan.loaders.remote.RemoteCacheStore.loadAll(RemoteCacheStore.java:154)
              at org.infinispan.loaders.CacheLoaderManagerImpl.loadState(CacheLoaderManagerImpl.java:279)
              at org.infinispan.loaders.CacheLoaderManagerImpl.preload(CacheLoaderManagerImpl.java:238)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.6.0_45]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [rt.jar:1.6.0_45]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [rt.jar:1.6.0_45]
              at java.lang.reflect.Method.invoke(Method.java:597) [rt.jar:1.6.0_45]
              at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:203)
              ... 18 more
      
      
      13:12:12,180 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
      JBAS014777:   Services which failed to start:      service jboss.infinispan.web.repl: org.jboss.msc.service.StartException in service jboss.infinispan.web.repl: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.preload() on object of type CacheLoaderManagerImpl
      
      
      13:12:12,211 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://172.17.254.230:9990/management
      13:12:12,212 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://172.17.254.230:9990
      13:12:12,214 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: JBoss AS 7.2.0.Final "Janus" started (with errors) in 11982ms - Started 142 of 262 services (1 services failed or missing dependencies, 118 services are passive or on-demand)
      
      
      

       

       

       

       

      and my infinispan log:

       

      10:01:21,433 ERROR [org.infinispan.server.hotrod.HotRodDecoder] (HotRodServerWorker-32) ISPN005009: Unexpected error before any request parameters read: java.lang.ClassCastException: java.lang.String cannot be cast to [B
              at org.infinispan.server.hotrod.AbstractEncoder1x$$anonfun$writeResponse$3.apply(AbstractEncoder1x.scala:118) [infinispan-server-hotrod-5.3.0.Final.jar:5.3.0.Final]
              at org.infinispan.server.hotrod.AbstractEncoder1x$$anonfun$writeResponse$3.apply(AbstractEncoder1x.scala:116) [infinispan-server-hotrod-5.3.0.Final.jar:5.3.0.Final]
              at scala.collection.Iterator$class.foreach(Iterator.scala:727)
              at scala.collection.AbstractIterator.foreach(Iterator.scala:1156)
              at org.infinispan.server.hotrod.AbstractEncoder1x.writeResponse(AbstractEncoder1x.scala:116) [infinispan-server-hotrod-5.3.0.Final.jar:5.3.0.Final]
              at org.infinispan.server.hotrod.HotRodEncoder.encode(HotRodEncoder.scala:69) [infinispan-server-hotrod-5.3.0.Final.jar:5.3.0.Final]
              at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.doEncode(OneToOneEncoder.java:66) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:59) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.Channels.write(Channels.java:704) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.Channels.write(Channels.java:671) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:248) [netty-3.6.5.Final.jar:]
              at org.infinispan.server.core.AbstractProtocolDecoder.writeResponse(AbstractProtocolDecoder.scala:183) [infinispan-server-core-5.3.0.Final.jar:5.3.0.Final]
              at org.infinispan.server.hotrod.HotRodDecoder.customDecodeKey(HotRodDecoder.scala:160) [infinispan-server-hotrod-5.3.0.Final.jar:5.3.0.Final]
              at org.infinispan.server.core.AbstractProtocolDecoder.decodeKey(AbstractProtocolDecoder.scala:125) [infinispan-server-core-5.3.0.Final.jar:5.3.0.Final]
              at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:73) [infinispan-server-core-5.3.0.Final.jar:5.3.0.Final]
              at org.infinispan.server.core.AbstractProtocolDecoder.decode(AbstractProtocolDecoder.scala:49) [infinispan-server-core-5.3.0.Final.jar:5.3.0.Final]
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) [netty-3.6.5.Final.jar:]
              at org.infinispan.server.core.AbstractProtocolDecoder.messageReceived(AbstractProtocolDecoder.scala:394) [infinispan-server-core-5.3.0.Final.jar:5.3.0.Final]
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88) [netty-3.6.5.Final.jar:]
              at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.5.Final.jar:]
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [rt.jar:1.6.0_45]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [rt.jar:1.6.0_45]
              at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_45]
      
      

       

      here's my jboss infinispan subsystem config:

       

      ...     
      <cache-container name="web" aliases="standard-session-cache" default-cache="repl" module="org.jboss.as.clustering.web.infinispan">
                <transport lock-timeout="60000"/>
                <replicated-cache name="repl" mode="ASYNC" batching="true" remote-timeout="30000" start="EAGER">
                          <remote-store cache="usertable" socket-timeout="60000" tcp-no-delay="true" fetch-state="false" preload="true" purge="false" shared="true"> 
                                    <remote-server outbound-socket-binding="remote-store-hotrod-server1"/> 
                                    <remote-server outbound-socket-binding="remote-store-hotrod-server2"/> 
                          </remote-store> 
                </replicated-cache>
                ...
      </cache-container>
      ...
      <socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
                ...
                <outbound-socket-binding name="remote-store-hotrod-server1"> 
                          <remote-destination host="172.17.254.231" port="11222"/> 
                </outbound-socket-binding>
                <outbound-socket-binding name="remote-store-hotrod-server2"> 
                          <remote-destination host="172.17.254.232" port="11222"/> 
                </outbound-socket-binding>
      </socket-binding-group>
      ...
      
      

       

       

      then my infinispan config:

      ...
      <subsystem xmlns="urn:infinispan:server:core:5.3" default-cache-container="clustered">
                <cache-container name="clustered" default-cache="default">
                          <transport executor="infinispan-transport" lock-timeout="60000"/>
                          ...
                          <distributed-cache name="usertable" mode="SYNC" owners="1"  start="EAGER">
                          </distributed-cache>
                </cache-container>
                ...
      </subsystem>
      ...
      

       

       

       

      I realized that Jboss AS 7.2.0.Final uses infinispan 5.2.1.Final but my infinispan data grid version is 5.3.0.Final, therefore I changed the infinispan  version to  5.2.1.Final and started a infinispan cluster with startServer.sh. Unluckily, I got the same exception.

       

       

      Here's some documents or discussions I found for reference:

      https://docs.jboss.org/author/display/ISPN/Cache+Loaders+and+Stores#CacheLoadersandStores-Remotecacheloader

      https://community.jboss.org/thread/197216

       

       

      I looked through the forum and stackoverflow for some clue, ended up with nothing else.

       

       

      Any ideas or any advice about using RemoteCacheStore are appreciated.