-
1. Re: Remote Store of Infinispan
nadirx Mar 21, 2012 10:18 AM (in response to ranga033)1.) The embedded Infinispan of AS7 (say cache container name - cluster) has a remote store configured.. Can this remote store be a Remote Server having clusters of Infinispan nodes (An Independent Cache Server )?
The remote store must be a (cluster of) Infinispan HotRod server(s).
2.) If case 1 is possible ( or atleast with a Remote Store as JDBC) , suppose 2 instances of AS7 are using this configuration and instance 1 goes down or has updated its info, will only Invalidate cache make instance 2 to discard its cache (in memory) and fetch it from Remote store ?
Could you rephrase the question please ?
Tristan
-
2. Re: Remote Store of Infinispan
ranga033 Mar 22, 2012 2:55 AM (in response to nadirx)Thanks Tristan!
Regarding 1.) You mentioned that Remote store must be a Infinispan HotRod server(s). So when the AS7 servers come up (each server is part of a cluster) , how do I ensure they connect to this remote store, by default I see that in domain.xml it points to the local cache containers, there is not configuration to connect to this Remote store ? Is it possible?
Rephrasing 2
2.) Assuming I have 3 nodes in a AS7 cluster. If all 3 nodes always connect to the Remote store (which is a cluster or Infinispan HotRod Servers) with no local cache , then fine. The cluster cache is available in a remote store and each of nodes can access it. If this is supported, I get the answer what I am looking for.
In case the above does not work, then I have local cache in each of the 3 AS7 server instances. The local cache has remote store configured(say JDBC or Infinispan HotRod servers). When AS7 server instance1 updates info in its local cache and subsequently stores this in the Remote Store, how do the other AS7 server instances update their local cache. Is this achieved only when the local cache is configured as an Invalidate cache.
Hope it is clearer now.
Thank you !
-
3. Re: Remote Store of Infinispan
nadirx Mar 22, 2012 9:05 AM (in response to ranga033)1.) You mentioned that Remote store must be a Infinispan HotRod server(s). So when the AS7 servers come up (each server is part of a cluster) , how do I ensure they connect to this remote store, by default I see that in domain.xml it points to the local cache containers, there is not configuration to connect to this Remote store ? Is it possible?
This configuration snippet shows how to connect to a remote HotRod store:
<distributed-cache name="default" mode="SYNC" owners="2" remote-timeout="30000" start="EAGER">
<remote-store cache="default" socket-timeout="60000" tcp-no-delay="true" fetch-state="false" preload="true" purge="false" shared="true">
<remote-server outbound-socket-binding="remote-store-hotrod-server"/>
</remote-store>
</distributed-cache>
You also need to add an outbound socket binding to the <socket-binding-group> element:
<outbound-socket-binding name="remote-store-hotrod-server">
<remote-destination host="192.168.1.10" port="11222"/>
</outbound-socket-binding>
2.) Assuming I have 3 nodes in a AS7 cluster. If all 3 nodes always connect to the Remote store (which is a cluster or Infinispan HotRod Servers) with no local cache , then fine. The cluster cache is available in a remote store and each of nodes can access it. If this is supported, I get the answer what I am looking for.
If you configure everything as in the example above you should have what you need:
- a backend cluster of hotrod servers using whatever type of clustering (repl or dist)
- a frontend cluster of AS7 instances with a distributed cache backed by the HotRod cluster (note the shared="true" option on the <remote-server /> element)
- if one of your frontend nodes updates an item it is propagated to the other frontend nodes via distribution and also sent to the HotRod cluster (for "persistence")
- if one of your frontend nodes dies, you get HA
I hope this answers your question.
Tristan
Tristan
-
4. Re: Remote Store of Infinispan
ranga033 Mar 24, 2012 9:40 AM (in response to nadirx)Thanks Tristan for the detailed reply, I will test this config and get back !
-
5. Re: Remote Store of Infinispan
ranga033 Apr 4, 2012 8:03 AM (in response to nadirx)Tristan, sorry, I got held up on something else. I just checked this today. Does this configuration need adding of any hotrod client jars in AS7 ? (if so where do I need to add them)
Getting few errors :-
[Server:server-one] 17:00:00,889 ERROR [org.jboss.msc.service.fail] (pool-13-thread-1) MSC00001: Failed to start service jboss.infinispan.web.repl: org.jboss.msc.servic
e.StartException in service jboss.infinispan.web.repl: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.
start() on object
[Server:server-one] at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:62) [jboss-as-clustering-common-7.1.1.Final.jar:7.1.1.Final]
[Server:server-one] Caused by: org.infinispan.CacheConfigurationException: Unable to instantiate class org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFacto
ry
[Server:server-one] Caused by: java.lang.ClassNotFoundException: org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory
-
6. Re: Remote Store of Infinispan
ranga033 Apr 6, 2012 8:30 AM (in response to ranga033)I could resolve this problem by adding the following :-
<module name="org.infinispan.client.hotrod"/>
in %JBOSS_HOME%/modules/org/infinispan/main/module.xml
So this is a issue with JBoss (AS7.1.1.FINAL) /Infinispan , since it did not work out of the box ?
-
7. Re: Remote Store of Infinispan
ranga033 Apr 9, 2012 3:25 AM (in response to nadirx)I have few questions on this ..
Whats the purpose of remote store when the inbuilt caches (within AS7) are already in cluster ?
Can I use a remote-store with two local caches (which are inbuilt in AS7). Will the local caches always lookup to the Remote store?
-
8. Re: Remote Store of Infinispan
ranga033 Apr 10, 2012 2:12 AM (in response to ranga033)Tristan, this is the scenario I am trying :-
1.) Have a JBoss cluster with 2 nodes (each using local web cache ) connected to a remote-store.
2.)Deployed a web application (replication-test war app)
3.) Request 1 was processed by Node1. ( Node 1 has a local web cache and a remote store)
4.) I brough down Node 1 and Request 2 is processed by Node 2 now. I was expecting this to be in a same session (My assumption was the local cache in Node 2 would get the session details from Remote-store.) But this was not happening. The request is treated as a new session.
Hence my question if local caches with a common remote store would help in failover as mentioned above ? If yes could you give me some pointers on how to debug this issue? -
9. Re: Remote Store of Infinispan
nadirx Apr 11, 2012 4:57 AM (in response to ranga033)Ah, no. You need to use replicated and not local.
-
10. Re: Remote Store of Infinispan
sonalm13 Dec 3, 2013 11:20 AM (in response to nadirx)Hi Tristan,
I tried the same configuration that you have mentioned. But when trying to start the server with configuration i am getting following error:
ERROR [org.jboss.msc.service.fail] (MSC service thread 1-4) MSC00001: Failed to start service jboss.infinispan.clustered.test-disc-cache: org.jboss.msc.service.StartException in service jboss.infinispan.clustered.test-disc-cache: Failed to start service
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1767) [jboss-msc-1.0.4.GA.jar:1.0.4.GA]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_40]
at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_40]
Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
at org.infinispan.CacheImpl.start(CacheImpl.java:666)
at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:551)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:514)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:396)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:410)
at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:89)
at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:80)
at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1811) [jboss-msc-1.0.4.GA.jar:1.0.4.GA]
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1746) [jboss-msc-1.0.4.GA.jar:1.0.4.GA]
... 3 more
Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders
at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_40]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_40]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_40]
at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_40]
at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:183)
... 18 more
Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not fetch transport
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:291)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.getTransport(TcpTransportFactory.java:165)
at org.infinispan.client.hotrod.impl.operations.FaultTolerantPingOperation.getTransport(FaultTolerantPingOperation.java:25)
at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:45)
at org.infinispan.client.hotrod.impl.RemoteCacheImpl.ping(RemoteCacheImpl.java:433)
at org.infinispan.client.hotrod.RemoteCacheManager.ping(RemoteCacheManager.java:632)
at org.infinispan.client.hotrod.RemoteCacheManager.createRemoteCache(RemoteCacheManager.java:613)
at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:524)
at org.infinispan.client.hotrod.RemoteCacheManager.getCache(RemoteCacheManager.java:520)
at org.infinispan.persistence.remote.RemoteStore.start(RemoteStore.java:89)
at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:122)
... 23 more
Caused by: org.infinispan.client.hotrod.exceptions.TransportException:: Could not connect to server: /w.x.y.z:11222
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:74)
at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:35)
at org.infinispan.client.hotrod.impl.transport.tcp.TransportObjectFactory.makeObject(TransportObjectFactory.java:16)
at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransportFactory.borrowTransportFromPool(TcpTransportFactory.java:287)
... 33 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_40]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735) [rt.jar:1.7.0_40]
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:117) [rt.jar:1.7.0_40]
at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.<init>(TcpTransport.java:65)
... 37 more
Can you suggest what could be the problem here?
Thanks,
Sonal
-
11. Re: Remote Store of Infinispan
nadirx Dec 3, 2013 11:46 AM (in response to sonalm13)Well, looking at that log,
is a hotrod server actually listening on w.x.y.z:11222 ?
Tristan
-
12. Re: Remote Store of Infinispan
sonalm13 Dec 4, 2013 1:10 AM (in response to nadirx)Ya, i got this in the log before getting the above mentioned error:
[org.infinispan.server.endpoint] (MSC service thread 1-15) JDGS010001: HotRodServer listening on w.x.y.z:11222
Sonal
-
13. Re: Remote Store of Infinispan
nadirx Dec 4, 2013 6:09 AM (in response to sonalm13)Sonal can you post the configuration of both servers on pastebin or similar please ?