ClusteredCacheLoader + LOCAL CacheMode

[lovelyliatroim]

Hi Folks,
I have set my cache mode to local and i also want to use the ClusteredCacheLoader. What i want to do, is keep my caches seperate from each other i.e not replicate but at the same time be able to ask other caches has it got data if the original doesnt have it.

I can get it to work fine in all other modes but i dont want replication or invalidation. Or maybe there is another approach to this??

I have the feeling that i cant do these 2 settings together but would like someone to confirm this and a reason why would be nice ;).

Log file doesnt give much away to me as to what the problem is and for what its worth here is the log file. Using jboss AS 5.0 beta 2 and Jboss cache 2.0.

2007-11-28 14:02:25,877 DEBUG [org.jboss.system.ServiceCreator] About to create bean: jboss.cache:service=PriceServerCache with code: org.jboss.cache.jmx.CacheJmxWrapper

2007-11-28 14:02:26,064 DEBUG [org.jboss.cache.jmx.CacheJmxWrapper] Registered in JMX under jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,064 DEBUG [org.jboss.system.ServiceCreator] Created mbean: jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,064 DEBUG [org.jboss.system.ServiceConfigurator] TransactionManagerLookupClass set to org.jboss.cache.transaction.GenericTransactionManagerLookup in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,111 DEBUG [org.jboss.system.ServiceConfigurator] IsolationLevel set to REPEATABLE_READ in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,111 DEBUG [org.jboss.system.ServiceConfigurator] CacheMode set to LOCAL in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,111 DEBUG [org.jboss.system.ServiceConfigurator] ClusterName set to JBossCache-Cluster in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,111 DEBUG [org.jboss.system.ServiceConfigurator] ClusterConfig set to [config: null] in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,158 DEBUG [org.jboss.system.ServiceConfigurator] StateRetrievalTimeout set to 20000 in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,158 DEBUG [org.jboss.system.ServiceConfigurator] SyncReplTimeout set to 15000 in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,158 DEBUG [org.jboss.system.ServiceConfigurator] LockAcquisitionTimeout set to 10000 in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,158 DEBUG [org.jboss.system.ServiceConfigurator] CacheLoaderConfiguration set to [config: null] in jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,158 WARN [org.jboss.cache.jmx.CacheJmxWrapper] MBean attribute 'CacheLoaderConfiguration' is deprecated; use 'CacheLoaderConfig'

2007-11-28 14:02:26,190 DEBUG [org.jboss.system.ServiceController] Creating service jboss.cache:service=PriceServerCache

2007-11-28 14:02:26,190 DEBUG [org.jboss.cache.jmx.CacheJmxWrapper] Constructing Cache

2007-11-28 14:02:27,299 DEBUG [org.jboss.cache.factories.InterceptorChainFactory] interceptor chain is:
class org.jboss.cache.interceptors.CallInterceptor
class org.jboss.cache.interceptors.CacheStoreInterceptor
class org.jboss.cache.interceptors.CacheLoaderInterceptor
class org.jboss.cache.interceptors.UnlockInterceptor
class org.jboss.cache.interceptors.PessimisticLockInterceptor
class org.jboss.cache.interceptors.NotificationInterceptor
class org.jboss.cache.interceptors.TxInterceptor
class org.jboss.cache.interceptors.CacheMgmtInterceptor
class org.jboss.cache.interceptors.InvocationContextInterceptor

2007-11-28 14:02:27,315 DEBUG [org.jboss.cache.CacheImpl.JBossCache-Cluster] Not using an EvictionPolicy

2007-11-28 14:02:27,315 DEBUG [org.jboss.system.ServiceController] starting service jboss.cache:service=PriceServerCache

2007-11-28 14:02:27,362 DEBUG [org.jboss.cache.transaction.GenericTransactionManagerLookup] Trying to lookup TransactionManager for JBoss, JRun4

2007-11-28 14:02:27,362 DEBUG [org.jboss.cache.transaction.GenericTransactionManagerLookup] Found TransactionManager for JBoss, JRun4

2007-11-28 14:02:27,362 DEBUG [org.jboss.cache.CacheImpl.JBossCache-Cluster] cache mode is local, will not create the channel

2007-11-28 14:02:27,362 DEBUG [org.jboss.cache.loader.CacheLoaderManager] preloading transient state from cache loader org.jboss.cache.loader.ClusteredCacheLoader@107eafc

2007-11-28 14:02:27,612 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] Error installing to Start: name=jboss.cache:service=PriceServerCache state=Create mode=Manual requiredState=Installed

java.lang.NullPointerException

at org.jboss.cache.loader.ClusteredCacheLoader.callRemote(ClusteredCacheLoader.java:90)

at org.jboss.cache.loader.ClusteredCacheLoader.get0(ClusteredCacheLoader.java:146)

at org.jboss.cache.loader.ClusteredCacheLoader.get(ClusteredCacheLoader.java:135)

at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadData(CacheLoaderInterceptor.java:530)

at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadNode(CacheLoaderInterceptor.java:408)

at org.jboss.cache.interceptors.CacheLoaderInterceptor.loadIfNeeded(CacheLoaderInterceptor.java:210)

at org.jboss.cache.interceptors.CacheLoaderInterceptor.invoke(CacheLoaderInterceptor.java:157)

at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:76)

at org.jboss.cache.interceptors.UnlockInterceptor.invoke(UnlockInterceptor.java:37)

at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:76)

at org.jboss.cache.interceptors.PessimisticLockInterceptor.invoke(PessimisticLockInterceptor.java:203)

at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:76)

at org.jboss.cache.interceptors.NotificationInterceptor.invoke(NotificationInterceptor.java:32)

at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:76)

at org.jboss.cache.interceptors.TxInterceptor.handleNonTxMethod(TxInterceptor.java:298)

at org.jboss.cache.interceptors.TxInterceptor.invoke(TxInterceptor.java:131)

at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:76)

at org.jboss.cache.interceptors.CacheMgmtInterceptor.invoke(CacheMgmtInterceptor.java:81)

at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:76)

at org.jboss.cache.interceptors.InvocationContextInterceptor.invoke(InvocationContextInterceptor.java:62)

at org.jboss.cache.CacheImpl.invokeMethod(CacheImpl.java:3939)

at org.jboss.cache.CacheImpl.get(CacheImpl.java:1441)

at org.jboss.cache.CacheImpl.get(CacheImpl.java:1415)

at org.jboss.cache.loader.CacheLoaderManager.preload(CacheLoaderManager.java:282)

at org.jboss.cache.loader.CacheLoaderManager.preloadCache(CacheLoaderManager.java:256)

at org.jboss.cache.CacheImpl.internalStart(CacheImpl.java:796)

at org.jboss.cache.CacheImpl.start(CacheImpl.java:708)

at org.jboss.cache.jmx.CacheJmxWrapper.start(CacheJmxWrapper.java:614)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)

at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)

at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)

at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)

at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)

at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:184)

at $Proxy0.start(Unknown Source)

at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:42)

at org.jboss.system.microcontainer.ServiceControllerContextAction.install(ServiceControllerContextAction.java:46)

at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)

at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:233)

at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:238)

at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:724)

at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:445)

at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:555)

at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:489)

at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:335)

at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:205)

at org.jboss.system.ServiceController.doChange(ServiceController.java:656)

at org.jboss.system.ServiceController.start(ServiceController.java:431)

at org.jboss.system.deployers.ServiceDeployer.start(ServiceDeployer.java:142)

at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:103)

at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:46)

at org.jboss.deployers.plugins.deployers.helpers.AbstractSimpleRealDeployer.deploy(AbstractSimpleRealDeployer.java:56)

at org.jboss.deployers.plugins.deployer.AbstractSimpleDeployer.commitDeploy(AbstractSimpleDeployer.java:52)

at org.jboss.deployers.plugins.deployer.DeployerWrapper.commitDeploy(DeployerWrapper.java:170)

at org.jboss.deployers.plugins.deployment.MainDeployerImpl.commitDeploy(MainDeployerImpl.java:592)

at org.jboss.deployers.plugins.deployment.MainDeployerImpl.commitDeploy(MainDeployerImpl.java:603)

at org.jboss.deployers.plugins.deployment.MainDeployerImpl.process(MainDeployerImpl.java:476)

at org.jboss.deployers.plugins.deployment.MainDeployerImpl.process(MainDeployerImpl.java:406)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:585)

at org.jboss.aop.joinpoint.MethodInvocation.invokeTarget(MethodInvocation.java:121)

at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:110)

at org.jboss.profileservice.aop.MainDeployerAspect.process(MainDeployerAspect.java:53)

at org.jboss.aop.advice.org.jboss.profileservice.aop.MainDeployerAspect_z_process_15232416.invoke(MainDeployerAspect_z_process_15232416.java)

at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)

at AOPContainerProxy$0.process(AOPContainerProxy$0.java)

at org.jboss.system.server.profileservice.ProfileServiceBootstrap.loadProfile(ProfileServiceBootstrap.java:371)

at org.jboss.system.server.profileservice.ProfileServiceBootstrap.bootstrap(ProfileServiceBootstrap.java:247)

at org.jboss.kernel.plugins.bootstrap.AbstractBootstrap.run(AbstractBootstrap.java:89)

at org.jboss.system.server.profileservice.ServerImpl.doStart(ServerImpl.java:403)

at org.jboss.system.server.profileservice.ServerImpl.start(ServerImpl.java:342)

at org.jboss.Main.boot(Main.java:210)

at org.jboss.Main$1.run(Main.java:522)

at java.lang.Thread.run(Thread.java:595)

[lovelyliatroim]

I will hazard a guess and say this is the route of my problem

Local caches don't join a cluster and don't communicate with other caches in a cluster. Therefore their elements don't need to be serializable - however, we recommend making them serializable, enabling a user to change the cache mode at any time. The dependency on the JGroups library is still there, although a JGroups channel is not started.

So a JGroups channel aint open and i assume effects my ClusteredCacheLoader???

If this is the case, any way of manually opening up the channel??

[manik]

You are correct, this has to do with a JGroups channel not being opened. Not much you can do, I'm afraid, except perhaps run in replicated mode and then remove the replication interceptor. :-/

[brian.stansberry]

Perhaps run in replicated mode but for every write set a cacheModeLocal Option? E.g.:

cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
cache.put(fqn, key, value);

[lovelyliatroim]

Ok thanks guys, dont have the time at the moment to dig deeper, just assessing at top level what can and cant be done. We thought we could use the cache standalone and also use what the ClusteredCacheLoader has to offer.

thanks for input,
LL

[lovelyliatroim]

Ok seeking some clarity on why you would use a ClusteredCacheLoader in my case.

I have seen the thread http://www.jboss.org/index.html?module=bb&op=viewtopic&t=118598

but in terms of my needs ;).

Ok running my cache in invalidation mode is a no go for me, i dont want data evicted if i update a data item on one cache node. So i cant run in invalidation mode.

replication mode without buddy replication. Doesnt make sense to use it here because all the data is replicated across all the nodes.

replication with buddy replication: This is where i am not so clear, im not so clear on how buddy replication works, i have set it up but dont know the reasoning behind why it does what it does. Ok i would have thought that you would share data with your buddy and the data would be in the same tree, not in a seperate branch i.e under /_BUDDY_BACKUP_ 0 branch. Maybe im missing some configuration setting to merge the 2 buddies data into the same tree??

If i dont have buddy replication then all the replication goes into the same tree, thats how i would have thought buddy replication would work except that you are just sharing with you and your buddy has and not with the whole cluster.

Can someone just clear up why 2 different trees and not the same as in replication without buddy replication??

p.s i using jmx console to see whats in my cache

[lovelyliatroim]

Perhaps run in replicated mode but for every write set a cacheModeLocal Option?

Kinda worked but not really, first put was surpressed i.e not replicated but the rest after it were. Im adding the nodes like so

 for(int i = 0; i <noOfItems;i++){
 Quote quote1 = new Quote(nodeName+requestCounter,"EWX","EUR");
 String path = "/quotes/"+quote1.getTreePath()+random;
 TNode root = cache.getRoot();
 TNode child = root.addChild(path);
 cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
 child.put("quote", quote1);
 requestCounter++;
 }

So i added 5 nodes, replicates the first 4, but the 5th it doesnt!! Any thoughts?

[lovelyliatroim]

Just to correct my last post, it is misleading.

The first one was suppressed and the rest wasnt. I played around with the placing of setCacheModeLocal and what i discovered was that it doesnt replicate the first one when i have code like this (not shown in first snippet because i didnt think it was a factor.

cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
 for(int i = 0; i <noOfItems;i++){
 Quote quote1 = new Quote(nodeName+requestCounter,"EWX","EUR");
 String path = "/quotes/"+quote1.getTreePath();
 TNode root = cache.getRoot();
 TNode child = root.addChild(path);
 cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
 child.put("quote", quote1);
 requestCounter++;
 }

If i remove the first line before the loop then everything replicates and if i move setCacheLOcal to the first line inside of the loop, that doesnt make a difference either. Weird ..........

[lovelyliatroim]

replication with buddy replication: This is where i am not so clear, im not so clear on how buddy replication works, i have set it up but dont know the reasoning behind why it does what it does. Ok i would have thought that you would share data with your buddy and the data would be in the same tree, not in a seperate branch i.e under /_BUDDY_BACKUP_ 0 branch. Maybe im missing some configuration setting to merge the 2 buddies data into the same tree??

So to answer my own question http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCacheBuddyReplicationDesign

They are meant to be seperate branches. Now if i have 4 nodes and i only want groups of 2 to replicate data but in the same tree/branch, i have to set up 2 different clusters with 2 nodes in it each cluster using normal replication without buddy replication or can i configure buddy replication to replicate in the same branch?? is that possible??

[lovelyliatroim]

So now im left asking myself what the hell is ClusteredCacheLoader used for??

Doesnt make sense to use it with a replicated cache, you could use it with invalidation and you cant use it with a local cache.

Then i come across this
http://www.jboss.org/index.html?module=bb&op=viewtopic&t=118598

2 Reasons are given as to why to use it.
Reason 1 doesnt make sense to me if you cant set your cache to LOCAL, to avoid the initial state transfer but if you have to run in replication or invalidation mode then this transfer has to happen or maybe i dont see the other case(s).
Reason 2 i cant comment on because i dont know what the overhead of writing to persisten store is yet!!(On my todo list ;) )

And the very last comment

The other reason for using a CCL is if you don't want any replication at all - i.e., caches are configured as LOCAL, but use a shared database as a cache loader. This way, you could use a CCL as well to prevent overloading your shared DB.

So i assume the above statement is incorrect, you cant use it when set to LOCAL or is there a case when you can?

Basically what i wanted was 2 standalone nodes without any replication/validation going on but with the ability to ask the other if they had data if it doesnt have it. Looks like its not to be...

[brian.stansberry]

Re: your loop:

root.addChild(path) is a cache write. You have to setCacheModeLocal(true) before anything that changes the state of the cache if you want to avoid any replication.

[lovelyliatroim]

Re: your loop:

root.addChild(path) is a cache write. You have to setCacheModeLocal(true) before anything that changes the state of the cache if you want to avoid any replication.

Still replicates even if i have it before it.

[brian.stansberry]

How about this?

for(int i = 0; i <noOfItems;i++) {
 Quote quote1 = new Quote(nodeName+requestCounter,"EWX","EUR");
 String path = "/quotes/"+quote1.getTreePath();
 cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
 cache.put(Fqn.fromString(path), "quote", quote1);
 requestCounter++;
}

If that works, we may have got a problem with setting an Option before using Node.addChild(Object name). The convenience API via the Cache interface provides a workaround as above, but it would be much better if this worked:

for(int i = 0; i <noOfItems;i++){
 Quote quote1 = new Quote(nodeName+requestCounter,"EWX","EUR");
 String path = "/quotes/"+quote1.getTreePath();
 TNode root = cache.getRoot();
 cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
 TNode child = root.addChild(path);
 cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
 child.put("quote", quote1);
 requestCounter++;
}

[nnnnn]

From your description, it sounds like you might want to use LOCAL cache mode and a TCPCacheLoader. Have you looked into that?

[satan3]

great