Externalizeable versus Serializeable For Cache Loaders

[jiwils]

In my application there are two non-cache loader cache instances (plus my application layer) and a single cache loader instance (without my application layer).

With this setup, my "application layer" does not seem to be adversly affected with the addition of the cache loader (as it was in the unshared cache loader case in conjunction with my application layer).

However the cache loader instance seems to lag behind (by quite a bit), maybe hours. I am not using TreeCacheAop, because my objects really only have one member, and I am not updating any objects "in place". Would implementing the externalizeable interface make much difference in cache loader performance versus the serializeable implementation?

[belaban]

sync or async repl ? Can you describe this a bit more in detail ?

[jiwils]

sync or async repl ? Can you describe this a bit more in detail ?

I will do my best...

My application, for testing purposes, consists of a single client, an application instance, and a cache loader instance. The client talks to the application instance that fronts the cache, and the cache loader instance contains just the cache configured to utilize a cache loader. Replication between the cache instances happens asynchronously. The reason that the application instance and cache loader instance are seperate is the use of a cache loader on the application instance had a large impact to performance.

The application uses two top-level nodes in the cache, one "/hash" node and one "/root" node. The /hash node's map contains a hash of all of the other locations contained as decendents of the /root node. I think that part of my problem is that as the /root tree grows, the map of the /hash node grows as well (and can get quite large).

When using the Sleepycat cache loader, the CPU utilization of the cache loader instance is extremely high (90+%), and persistence of cache data (viewed by watching ReplicationInterceptor TRACE) lags way behind. If I use the File-based cache loader, the lag is greatly diminished and the CPU utilization is less, but the issue still remains.

My question is related to how might I improve on this lag. Could I create a customized File-based cache loader so that it only writes the /hash node's changes periodically? Could I try to change my objects to externalizeable from serializeable? Cache-to-cache transient state transmission seems rather fast. Could I perdiodically use that method for persistence instead of the single changes captured by the cache loader? Any other thoughts or impressions?

[belaban]

Yes, Externalizable will help. Another solution is to create a CacheLoader that sits in front of other CacheLoaders, and essentially acts as a queue for requests. We have this on the roadmap as 'Asynchronous CacheLoader'.
Note that this reduces reliability though, but you should be okay as you use async repl anyway.