Large node set population performance

[ben_ben]

Hi,

We're using Jboss Cache to cache the details of a large number of users (~50,000). Once populated the cache performance is fine and there are no problems. However, populating the cache in the first place is an extremely time-consuming process.

Data is gathered through a JDBC call and takes approximately 2 seconds to create 1000 objects to cache. As the result set is processed it takes the same amount of time to process the last 1000 as it did the first 1000. The problem comes when populating the cache. Adding the objects to the cache takes progressively longer, the first 1000 objects takes around 5 seconds but by the time it gets up to loading the 20th set of 1000 objects it takes close to a minute. The problem gets progressively worse the more objects are added to the cache.

How can I increase the loading performance? Should I be arranging the nodes in the TreeCache in a certain way? Are there any relevant settings to play about with?

Thanks

Ben

[manik]

When you say populating, are you referring to loading the cache from a JDBC cache loader on cache startup?

Or are you referring to oruy own application which reads from a DB and puts stuff into the cache using Cache.put() commands?

Also, which version of JBC are you referring to?

And finally, how have you organised your data? I.e., how wide/deep is your tree,a nd how many key/value pairs do you store in each node?