Infrequent synchronizing with the database incompatible with

[ari1974]

We are attempting to both:
1. Effectively cache data in our container managed entities, such that our users receive a rapid response to their requests
2. Occasionally synch the cached entities with the database to support the infrequent case where we need to make alterations by hand to our data.

We are employing <commit-option>D</commit-option> with an <optiond-refresh-rate>3600</optiond-refresh-rate> to synch the entity cache with the database every hour. We are running on jboss 3.2.4.
Because of the manner in which the optiond-refresh-rate was implemented, this is proving to be completely incompatible with effective caching.
I am hoping that there is a solution that will allow us to infrequently synchronize the information in cached entities with the data in our database.
Please respond if there is.

Here is the description jboss gives of option d in AdminDevel_323.pdf:
{
D, is a JBoss specific feature which is not described in the EJB
specification. It is a lazy read scheme
where bean state is cached between transactions as with option A, but the
state is periodically resynchronized
with that of the persistent store. The default time between reloads is 30
seconds, but may
configured using the optiond-refresh-rate element.
}

Based on my black box experimentation, this is how JBoss actually implements option D:
When you start the server, JBoss kicks off a process that removes all the beans from cache at an interval specified by the refresh rate. So if you start your server at 9:20 A.M. with an optiond-refresh-rate of an hour, any entities you cache at 10:19 A.M. will be removed from the cache at 10:20 A.M. The next removal of all entities from cache will occur at 11:20 A.M., and so on.
Our issue is that we have a ton of data in the database, but very little of it is frequently used. And that data which is used tends not to be shared between users.
If optionD left the entities in cache, but caused them to resynchronize with the persistent store as described, that would be perfectly adequate to our needs.
The vast majority of user requests would be against cached data, as the hourly synch of cached entities with the database would be unlikely to occur at the time a user was requesting that specific data. And after the synch, the information would be immediately accessible for any user request.
The current optiond implementation guarantees that each user request that accesses data which hasn't been used for an hour will result in a cache miss.
As this is true for the majority of requests in our system, it is highly problematic for us. By removing all the cached entities at the refresh interval, rather than synching those entities with the database, the jboss implementation prevents us from taking advantage of long running usage patterns.

I'm hoping that our particular container cache configuration is what is causing the abberrant optionD behavior.
Is there another policy or set of options, besides the one we are using (given below), that would resynchronize the cache with the database rather than destroying it?
I know from testing that optionD is completely ignored when using org.jboss.ejb.plugins.NoPassivationCachePolicy, so that doesn’t work because it doesn’t synch the entity cache with the database.
Removing all the entities from cache seems like it doesn't match the documented behavior, and, for example, it means max-bean-age is completely ignored if it is over the refresh period.
Here are our current settings:

<container-configuration extends="Standard CMP 2.x EntityBean">
<container-name>CMP Cache Config</container-name>

<container-cache-conf>
<cache-policy>org.jboss.ejb.plugins.LRUEnterpriseContextCachePolicy</cache-policy>
<cache-policy-conf>
<min-capacity>50</min-capacity>
<max-capacity>1000000</max-capacity>
<overager-period>1200</overager-period>
<max-bean-age>86400</max-bean-age>
<resizer-period>1200</resizer-period>
<max-cache-miss-period>60</max-cache-miss-period>
<min-cache-miss-period>1</min-cache-miss-period>
<cache-load-factor>0.75</cache-load-factor>

</cache-policy-conf>
</container-cache-conf>
<commit-option>D</commit-option>
<optiond-refresh-rate>3600</optiond-refresh-rate>
</container-configuration>


Please let me know if there are other options available in JBoss that might meet our goals of occasionally synching with the persistent store, and still maintaining large numbers of cached entities over long periods of time.

[aloubyansky]

I don't see much sense in what you describe. You could try cache invalidation framework. Again, it will evict instances from the cache at invalidation time.

[ari1974]

Invalidating the object and removing it from cache is precisely the issue. As to the sense in what I describe -- the whole point of employing an entity cache is for user visible performance improvements. The current method employed with optionD, removing all entities from cache, has severe performance implications. It guarantees the user requests will be run against uncached data after all the entities are invalidated. This is contrary to the documentation for optionD, which explicitly promises that cached bean state is resynchronized with the persistent data store. There is a significant difference between resynchronizing cached state, and removing all cached state. In the first case, the application server, transparent to user requests, updates the cache. The time taken to resynchronize is not visible to the users. In the second case, you guarantee that for the next user request, the application server will have to go the persistent data store to obtain information. Thus, the response to user requests is much, much slower.

Hopefully, the distinction I'm making at least makes sense now. The current option D implementation has severe consequences for the behavior of our application. While optionD is not a J2EE standard, my guess is that the need for an appropriate combination of caching and resynching with the data store is a common need for application developers. Given that, I would assume other application servers have a different implementation of similar functionality. I'll look at the behavior of other application servers in this regard. If they all behave the same way, I'll assume that what I'm describing is not a common need, or is extraordinarily difficult to implement. Either way, I'll post my results.