-
1. Re: Hibernate/Cache conceptual mismatch
brian.stansberry Aug 18, 2006 10:32 AM (in response to sebersole)Is there any time when a write lock on the cache is necessary?
To express the question more clearly, in the Hibernate use case is there a need for locking semantics on the cache stronger than READ_UNCOMMITTED? If not, perhaps READ_UNCOMMITTED would work.
It wouldn't work if the contention you're talking about is between writes. -
2. Re: Hibernate/Cache conceptual mismatch
sebersole Aug 19, 2006 8:50 AM (in response to sebersole)What I am saying is that there is two usage patterns which force Hibernate to do puts into the cache. But to JBossCache, a put is a put is a put and as such always receives a write lock; it does not distinguish the purpose or the context of the put.
(1) Hibernate loads data from the database and perfoms a put into the cache. This is the one discussed before. This is logically just a read. We are not putting a change of system state into the cache. Thus this should be a read lock to minimize contention.
(2) User, via Hibernate, modifies some data requiring Hibernate to do a put into the cache to store the new state. This should be write locked. -
3. Re: Hibernate/Cache conceptual mismatch
brian.stansberry Aug 19, 2006 10:30 AM (in response to sebersole)OK, that's clear. Figured it couldn't be that easy, but had to ask :)
Not sure what you mean about the loader idea as locking and cache loading are separate aspects, but in any case the load semantic in the API sounds cleaner. Manik can better comment on this, but there's a big drive in 2.0 to reduce the number of methods in the API, so an alternative is some kind of Option to only use the read lock. -
4. Re: Hibernate/Cache conceptual mismatch
ben.wang Aug 21, 2006 12:59 AM (in response to sebersole)I think what you describe is a mismatch indeed. From Cache point of view, when you deposit any data, if there is concurrency, then some kind of lokc is necessary. So I think what you are looking for is lock level that can be changed per API call. Something like Brian suggested to put into Option.
However, it can be tricky though. Just imagine that I have /a/b, and /a/c. If node "b" and "c" have different lock levels during a tx, then there can be data integrity issue.
Steve, is this mismatch causing deadlock or performance slowdown? -
5. Re: Hibernate/Cache conceptual mismatch
sebersole Aug 21, 2006 8:27 AM (in response to sebersole)Highly likely.
Consider 2 different Hibernate Sessions performing the same load concurrently:
session.load( MyEntity.class, new Long(1) );
And assume that MyEntity#1 is not currently in the cache. So what happens?
Well, in response to the load from the first session Hibernate will load the data from the database and then perform the put into the cache, acquiring: (1) a read lock in the database and (2) a write lock in the cache.
Now, the second load request gets processed. So Hibernate checks the cache to see if the data exists there. However, we have previously acquired a write lock in the other session so this process is blocked until that write lock is released when the first session ends its transaction. -
6. Re: Hibernate/Cache conceptual mismatch
ben.wang Aug 21, 2006 12:53 PM (in response to sebersole)For the case of querying/loading, if Cache does not participate in the tx, then I think it wont probably get deadlock and the response time will be much faster as well? Am I correct?
-
7. Re: Hibernate/Cache conceptual mismatch
manik Aug 22, 2006 4:55 AM (in response to sebersole)
Now, the second load request gets processed. So Hibernate checks the cache to see if the data exists there. However, we have previously acquired a write lock in the other session so this process is blocked until that write lock is released when the first session ends its transaction.
I thought you use the fail silently Option when interacting with the cache? This will prevent the cache from participating in any ongoing tx so the write lock will only exist for the duration of the cache.put() method call, not the entire transaction.
Agreed about a read-lock-only option when calling a put though, this will helpimprove concurrency a good deal more.
Curious, how do the likes of EHCache, etc. deal with this? I don't believe they have 2 different put() methods... -
8. Re: Hibernate/Cache conceptual mismatch
otaylor Sep 18, 2006 8:50 AM (in response to sebersole)Wanted to pick up this old thread and add a few observations I've made:
EHCache and similar are working on an entirely different code path ... the CacheConcurrencyStrategy is ReadWriteCache rather than TransactionalCache; ReadWriteCache is a pretty complex scheme, but the basic idea of it is that it stores lock objects in the cache that wrap the actual cached objects and keeps lock state and timestamps in the lock objects.
AFAIK the two TreeCache based caches are the only users of TransactionalCache.
Yes, the fail-fast variant of put is used currently. Issues that I'm aware of with it:
- It doesn't work with optimistic locking (just broken? doesn't make senes? not sure)
- It doesn't replicate quite properly (the "fail-silently" option isn't replicated)
- It isn't handled for INVALIDATE_SYNC at all right (The 0-timeout isn't propagate)
- It's not 100% correct
To expand on the last, the problem is that you can have:
a) get() returns null
b) new data or delete comimtted in database and cache
c) put() overwrites the newer data
I'd argue that get() needs to establish a hard read lock (even when the node isn't there, even if the isolation mode is READ_COMMITTED) that is upgraded to a write lock on put() and then released/downgraded afterwards. (Doing the get()/put() pair in their own transaction might make sense, just as the fail-fast put is done in its own transaction now)
This would likely require some Hibernate <=> cache changes to make sure that a failed get() is always followed by another call (a put() on success, but the item might not have been in the database, or there could have been other failures.)
The above is of course somewhat similar to what the cache loader infrastructure does, but wedging into that would be more invasive on the Hibernate side.
Anyways, I don't want to go into a lot more detail here, but just wanted to point out that dealing only with the put() and not considering the get()/put() cycle has some issues. -
9. Re: Hibernate/Cache conceptual mismatch
manik Sep 19, 2006 7:39 AM (in response to sebersole)putFailFast() is a kludge that was introduced in earlier versions of JBC, and is deprecated.
The correct strategy for achieving this behaviour - regardless of node locking scheme - is to set an Option - FAIL_SILENTLY. -
10. Re: Hibernate/Cache conceptual mismatch
otaylor Sep 20, 2006 9:18 AM (in response to sebersole)I was just using "fail-fast" as a shorthand for the whole set of behaviors that are used now:
- Lock temporarily, not part of the current transaction
- Time out immediately on lock failure
- Don't throw an exception on lock failure
I'm not sure I buy that these are all really independent, especially when have additional special lock handling as discussed here. But I didn't mean to imply that putFailFast was actually used. -
11. Re: Hibernate/Cache conceptual mismatch
otaylor Sep 25, 2006 8:29 PM (in response to sebersole)Hmmm, dug around in this some more and it turns out that (the non-optimistic) Hibernate TreeCache glue *is* still literally using putFailFast, even in latest CVS. Looking at the details of the behavior bundle, I'm not sure it could be switched over without some adiditional work.
Isolated from parent transaction:
putFailFast: Doesn't provide, Hibernate suspends its transaction itself before calling into the TreeCache. (There's a bug here, the suspended transaction leaks onto the invocation context used for putFailFast, so putFailFast creates its locks owned by the suspended transaction!)
FAIL_SILENTLY: Provides automatically
Replicates asynchronously even if the cache is REPL_SYNC
putFailFast: Provides (hacked into the ReplicationInterceptor)
FAIL_SILENTLY: Doesn't provide. Maybe could be a separate option?
Lock failure immediate without waiting for a lock timeout:
putFailFast: Provides
FAIL_SILENTLY: Doesn't provide (??? Am I missing something?)
Lock failure ignored:
putFailFast: Doesn't provide; for local failure, caller can catch TimeoutException. failure on (async) replication is logged at level ERROR.
FAIL_SILENTLY: Provides, sort of. Except for the one particular case of JBCACHE-767 (now) silent isn't very silent... it logs at level INFO.
For the short term I'm going to see if I can come up with fixes for the two putFailFast bugs noted above (leak of locks onto suspended transaction, ERROR logging on replication). -
12. Re: Hibernate/Cache conceptual mismatch
otaylor Sep 26, 2006 10:33 AM (in response to sebersole)JIRA references for the two bugs:
JBCACHE-785: InvocationContext and suspended transactions
http://jira.jboss.com/jira/browse/JBCACHE-785
(I didn't come up with an easy fix. Hibernate workaround patch included)
JBCACHE-786: errors from "OK" putFailFast replications
http://jira.jboss.com/jira/browse/JBCACHE-786
(Patch included) -
13. Re: Hibernate/Cache conceptual mismatch
manik Sep 26, 2006 10:56 AM (in response to sebersole)Thanks for this stuff, Owen - will look into it soon.
-
14. Re: Hibernate/Cache conceptual mismatch
brian.stansberry Oct 10, 2006 2:38 PM (in response to sebersole)There was a brief off-line discussion on the topic of adding the ability to specify lock-types externally (maybe via Options) such that Hibernate could simply request that "this put only acquire a read lock". Here's the key point:
"Manik Surtani" wrote:
I don't think specifying lock types will work, since this is only symbolic. (i.e., with optimistic locking, locking is completely different)
Perhaps what we need, literally, is completely separate behaviour for put() when the FAIL_SILENTLY option is used -which leads me to think that this shouldn't be an option at all, but a separate put() method - perhaps putFailFast() again, but implemented quite differently, involving all the specific needs of this put(), including:
* suspended txs
* minimal locking
* 0 timeouts
* async replication
+1 on a separate method. I'm the one who raised the original Option idea, but I think now that was wrongheaded. The term "Hibernate 2nd level cache" makes the use case sound specialized, but really it's one of the basic use cases for JBC -- read from database, then write to cache to avoid future reads. Programming for that use case needs to be dead simple; having to deal with Options is too much.