6 Replies Latest reply on Mar 26, 2009 8:06 PM by memoryerror

    putForExternalRead doesn't solve our problem

    memoryerror Newbie

      After porting to 3.0.0.GA and putForExternalRead, it turned out we hadn't read the fine print, which says nothing happens if the node is already there (not the data itself--the node containing the data--we had misread the Javadocs).

      Because we have more than one piece of data stored in a node, the node is often there even though some data is not.

      So, puts of data weren't happening when we wanted them to, and the cache was returning nulls instead of data fetched to fill the node after a cache miss.

      We realize this is by design and documented as such, but the design puzzles us. The put should succeed if the node is there but the key for the data being put is missing, and should succeed if the node and key are there but the value in the node for that key is null.

      We are, after all, trying to do a cache fill after a cache miss. A fill that doesn't fill isn't very useful.

      Furthermore, if all of the data in a node ages out of the cache due to LRU, is the data removed from the cache and the node left in place, or is the node also removed? Because if the node is not removed, then PFER wouldn't work in that case either (which makes the design even more puzzling to us).

      JBoss support's suggestion was that we mimic PFER ourselves, which was what we had originally asked for help with, but when it got to the part about suspending transactions ourselves, that's when we said we're using JPA and the JPA spec says we'll go to JPA jail if we try to muck with transactions at all while being managed by the container. So that's not an option.

      For now, we're hoping changing the isolation level to READ_COMMITTED from the default helps. If not, we'll have to lengthen the timeout.

      Ideally JBoss Cache would have a method with a better name than putFastForCacheFillFromExternalReadAfterCacheMissEvenIfNodeIsPresent, but with that semantics.

      The behavior of PFFCFFERACMEINIP would be remarkably similar to the behavior of PFER, except for modifying the first bullet in the Javadocs:

      - Only goes through if the node specified does not exist, or exists but does not have the specified key for the value being put, or exists and has the specified key for the value being put but the value is null.

      - Force asynchronous mode for replication to prevent any blocking.
      invalidation does not take place.

      - 0ms lock timeout to prevent any blocking here either. If the lock is not acquired, this method is a no-op, and swallows the timeout exception.

      - Ongoing transactions are suspended before this call, so failures here will not affect any ongoing transactions.

      - Errors and exceptions are 'silent' - logged at a much lower level than normal, and this method does not throw exceptions.

      However, it occurs to us that the folks at hibernate.org seem plenty satisfied with the current behavior of PFER, and their use case is basically the same as our use case, which gets us to wondering why they're happy and we're not.

      Is our problem due to storing multiple items per node by key? If so, we could change to storing each item in a separate node. That seemed expensive (a lot of maps), but maybe that's how this is supposed to work?

      If so, we have a followup question--currently when we invalidate data locally (to force a cache miss and cache fill on the next access to that data), we just clear the data in a node, but leave the node in place. Are we supposed to invalidate by deleting the node itself? If so, how does *that* work when multiple threads are trying to access the same node? Won't readers back up behind a write lock, thereby re-introducing the very problem we were trying to solve with PFER in the first place? (Puts due to cache misses on reads timing out because of contention for write locks.)

        • 1. Re: putForExternalRead doesn't solve our problem
          Manik Surtani Master

           

          "jshowalter" wrote:

          We realize this is by design and documented as such, but the design puzzles us. The put should succeed if the node is there but the key for the data being put is missing, and should succeed if the node and key are there but the value in the node for that key is null.


          The cache was designed to use a Node as a unit of atomicity and as such all loads, stores, locks, evicts, etc are all based on a node and not the contents of a node. Nodes should only contain "related" data such that the above operations can happen on the node as a whole.

          "jshowalter" wrote:

          Is our problem due to storing multiple items per node by key? If so, we could change to storing each item in a separate node. That seemed expensive (a lot of maps), but maybe that's how this is supposed to work?


          Yes. This is the way Hibernate does it, storing a single entity per Node (and using a dummy static "key"). Regarding the cost of this, this was addressed in JBCACHE-1082 which optimises nodes for single elements (using an internal SingletonMap where possible), so you need not worry about the performance impact too much.

          "jshowalter" wrote:

          If so, we have a followup question--currently when we invalidate data locally (to force a cache miss and cache fill on the next access to that data), we just clear the data in a node, but leave the node in place. Are we supposed to invalidate by deleting the node itself? If so, how does *that* work when multiple threads are trying to access the same node? Won't readers back up behind a write lock, thereby re-introducing the very problem we were trying to solve with PFER in the first place? (Puts due to cache misses on reads timing out because of contention for write locks.)


          Yes, you should remove the node. Regarding locks, there is a cache option - lockParentForInsertRemove - which is off by default. With this option being switched off, you don't have the problem of WL contention but you do introduce the possibility of phantom reads (but this is allowed in R_R semantics anyway).

          • 2. Re: putForExternalRead doesn't solve our problem
            memoryerror Newbie

            Thank you for the quick reply. Two followup questions (one from the original post):

            - "if all of the data in a node ages out of the cache due to LRU, is the data removed from the cache and the node left in place, or is the node also removed"?

            - "but you do introduce the possibility of phantom reads (but this is allowed in R_R semantics anyway)." We're using READ_COMMITTED. That should prevent phantom reads, shouldn't it?

            • 3. Re: putForExternalRead doesn't solve our problem
              Manik Surtani Master

               

              "jshowalter" wrote:

              - "if all of the data in a node ages out of the cache due to LRU, is the data removed from the cache and the node left in place, or is the node also removed"?


              Like I said, the unit of atomicity is a node. So, it is an entire node that times out due to LRU, not any specific key/value pair within a node. As such, eviction removes the entire node *unless* the node has children, and is needed as a structural mechanism to access those children. In the latter case, the node is emptied and flagged as being evicted, but the node itself is still left as a structural piece.

              "jshowalter" wrote:

              - "but you do introduce the possibility of phantom reads (but this is allowed in R_R semantics anyway)." We're using READ_COMMITTED. That should prevent phantom reads, shouldn't it?


              Nope. :-)

              http://api.openoffice.org/docs/common/ref/com/sun/star/sdbc/TransactionIsolation.html


              • 4. Re: putForExternalRead doesn't solve our problem
                memoryerror Newbie

                How does one go about adding a single data item to a node? Do we have to provide a key at all in that case, or can we just call put(fqn, the data item)?

                • 5. Re: putForExternalRead doesn't solve our problem
                  Manik Surtani Master

                  You'd use a key. A dummy key if it is irrelevant to you, e.g., a public static final Object DUMMY_KEY that you could define.

                  • 6. Re: putForExternalRead doesn't solve our problem
                    memoryerror Newbie

                    "That should prevent phantom reads, shouldn't it?" -- Sorry, I meant dirty reads. The way we're using the cache makes phantom reads not a problem. And read committed prevents dirty reads. So I think we're in good shape--I have a single-data-item-per-node version of the cache up and running now and am in the process of testing it.