11 Replies Latest reply on Mar 1, 2009 4:06 PM by memoryerror

    Why does TreeCache notify when putting data gotten by a read

    jimshowalter

      We are very happy with TreeCache and CacheLoader--the documentation for both is first-rate, and the code works perfectly in our application.

      However, there is one thing about the design that puzzles us, and we're hoping you can explain it.

      In our implementation, we currently don't use a CacheLoader. Instead, we have code that follows this pattern:

      result = null;

      try
      {
      result = cache.get(key for what we want);
      }
      catch (Exception e)
      {
      dead...
      }

      if (result == null)
      {
      try
      {
      result = database.get(key for what we want);
      }
      catch (Exception e)
      {
      dead...
      }

      cache.put(key for what we want, result); // Spurious notification sent here.
      }

      return result;
      }

      That was how we got our prototype working, but we always planned to implement our own CacheLoader to save and retrieve our data from our database, so that we wouldn't be doing the puts to the cache of what we retrieved from the database.

      The reason we didn't want to do the puts ourselves on reads from the database is we figured that would send spurious notifications to other caches listening in our cluster, at the point commented "// Spurious notification sent here.".

      We think of that notification as spurious because it wasn't caused by a change to the data--it was caused simply by reading some data and putting it in the cache, and why would other listeners in the cluster care about a non-data-changing event? (They can fetch the data themselves if they need it, and if they don't need it, why would they want to be notified that another cache in the cluster happened to want it?)

      We figured when you call a CacheLoader, you do a put-*without*-notification to the cache. We figured put-without-notification is a hidden method, because it's not in the TreeCache API (and if it was, we would have called it instead of the standard put).

      So last week we dug into the CacheLoaderInterceptor code, and discovered that you don't call a special put-without-notification method. Instead, you do this:

      private NodeSPI loadNode(InvocationContext ctx, Fqn fqn, NodeSPI n, TransactionEntry entry) throws Exception
      {
      if (trace) log.trace("loadNode " + fqn);
      Map nodeData = loadData(fqn);
      if (nodeData != null)
      {
      if (trace) log.trace("Node data is not null, loading");

      cache.getNotifier().notifyNodeLoaded(fqn, true, Collections.emptyMap(), ctx); // Spurious notification sent here.
      if (isActivation)
      {
      cache.getNotifier().notifyNodeActivated(fqn, true, Collections.emptyMap(), ctx); // Spurious notification sent here.
      }

      n = createNodes(fqn, entry);
      // n.clearDataDirect();
      n.setInternalState(nodeData);

      // set this node as valid?
      if (usingOptimisticInvalidation) n.setValid(true, false);

      cache.getNotifier().notifyNodeLoaded(fqn, false, nodeData, ctx); // Spurious notification sent here.
      if (isActivation)
      {
      cache.getNotifier().notifyNodeActivated(fqn, false, nodeData, ctx); // Spurious notification sent here.
      }
      }

      if (n != null && !n.isDataLoaded())
      {
      if (trace) log.trace("Setting dataLoaded to true");
      n.setDataLoaded(true);
      }
      return n;
      }

      Why do you notify other caches in a cluster when data is loaded from the database?

        • 1. Re: Why does TreeCache notify when putting data gotten by a
          mircea.markus

          notification and replication are different things.
          Notification is made only locally.

          cache.getNotifier().notifyNodeActivated(fqn, true, Collections.emptyMap(), ctx); // Spurious notification sent here.

          this code will notify all CacheListener that a certain node is loaded, and all the cache listeners are local to the cache Vm/Instance.
          I suggest reading CacheListener documentation.

          When doing a cache.put on the other hand, the operation is replicated in all cache nodes.

          • 2. Re: Why does TreeCache notify when putting data gotten by a
            jimshowalter

            Thank you for the clarification. We will implement a CacheLoader now that we understand the distinction.

            • 3. Re: Why does TreeCache notify when putting data gotten by a
              jimshowalter

              Is there a put method we can call that doesn't cause replication? If so, we would not have to implement a CacheLoader, which would save a fair amount of time.

              • 4. Re: Why does TreeCache notify when putting data gotten by a
                mircea.markus

                 

                cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
                cache.put("/a/b/c", "key", "value"); //this won't get replicated


                take a look in docs at 'options' api.


                • 5. Re: Why does TreeCache notify when putting data gotten by a
                  jimshowalter

                  Very cool--thanks!

                  • 6. Re: Why does TreeCache notify when putting data gotten by a
                    jimshowalter

                    However, we also do puts to the cache that we want to replication. What we really want is:

                    if put due to read from database
                    setCacheLModeLocal(true)
                    else
                    setCacheModeLocal(false)
                    end if

                    But with puts to the cache happening in multiple nodes, and puts due to reads happening in other nodes, won't setCacheModeLocal affect all of them at once? We need it to be selective on individual notes, for single puts.

                    cache.putWithoutReplicate(...) would be really useful.

                    • 7. Re: Why does TreeCache notify when putting data gotten by a
                      mircea.markus

                       

                      cache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
                      affects only the immediate call Does not affect calls from other threads.

                      • 8. Re: Why does TreeCache notify when putting data gotten by a
                        jimshowalter

                        We have some followup questions. We are writing unit tests that operate out of container and with the TreeCache not clustered.

                        When we create the TreeCache locally for out-of-container testing, we call setCacheMode(TreeCache.LOCAL):

                        TreeCache theTreeCache = new TreeCache();
                        theTreeCache.setCacheMode(TreeCache.LOCAL);
                        theTreeCache.createService();
                        theTreeCache.startService();

                        When we're in a call, we need to be able to determine whether we should be sending replication notifications or not. So our code looks like this:

                        public final void cache(Object somethingToCache, ReplicationMode replicationMode) throws ServiceException
                        {
                        TreeCache treeCache = mTreeCache;
                        Fqn typePath = MetadataCachePaths.typePathFor(somethingToCache);

                        if ((replicationMode == ReplicationMode.Replicate) && (treeCache.getCacheModeInternal() != 1))
                        {
                        treeCache.getInvocationContext().getOptionOverrides().setCacheModeLocal(false);
                        }
                        else
                        {
                        treeCache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
                        }

                        treeCache.put(typePath, BaseTypeKey, customizedBaseType);
                        }

                        We're not sure about several things about our approach:

                        1) Does setCacheModeLocal(TreeCache.LOCAL) when we create the TreeCache set the default mode? If so, does that mean that the getOptionOverrides().setCacheModeLocal leaves the overall default mode alone, so we don't have to first save the mode, then set it in the invocation context, then set it back to the original mode?

                        2) When we create TreeCache locally, the call to getOptionOverrides().setCacheModeLocal blows up with an NPE because there are no option overrides. Are we supposed to set them ourselves when we create the local TreeCache?

                        3) Is calling getCacheModeInternal the right way to determine whether an instance of TreeCache is clustered or not? If that's not the right way to make that determination, what is?

                        4) Is there an example somewhere that shows how to use TreeCache out of container, and an example that shows how to determine if a TreeCache is clustered and how to enable/disable replication notifications based on whether it's clustered?

                        • 9. Re: Why does TreeCache notify when putting data gotten by a
                          jimshowalter

                          Typo. The code example should have been:

                          public final void cache(Object somethingToCache, ReplicationMode replicationMode) throws ServiceException
                          {
                          TreeCache treeCache = mTreeCache;
                          Fqn typePath = MetadataCachePaths.typePathFor(somethingToCache);

                          if ((replicationMode == ReplicationMode.Replicate) && (treeCache.getCacheModeInternal() != 1))
                          {
                          treeCache.getInvocationContext().getOptionOverrides().setCacheModeLocal(false);
                          }
                          else
                          {
                          treeCache.getInvocationContext().getOptionOverrides().setCacheModeLocal(true);
                          }

                          treeCache.put(typePath, BaseTypeKey, somethingToCache);
                          }

                          • 10. Re: Why does TreeCache notify when putting data gotten by a
                            memoryerror

                            Is there a way to tell the cache to only send notifications for certain operations, period?

                            We were able to turn cacheModeLocal on and off as needed based on whether an operation was supposed to notify other caches in a cluster, and on whether the cache is in a cluster.

                            But now we have a new problem: suppose a sequence of calls is made to the cache, all under one transaction. Some of those calls result in local faults, reads from the database, and local puts--which are of no interest to other caches--but other calls are caching of new data that is of interest to other caches.

                            If we set cacheModeLocal(true) for the first kind of operation and false for the second kind of operation, but both kinds of operations are performed on the same thread under the same transaction, then the final state of cache mode local is unpredictable, and it can't be right all the time for either kind of operation (either we wind up sending unwanted chatter to other caches in the cluster, or we wind up not sending necessary notifications to other caches in the cluster).

                            What we really want is to be able to say is notify only for these puts and not for these other puts, and have the setting for the two kinds of puts "stick".

                            • 11. Re: Why does TreeCache notify when putting data gotten by a
                              memoryerror

                              Or is this question moot if we port to 2.x and call putForExternalRead for the local puts we don't want to notify other caches about?