8 Replies Latest reply on Jul 16, 2008 7:07 AM by manik

    Config Question

    lovelyliatroim

      Just looking for some confirmation on some config questions, i.e what the best config is for my server.

      TransactionManager.
      do i need to configure a transaction manager even though i dont have the need for transactions? ( I assume the answer is no but thought id ask anyways, if yes then please why?)

      If no transaction manager then the isolation should be "NONE", correct??


      Thanks,
      LL

        • 1. Re: Config Question
          manik

          The TM should only be used if either:

          1. You have a need for JTA transactions in your application and you need the cache to participate in these transactions.
          2. You are using OPTIMISTIC locking.

          If either of the above are true, you should use a TM.

          There is a third argument to use a TM, and this is to batch up modifications. E.g., multiple calls to cache.put() will result in multiple replication events but if you wrap these in a transaction, then replication occurs when the transaction commits, as a single message, which could be more efficient.

          Isolation level applies to in-memory locking as well so it is relevant even if you are not using a TM.

          • 2. Re: Config Question
            lovelyliatroim

            Forgot something

            I see also alot of these print outs for my cache using the ExpirationPolicy


            14:09:53 [EvictionTimer-0] WARN eviction.ExpirationAlgorithm - Unable to remove nodes to reduce region size below 5000. Set expiration for nodes in this region


            Now my problem with that is that the region its talking about would only have 2-3 nodes in this region but i get the above output every time the expiration policy runs. Why is this??

            I think i kind of know the reason but cant say for sure but here is what i am doing


            I have a branch called "root/a", i configure the expiration policy for the region "root/a" to be allowed to have 5000 nodes.

            When an items are added to the branch "root/a" it can look like so
            "root/a/b/c/1"
            "root/a/b/c/2"
            "root/a/d/c/1"

            Now the expiration key is only added to the very last node, so in this case the numeric values. So when eviction kicks in and nodes are valid for eviction it will only evict the numerical nodes, so after eviction i have the following
            "root/a/b/c"
            "root/a/b/c"
            "root/a/d/c"

            Now why would i keep getting that warning message when i am not exceeding the limit configured for that region?

            Another problem i have with the above strategy for my expiration policy is that i end up with dead branches after eviction i.e the data item i need is evicted but the path to reach it lives on. So take for example

            "root/a/b/c/1"
            And the data item i want to cash is in node "1", after eviction my tree looks like so
            "root/a/b/c"

            Essentially the nodes "b" and "c" are dead wood, the data item has been evicted but the path lives on. Is there a better approach to this??

            A thought would be to use the path keys on one node level so for example
            "root/a/b-c-1"

            Which i have no problem in doing but how does the performance compare for locating an item??So you could say depth vs breath, which is the quickest for getting a dataitem??

            Any other approaches that i could use??


            Thanks Guys,
            LL





            • 3. Re: Config Question
              lovelyliatroim

               


              The TM should only be used if either:

              1. You have a need for JTA transactions in your application and you need the cache to participate in these transactions.
              2. You are using OPTIMISTIC locking.

              If either of the above are true, you should use a TM.

              There is a third argument to use a TM, and this is to batch up modifications. E.g., multiple calls to cache.put() will result in multiple replication events but if you wrap these in a transaction, then replication occurs when the transaction commits, as a single message, which could be more efficient.

              Isolation level applies to in-memory locking as well so it is relevant even if you are not using a TM.


              Thanks for that Manik.



              • 4. Re: Config Question
                manik

                Sorry for the slow response re: your eviction questions.

                The thing with the expiration policy is that it needs the expiration keys to consider nodes for removal - which includes your "structural" nodes like /root/a/b/c.

                Now if a node is considered for eviction and the node has children (e.g., /root/a/b is considered for eviction) it will not be removed, but instead it will be "emptied" of any data it may possess, freeing up memory.

                • 5. Re: Config Question
                  manik

                  Also, re: the depth vs breadth argument, depth searches are performed by walking a tree structure while breadth searches use hash buckets. So up to certain limits, breadth searches will definitely be quicker.

                  • 6. Re: Config Question
                    lovelyliatroim

                     


                    Sorry for the slow response re: your eviction questions.

                    No problem.


                    Now if a node is considered for eviction and the node has children (e.g., /root/a/b is considered for eviction) it will not be removed, but instead it will be "emptied" of any data it may possess, freeing up memory.

                    Im not fully sure of what you are saying here, what do you mean by "emptied"? Lets go back to the example

                    "root/a/b/c/1"
                    "root/a/b/c/2"
                    "root/a/d/c/1"

                    All the numerics are tagged/marked for eviction. Eviction kicks in and afterwards I have

                    "root/a/b/c"
                    "root/a/b/c"
                    "root/a/d/c"

                    I have 2 issues
                    1. I have an extra 6 obsolete nodes or dead paths sitting in memory, after a while they will all add up.
                    2. The log message occurs every time the eviction thread kicks in so every 10-15 seconds, i get lots of the above warning messages which floods my log file.

                    Now would this solution work. When i make an entry, i tag the parent root node as well with the time to expire of the child +1 minute. So child will be evicted first and then parent later on. So lets take the path "root/a/b/c/1"if i tag node "a" with (time to expire of child+1) and tag the child "1" with the expiration value. When the child is removed and the eviction policy thread comes around again and sees that the node "a" is expired, will it remove "a","b" and "c" thus removing the dead nodes?? (I should actually test this myself and see if it works)

                    I know i can filter the log level for the warning on the eviction.ExpirationAlgorithm class but dont really want to do that just in case something does go wrong.

                    Thanks Manik,
                    LL







                    • 7. Re: Config Question
                      lovelyliatroim

                       


                      So up to certain limits, breadth searches will definitely be quicker.


                      What range of limits would you be talking roughly?? If there is a benchmark great but if not what range would you expect performance to decline??

                      Thanks,
                      LL

                      • 8. Re: Config Question
                        manik

                        Adding an expiry time to intermediate nodes would help your situation, yes. The only drawback is the extra storage (one more key/value pair per node in the path).

                        Regarding ranges, this depends on what you use as Fqns and what their hashcodes are like. Basically, each node has a ConcurrentHashMap to hold all direct children. The CHM is keyed on child name (Fqn.getLastElement() for the child's Fqn). So, for example, if your Fqn elements are all Strings, then the CHM would be keyed on Strings.

                        Benchmarks here are of limited use since it really depends on the kind of Fqns you use and how well their hashcodes are spread. If you do use something generic like Strings and you do decide to benchmark it, I guess it would be useful to share.