1 2 Previous Next 26 Replies Latest reply on Jul 3, 2011 9:03 PM by kodadma Go to original post
      • 15. Re: That is just too much overhead
        kodadma

        I just tried CR6 out- the same results. It seems that when cache reaches max limit - all working threads stuck forever (or almost forever). This is reproducible even for 2 threads and 1G of heap memory (1M entries limit).

        • 16. Re: That is just too much overhead
          kodadma

          On a DataContainer interface:

           

            /**

              * Returns a set of keys in the container. When iterating through the container using this method,

              * clients should never call {@link #get()} method but instead {@link #peek()}, in order to avoid

              * changing the order of the underlying collection as a side of effect of iterating through it.

              *

              * @return a set of keys

              */

             Set<Object> keySet();

           

             /**

              * @return a set of values contained in the container

              */

             Collection<Object> values();

           

             /**

              * Returns a mutable set of immutable cache entries exposed as immutable Map.Entry instances. Clients

              * of this method such as Cache.entrySet() operation implementors are free to convert the set into an

              * immutable set if needed, which is the most common use case.

              *

              * If a client needs to iterate through a mutable set of mutable cache entries, it should iterate the

              * container itself rather than iterating through the return of entrySet().

              *

              * @return a set of immutable cache entries

              */

             Set<InternalCacheEntry> entrySet();

           

          These three methods are killers for any reasonably large off-heap caches. I think significant reengineering is required (move from Collections to iterators/scanners) in DataContainer to make integration of OffHeap storage possible.

          • 17. Re: That is just too much overhead
            kodadma

            Puts when eviction is active are very expensive. LRU.execute takes approx 95% of overall execution time of DefaultDataContainer.put() You may need to reconsider the eviction implementation for DefaultDataContainer.

            • 18. Re: That is just too much overhead
              kodadma

              How to activate new DataContainer?

               

              This code does not work.

               

              OffHeapDataContainer dc = new OffHeapDataContainer(mCache);

               

              EmbeddedCacheManager manager = new DefaultCacheManager();

               

              Configuration config = new Configuration().fluent().                                              dataContainer().                                                   dataContainer(dc).                                                         build();

               

              manager.defineConfiguration("name", config);

               

              ispnCache  = manager.getCache("name");

              • 19. Re: That is just too much overhead
                shane_dev

                FYI - What you are describing is perhaps similar to the issue we found. I'm not sure anything can be done about PUT performance resulting in eviction but if you have a 90% GET rate you are likely running into our issue.

                 

                http://community.jboss.org/thread/167778

                • 20. Re: That is just too much overhead
                  kodadma

                  Shane, I went through the ISPN code myself but no so thouroghly as you apparently. There are too many places in the code (eviction implementation is only one of them) which make me think that  religious following "right" OOP design patterns and rules is not always right approach. For example, I have no idea how to plug my own DataContainer in despite the fact that there is API call in FluentConfig which is supposed to do this, but it does not. Even if I get answer from ISPN developers on this question, the next one will definetely follow: Ok, how to ged rid of LockingInterceptor entirely? I do locking myself in my container and I do not want to create separate lock object for every key-value pair in my container (btw, it seems that lock striping is broked).    

                  • 21. Re: That is just too much overhead
                    kodadma

                    There is no way to override dataContainer settings in ISPN 5.0 CR6 as since OverrideConfigurationVisitor which extends AbstractConfigurationBeanVisitor does not implement

                     

                    public void visitDataContainerType(DataContainerType bean)

                     

                    method. I think its a bug.

                    • 22. Re: That is just too much overhead
                      kodadma

                      I have patched *OverrideConfigurationVisitor*  to allow data container config overriding and was able to run tests on ISPN 5 + OffHeapDataContainer. Preliminary results are very encouraging. I will keep this forum posted as soon as I finish the integration.

                      • 23. Re: That is just too much overhead
                        sannegrinovero
                        I have patched *OverrideConfigurationVisitor*  to allow data container config overriding and was able to run tests on ISPN 5 + OffHeapDataContainer. Preliminary results are very encouraging. I will keep this forum posted as soon as I finish the integration.

                        Hi Vladimir,

                        I can't look at the code right now but that sounds as a fix we would need to integrate in Infinispan 5 before it's final, can you please open a bug on JIRA and contribute your fix? a pull request via github would be ideal, but a "traditional" patch would be fine too.

                        Thanks,

                        Sanne

                        • 24. Re: That is just too much overhead
                          kodadma

                          Add this method to OverrideConfigurationVisitor

                           

                          {code}

                          @Override

                          public void visitDataContainerType(DataContainerType config) {

                           

                              this.dataContainerType = config;

                              this.dataContainerType.overriddenConfigurationElements.add("dataContainer");

                          }

                          {code}

                           

                          and define class member:

                           

                           

                          {code}

                          private DataContainerType dataContainerType = null;

                          {code}

                           

                          This will make DataContainer config overriding possible but for all other configs you better check the code. It seems that it is broken.

                          • 25. Re: That is just too much overhead
                            kodadma
                            • 26. Re: That is just too much overhead
                              kodadma

                              OK, I have described performance of Infinispan + Koda (OffHeap storage) integration here:

                               

                              http://designedtoscale.blogspot.com/2011/07/introducing-koda-key-code-value-in.html

                              1 2 Previous Next