1 2 Previous Next 25 Replies Latest reply on Apr 1, 2015 12:21 PM by gaprashanth Go to original post
      • 15. Re: How to set limit on the memory use for Cache
        kodadma

        Anybody is interesting in BigMemory -type Off Heap storage for Infinispan? You can set hard limit on a max allocated memory (exact limit). The memory is allocated off-heap so you can keep your JVM heap low. Max query latencies in hundreds milliseconds even for 30G cache sizes (similarily sized vanilla Ehcache gives 60-70 sec GC pauses). Faster than BigMemory (much faster). Just curious?

        • 16. Re: How to set limit on the memory use for Cache
          sannegrinovero

          Hi,

          Anybody is interesting in BigMemory -type Off Heap storage for Infinispan?

          You mean you want to help implementing it? A prototype is coming very soon.

           

          You can set hard limit on a max allocated memory (exact limit). The memory is allocated off-heap so you can keep your JVM heap low.

          Yes it would be nice to manage the exact amount of memory, but it would still require to serialize all stored objects. Actually, if we would serialize all objects in Java, we could estimate the size in-VM quite accurately without needing an off-heap storage.

           

          Max query latencies in hundreds milliseconds even for 30G cache sizes (similarily sized vanilla Ehcache gives 60-70 sec GC pauses).

          I think you could get queries in the hundreds of milliseconds or much better even when having 30G heap sizes, with proper configuration; Would be nice trying, I don't have that amount of memory but got queries in the single-millisecond range on a 15G heap.

           

          Faster than BigMemory (much faster). Just curious?

          What is faster than BigMemory ? Yes I'm curious too

          • 17. Re: How to set limit on the memory use for Cache
            cbo_

            Yeah, pretty sure there would be interest in a proof-of-concept for something like that.  Wondering, though, if those access latencies / GC pauses are really that high?!?

            • 18. Re: How to set limit on the memory use for Cache
              kodadma

              Sanne Grinovero wrote:

               

              Faster than BigMemory (much faster). Just curious?

              What is faster than BigMemory ? Yes I'm curious too

              This is BigMemory - type off heap storage which can be attached to ANY Java IMDG (in theory). The product is in a closed alpha testing phase right now. The product name and and bechmark results will be published soon. By itself, it is very fast (even with serialization/deserialization). Much faster than BigMemory with much lower latencies and (surprise) even faster than some Java local (on heap) caches. Unfortunately, integration with  ISPN is not going very well. It works but we have numerous issues with GCs (ISPN produces a lot of objects and is not tuned for memory consumption). Another issue I am observing right now is high lock contention in Cache (CachLoader/CacheStore?) implementation. May be I am doing something wrong but I can not even get sustained 120% of CPU usage for my 2CPU-cores laptop and 2 threads.

               

              Btw, having GC pauses under control with extra large Java heaps (> 4-8 G) is not even an art - its Vodoo magic. Are you Vodoo? I am not.

              • 19. Re: How to set limit on the memory use for Cache
                mircea.markus

                The product is in a closed alpha testing phase right now. The product name and and bechmark results will be published soon.

                I'm very curious as well

                Unfortunately, integration with  ISPN is not going very well. It works but we have numerous issues with GCs (ISPN produces a lot of objects and is not tuned for memory consumption).

                Did you run any profile on it? Would be curious to see which code generates the object-noise.

                Another issue I am observing right now is high lock contention in Cache (CachLoader/CacheStore?) implementation.

                Does that mean that your bigmemory integrates within infinispan as a CacheLoader? 

                • 20. Re: How to set limit on the memory use for Cache
                  cbo_

                  With regard to GCs and lock contention it might be worthwhile to post the environment you are running in (hardwar, VM and OS in particular).

                  • 21. Re: How to set limit on the memory use for Cache
                    kodadma

                    Yes, I am doing CacheLoader/Store because I have not found different way of integration yet (except custom interceptor). I will take a look at DataContainer API. Ideally, it would be nice to have ability to bypass ISPN internal cache storage (DataContainer?) entirely and do not keep any data in ISPN internal cache at all. We do eviction and locking ourselves and quite efficiently.

                    • 22. Re: How to set limit on the memory use for Cache
                      galder.zamarreno

                      Vladimir, if you're not interested in:

                      - internal data container

                      - eviction

                      - locking

                       

                      What is it that you 'want' from Infinispan?

                      • 23. Re: How to set limit on the memory use for Cache
                        manik

                        Or contribute to it!  Join infinispan-dev and talk about contributing...

                        • 24. Re: How to set limit on the memory use for Cache
                          govinds

                          Hi,
                          when can we expect such a solution in infinispan to limit the cache size.(except limiting the no of entries)

                          This question is about four years old but still I think there is no better solution for this.

                          • 25. Re: How to set limit on the memory use for Cache
                            gaprashanth

                            Hi All,

                             

                            yes even I would like to re-visit this thread after 4 years. Do we have a way of setting the cache partition limit in terms of memory units rather than numbers ?

                            1 2 Previous Next