-
15. Re: How to set limit on the memory use for Cache
kodadma Jun 6, 2011 1:47 PM (in response to mircea.markus)Anybody is interesting in BigMemory -type Off Heap storage for Infinispan? You can set hard limit on a max allocated memory (exact limit). The memory is allocated off-heap so you can keep your JVM heap low. Max query latencies in hundreds milliseconds even for 30G cache sizes (similarily sized vanilla Ehcache gives 60-70 sec GC pauses). Faster than BigMemory (much faster). Just curious?
-
16. Re: How to set limit on the memory use for Cache
sannegrinovero Jun 6, 2011 2:42 PM (in response to kodadma)Hi,
Anybody is interesting in BigMemory -type Off Heap storage for Infinispan?
You mean you want to help implementing it? A prototype is coming very soon.
You can set hard limit on a max allocated memory (exact limit). The memory is allocated off-heap so you can keep your JVM heap low.
Yes it would be nice to manage the exact amount of memory, but it would still require to serialize all stored objects. Actually, if we would serialize all objects in Java, we could estimate the size in-VM quite accurately without needing an off-heap storage.
Max query latencies in hundreds milliseconds even for 30G cache sizes (similarily sized vanilla Ehcache gives 60-70 sec GC pauses).
I think you could get queries in the hundreds of milliseconds or much better even when having 30G heap sizes, with proper configuration; Would be nice trying, I don't have that amount of memory but got queries in the single-millisecond range on a 15G heap.
Faster than BigMemory (much faster). Just curious?
What is faster than BigMemory ? Yes I'm curious too
-
17. Re: How to set limit on the memory use for Cache
cbo_ Jun 6, 2011 3:46 PM (in response to gaprashanth)Yeah, pretty sure there would be interest in a proof-of-concept for something like that. Wondering, though, if those access latencies / GC pauses are really that high?!?
-
18. Re: How to set limit on the memory use for Cache
kodadma Jun 6, 2011 7:48 PM (in response to sannegrinovero)Sanne Grinovero wrote:
Faster than BigMemory (much faster). Just curious?
What is faster than BigMemory ? Yes I'm curious too
This is BigMemory - type off heap storage which can be attached to ANY Java IMDG (in theory). The product is in a closed alpha testing phase right now. The product name and and bechmark results will be published soon. By itself, it is very fast (even with serialization/deserialization). Much faster than BigMemory with much lower latencies and (surprise) even faster than some Java local (on heap) caches. Unfortunately, integration with ISPN is not going very well. It works but we have numerous issues with GCs (ISPN produces a lot of objects and is not tuned for memory consumption). Another issue I am observing right now is high lock contention in Cache (CachLoader/CacheStore?) implementation. May be I am doing something wrong but I can not even get sustained 120% of CPU usage for my 2CPU-cores laptop and 2 threads.
Btw, having GC pauses under control with extra large Java heaps (> 4-8 G) is not even an art - its Vodoo magic. Are you Vodoo? I am not.
-
19. Re: How to set limit on the memory use for Cache
mircea.markus Jun 7, 2011 8:00 AM (in response to kodadma)The product is in a closed alpha testing phase right now. The product name and and bechmark results will be published soon.
I'm very curious as well
Unfortunately, integration with ISPN is not going very well. It works but we have numerous issues with GCs (ISPN produces a lot of objects and is not tuned for memory consumption).
Did you run any profile on it? Would be curious to see which code generates the object-noise.
Another issue I am observing right now is high lock contention in Cache (CachLoader/CacheStore?) implementation.
Does that mean that your bigmemory integrates within infinispan as a CacheLoader?
-
20. Re: How to set limit on the memory use for Cache
cbo_ Jun 7, 2011 8:27 AM (in response to mircea.markus)With regard to GCs and lock contention it might be worthwhile to post the environment you are running in (hardwar, VM and OS in particular).
-
21. Re: How to set limit on the memory use for Cache
kodadma Jun 7, 2011 2:14 PM (in response to mircea.markus)Yes, I am doing CacheLoader/Store because I have not found different way of integration yet (except custom interceptor). I will take a look at DataContainer API. Ideally, it would be nice to have ability to bypass ISPN internal cache storage (DataContainer?) entirely and do not keep any data in ISPN internal cache at all. We do eviction and locking ourselves and quite efficiently.
-
22. Re: How to set limit on the memory use for Cache
galder.zamarreno Jun 9, 2011 6:00 AM (in response to kodadma)Vladimir, if you're not interested in:
- internal data container
- eviction
- locking
What is it that you 'want' from Infinispan?
-
23. Re: How to set limit on the memory use for Cache
manik Jun 23, 2011 11:38 AM (in response to mircea.markus)Or contribute to it! Join infinispan-dev and talk about contributing...
-
24. Re: How to set limit on the memory use for Cache
govinds Mar 12, 2015 2:17 AM (in response to gaprashanth)Hi,
when can we expect such a solution in infinispan to limit the cache size.(except limiting the no of entries)This question is about four years old but still I think there is no better solution for this.
-
25. Re: How to set limit on the memory use for Cache
gaprashanth Apr 1, 2015 12:21 PM (in response to govinds)Hi All,
yes even I would like to re-visit this thread after 4 years. Do we have a way of setting the cache partition limit in terms of memory units rather than numbers ?