-
15. Re: That is just too much overhead
kodadma Jun 23, 2011 5:17 PM (in response to kodadma)I just tried CR6 out- the same results. It seems that when cache reaches max limit - all working threads stuck forever (or almost forever). This is reproducible even for 2 threads and 1G of heap memory (1M entries limit).
-
16. Re: That is just too much overhead
kodadma Jun 23, 2011 6:04 PM (in response to kodadma)On a DataContainer interface:
/**
* Returns a set of keys in the container. When iterating through the container using this method,
* clients should never call {@link #get()} method but instead {@link #peek()}, in order to avoid
* changing the order of the underlying collection as a side of effect of iterating through it.
*
* @return a set of keys
*/
Set<Object> keySet();
/**
* @return a set of values contained in the container
*/
Collection<Object> values();
/**
* Returns a mutable set of immutable cache entries exposed as immutable Map.Entry instances. Clients
* of this method such as Cache.entrySet() operation implementors are free to convert the set into an
* immutable set if needed, which is the most common use case.
*
* If a client needs to iterate through a mutable set of mutable cache entries, it should iterate the
* container itself rather than iterating through the return of entrySet().
*
* @return a set of immutable cache entries
*/
Set<InternalCacheEntry> entrySet();
These three methods are killers for any reasonably large off-heap caches. I think significant reengineering is required (move from Collections to iterators/scanners) in DataContainer to make integration of OffHeap storage possible.
-
17. Re: That is just too much overhead
kodadma Jun 23, 2011 8:52 PM (in response to kodadma)Puts when eviction is active are very expensive. LRU.execute takes approx 95% of overall execution time of DefaultDataContainer.put() You may need to reconsider the eviction implementation for DefaultDataContainer.
-
18. Re: That is just too much overhead
kodadma Jun 24, 2011 1:44 AM (in response to kodadma)How to activate new DataContainer?
This code does not work.
OffHeapDataContainer dc = new OffHeapDataContainer(mCache);
EmbeddedCacheManager manager = new DefaultCacheManager();
Configuration config = new Configuration().fluent(). dataContainer(). dataContainer(dc). build();
manager.defineConfiguration("name", config);
ispnCache = manager.getCache("name");
-
19. Re: That is just too much overhead
shane_dev Jun 24, 2011 11:35 AM (in response to kodadma)FYI - What you are describing is perhaps similar to the issue we found. I'm not sure anything can be done about PUT performance resulting in eviction but if you have a 90% GET rate you are likely running into our issue.
-
20. Re: That is just too much overhead
kodadma Jun 24, 2011 1:09 PM (in response to shane_dev)Shane, I went through the ISPN code myself but no so thouroghly as you apparently. There are too many places in the code (eviction implementation is only one of them) which make me think that religious following "right" OOP design patterns and rules is not always right approach. For example, I have no idea how to plug my own DataContainer in despite the fact that there is API call in FluentConfig which is supposed to do this, but it does not. Even if I get answer from ISPN developers on this question, the next one will definetely follow: Ok, how to ged rid of LockingInterceptor entirely? I do locking myself in my container and I do not want to create separate lock object for every key-value pair in my container (btw, it seems that lock striping is broked).
-
21. Re: That is just too much overhead
kodadma Jun 24, 2011 3:16 PM (in response to kodadma)There is no way to override dataContainer settings in ISPN 5.0 CR6 as since OverrideConfigurationVisitor which extends AbstractConfigurationBeanVisitor does not implement
public void visitDataContainerType(DataContainerType bean)
method. I think its a bug.
-
22. Re: That is just too much overhead
kodadma Jun 24, 2011 6:00 PM (in response to kodadma)I have patched *OverrideConfigurationVisitor* to allow data container config overriding and was able to run tests on ISPN 5 + OffHeapDataContainer. Preliminary results are very encouraging. I will keep this forum posted as soon as I finish the integration.
-
23. Re: That is just too much overhead
sannegrinovero Jun 25, 2011 8:44 AM (in response to kodadma)I have patched *OverrideConfigurationVisitor* to allow data container config overriding and was able to run tests on ISPN 5 + OffHeapDataContainer. Preliminary results are very encouraging. I will keep this forum posted as soon as I finish the integration.
Hi Vladimir,
I can't look at the code right now but that sounds as a fix we would need to integrate in Infinispan 5 before it's final, can you please open a bug on JIRA and contribute your fix? a pull request via github would be ideal, but a "traditional" patch would be fine too.
Thanks,
Sanne
-
24. Re: That is just too much overhead
kodadma Jun 27, 2011 11:20 PM (in response to sannegrinovero)Add this method to OverrideConfigurationVisitor
{code}
@Override
public void visitDataContainerType(DataContainerType config) {
this.dataContainerType = config;
this.dataContainerType.overriddenConfigurationElements.add("dataContainer");
}
{code}
and define class member:
{code}
private DataContainerType dataContainerType = null;
{code}
This will make DataContainer config overriding possible but for all other configs you better check the code. It seems that it is broken.
-
25. Re: That is just too much overhead
kodadma Jul 1, 2011 7:41 PM (in response to kodadma)I have created JIRA:
-
26. Re: That is just too much overhead
kodadma Jul 3, 2011 9:03 PM (in response to kodadma)OK, I have described performance of Infinispan + Koda (OffHeap storage) integration here:
http://designedtoscale.blogspot.com/2011/07/introducing-koda-key-code-value-in.html