storeAsBinary forces me to handle MarshalledValue

[ksobolewski]

Hi, I'm a new incarnation of me from the times of JBoss Cache, where I reported some bugs I had to register again because my company changed names (three times) and my old email does not work anymore...

We're now evaluating Infinispan as a replacement for JBoss Cache (yes, it took us some time). We very much like the storeAsBinary option, because we care about memory usage more than performance (and we also like the defensive option, not because of memory safety, but because it discards the POJO representation and saves memory).

But there is a glitch. When I enable storeAsBinary, I am "forced" to handle MarshalledValue objects directly in some places, most notably in the map-reduce framework and in the JdbcStringBasedCacheStore's Key2StringMapper, where I receive my (keys and) value objects wrapped in it. I don't like it and I'd like to know if it's a deliberate decision or a bug

[sannegrinovero]

Hi Krzysztof! welcome back.

I think that's a bug, please report it on JIRA. I guess there might be situations in which dealing with MarshalledValue could be more efficient, but we should at the very least have an option for it (if there really was need for), and by default I agree this would not be expected.

I'm curious about your interest in memory consumption: are you getting strong benefits from storing each entry in its marshalled form?

[ksobolewski]

Thanks, I'll investigate more closely and maybe even prepare a test case

Yes, the memory consumption benefits of storing entries in marshaled form are quite significant. It depends on the object and the marshaled form, of course. We are even so not concerned with performance that we're compressing the marshaled byte array But even then it could be improved further on the Infinispan's side. Since we're trying to keep millions of entries in memory, we'd like to see fewer objects per entry in the internal data structures - there is the entry object, then there is the MarshalledValue for both the key and the value, and then there are the fixed/variable length byte array wrappers... It all adds up to lots of overhead.

[ksobolewski]

Hm, there is a test:

org.infinispan.loaders.BaseCacheStoreTest.testLoadAndStoreMarshalledValues()

that explicitly inserts MarshalledValue into a CacheStore. (JdbcStringBasedCacheStoreTest overrides it to declare that it is expected that it throws UnsupportedKeyTypeException for JdbcStringBasedCacheStore.) Added by 44e92d025e6cad76070723cb8fcf4920da9433b8 in response to ISPN-1653 (CacheLoader with BdbjeCacheStore throws NotSerializableException: org.infinispan.marshall.MarshalledValue). So I'm confused. From the discussion under ISPN-1653 it looks like it's intentional that MarshalledValued is passed to cache stores (and hence to the Key2StringMapper) when storeAsBinary is enabled. I still don't like it but if that's final then I can somehow live with it

[ksobolewski]

I created ISPN-3468 for the map-reduce bug I mentioned. It's more limited than I thought and is clearly a bug.