Marshalling problem with hotrod client in 9.4.0

[janbartel]

This problem may potentially happen with earlier versions of infinispan/hotrod client, we have only tested 9.4.0 after upgrading from 9.1.0.

Jetty uses a custom protobuf-based marshaller for the webapp session data.  During deserialization, we use the thread context classloader to select the classloader for the correct webapp in order to deserialize the data. Our code ensures that before we call cache.get(key), we have set the thread context classloader correctly.

Since upgrading to 9.4.0, the thread that calls our marshaller is a HotRod-client-async-pool thread, not the application thread that called cache.get(key). Thus, the async pool thread does not know about the correct classloader and the session data cannot be deserialized correctly.

Is there some way to adorn the async pool thread with the correct classloader information? Alternatively, can we revert to using the application thread to do the deserialization?

thanks
Jan

[rvansa]

Hi, could you try calling RemoteCache.withDataFormat(...).get(...) and provide the webapp-specific value marshaller directly instead of loading that from thread-local?

[janbartel]

Hi Radim,

Thanks for that suggestion. It could work, but as I understand it, I would need to do the following before each call to cache.get:

DataFormat.Builder builder = new DataFormat.Builder();

SpecialWebappMarshaller marshaller = new SpecialWebappMarshaller();

marshaller.setClassLoader(Thread.currentThread().getContextClassLoader());

builder.valueMarshaller (marshaller);

cache.withDataFormat(builder.build()).get(key);

That's a lot of object creation before every single get(). Am I missing a more efficient technique? BTW, I would also need to implement the above code via reflection, because a) we only use the BasicCache api to interact transparently both with a remote or embedded cache  and b) I need this work with both 9.1.0 and 9.4.0.

thanks

Jan

[rvansa]

Creating the DataFormat on each invocation would be inefficient, of course. I misread your first post, thinking that you store webapp-specific info in thread-local variable - who sets the thread context class loader in your case, is it the webapp container (WildFly, Tomcat...) or your code? Could you store the DataFormat in a ThreadLocal field? (notice the withInitial() static method). Even better approach would be to store the resulting BasicCache instance because each withDataFormat() creates a wrapping instance. Especially since you need to cope with multiple versions, your code probably needs some level of indirection. Rather than reflection I would recommend polymorphic implementations in modules based on your dependencies.

The reason everything happens now in the async thread pool is because seeing the limitations with blocking invocations in a distributed environment Infinispan goes to async-by-default, and the .get(key) is just a convenience method on .getAsync(key).join(). In the past it was the other way, .getAsync() just executed .get() in a limited thread-pool which could get depleted very fast, and such invocation also used more thread context switches. Future APIs will encourage reactive programming model even more (though often providing convenience classes that do the CompletableFuture.join() for user).