This week I have been working on a few head-scratching AS7 issues related to ObjectMessage.
Good news is that most of them are due to incorrectly deployed Java EE modules and can be fixed.
Bad news is that some are due to Java Serialization and ClassLoader issues.
Sometimes ago, Jason wrote a article about Modular serialization that explains how it works in AS7.
This directly relates to ObjectMessage since an Object created inside a modular environment can not be deserialized using standard Java serialization. The modules does not make visible their dependencies. This is a *good thing* in general but gets in the way for deserialization.
It's easy to fix serialization/deserialization inside AS7 by using JBoss marshalling.
However for HornetQ, we can expect a mix of environment. Normal use cases is to have some clients in a modular environment (eg producer in a Servlet, MDB) and some in a non-modular environment (consumer/producer in a regular Java application relying on the classpath).
To make things worse, we can not negotiate which serialization to use when sending the message. A same ObjectMessage can be delivered to clients in modular and non-modular environments.
Another thing to take into account is that HornetQ is not aware of JBoss modules. It does not depend on it and does not need to... except for ObjectMessage.
The last 2 days I have played with HornetQ to reach a possible solution.
I did the dumbest thing that could possibly work. I added dependencies to JBoss marshalling and modules jar to HornetQ. When a ObjectMessage's body is serialized, I serialize it:
1. a first time using Java serialization
2. a second time using JBoss marshalling module-aware serialization (if it's in a modular environment)
Yes, the payload object is serialized twice
When a message is deserialized:
1. I deserialize it using standard Java serialization (using ObjectInputStreamWithClassLoader)
2. if that fails (ClassNotFoundException) and I am in a modular environment, I try to deserialize it using JBoss marshalling
This solve the AS7 issue I am facing but not all of them. For example if a message is produced in a non-modular environment, there is no way to have it deserialized in a modular environment whithout tweaking the modules.
It also introduces a dependency from HornetQ to JBoss modules and marshalling. This can be circumvent by having in HornetQ a way to register an additional handler for object message serialization that I could set when it is running inside AS7.
Frankly, this looks ugly and I have a hard time finding a solution which solves all these cases ( I'll think more about it over the week-end as I'd like to find a solution that fixes all the edge cases).
I am really wondering if all this additional complexity is worth for a few edge use cases. Best thing is to educate user and move away from ObjectMessage.