6 Replies Latest reply on Jul 20, 2010 10:27 AM by galder.zamarreno

    Custom Marshaller

    shane_dev

      Does JBoss Marshalling support backwards compatibility? For example, when fields are added/removed to a class.

       

      Otherwise, I'm thinking about creating a MessagePack or Protobuf marshaller since they do.

        • 1. Re: Custom Marshaller
          dmlloyd

          Yes.  Please see http://download.oracle.com/docs/cd/E17409_01/javase/6/docs/platform/serialization/spec/version.html#6754 for information about how backwards compatibility works with serialization.

          • 2. Re: Custom Marshaller
            shane_dev

            Thanks for the link. I hadn't realized that Java serialization supports adding and even removing fields. Looks like it will work for our needs.

            • 3. Re: Custom Marshaller
              mircea.markus

              There is a protbuf based marshaller on its way: https://jira.jboss.org/browse/ISPN-508

              igonre the hotrot bit, the marshaller should/will be reusable beween infinispan core and hotrod.

              • 4. Re: Custom Marshaller
                shane_dev

                I saw that. I was curious as to how you'd handle the fact that the classes are generated. I ended up looking at MessagePack since it doesn't rely on generated classes. However, it does appear that you need to pass it a schema (as a String) for deserializing objects. Seems similar to the concept of the "Descriptor" in Protobuf's DynamicMessage.

                • 5. Re: Custom Marshaller
                  galder.zamarreno

                  Shane, protobuf does not have much issues for writing, since you always write a Builder transformed into a Message or a Message directly. The reading part is the difficult one since it relies on type knowledge. Also, DynamicMessage is only available for C++ and Java, no Python.

                   

                  I'm currently investigating several multi-platform serialization mechanisms. Thanks for heads up wrt MessagePack, I'll add to the investigation.    

                  • 6. Re: Custom Marshaller
                    galder.zamarreno

                    Btw Shane, looks to me MessagePack does rely on generated classes for any custom objects except for very specific cases. For example, it supports generic marshalling with objects based on JDK classes, i.e.  A map of <String, Int>. But you can also pass a custom object with only basic types (http://github.com/msgpack/msgpack/blob/master/java/test/thrift-protobuf-compare/tpc/src/serializers/msgpack/MessagePackGenericSerializer.java) and deserialize it given the schema. For complex objects, building such objects would be quite a pain and would require some extra bit of memory to create these intermediate objects.

                     

                    I'm yet to see a library that allows generic marshalling for complex objects without relying on slow reflection. For example, you could have clients registering all schemas on startup and serialization will include a key to the schema used. The reading part could use the key to find out the schema and build the object in an type agnostic way. This would allow for proper generic marshalling. That's what we do internally in Infinispan with JBoss Marshalling. The thing we have missing is that it's not portable and the fact that we still do not allow users to plug in their schemas. The latter will come soon.