Custom serializers

[gregor.sfiligoj]

I just deployed on Beta6 a project I developed for AppEngine. The project use Objectify as datastore's "interface" and also Endpoints.

Because of some Jackson circular reference problems during JSON serialization I need to have a custom serializer for objectify's Key class.

The serializer is defined in "serializers" field of @Api annotation. But it seems that CapeDwarf ignores it.

Are custom serializers already supported?

[alesj]

Before answering this :-), how do you imagine serializers work?

As I actually misunderstood it the first time, hence CapeDwarf impl is probably wrong.

We do handle that, but it looks like it doesn't kick-in for you for some reason.

* https://github.com/capedwarf/capedwarf-jboss-as/blob/master/extension/src/main/java/org/jboss/as/capedwarf/deployment/CapedwarfEndpointsJaxrsProcessor.java#L50

Do you have any .api file present in your deployment?

We actually don't read it, but they do serve us as marker files.

Can you provide a test for this?

Or attach a simple example?

[gregor.sfiligoj]

You need serializers for customize the Java objects to JSON translation.

In my project for example, the Jackson implementation (which Appengine uses) marshall Java objects via reflection. In some situations, where objects have reference to other objects of same type for example, Jackson find a "circular reference" and returns an exception... In this case, using a custom serializer is possible to avoid this.

So in general, if I have an Example.class for which I'm not happy with the Jackson serialization, I write a custom ExampleSerializer.class and add it to serializers array of @Api annotation (or using @ApiSerializer annotation).

Give me some time to get the source and check what happens now with your implementation ;-)

A very simple example is in the Appengine documentation: https://developers.google.com/appengine/docs/java/endpoints/annotations#apiserializer

[alesj]

A very simple example is in the Appengine documentation: https://developers.google.com/appengine/docs/java/endpoints/annotations#apiserializer

Yeah, this one is completely broken. :-)

You're not allowed to have primitive types as input/output.

We already have a few tests here:

* https://github.com/GoogleCloudPlatform/appengine-tck/tree/master/core/endpoints/src/test/java/com/google/appengine/tck/endpoints

Can you add your serializer impl to this thread?

Anyway, I had a f2f with Endpoints guys, so I now know what to do. :-)

[gregor.sfiligoj]

"You're not allowed to have primitive types as input/output."

This is not true!

It is true, that you are not allowed to have primitive types, enums and also primitive "classes" (Integer, Long and so on) as input/output in API METHODS not objects returned as method result or useed as method input.

For example, for a Long returning type the Google client library generator creates a LongResponse class which has getter/setter for a Long type!

We need custom serializers for objects used as input arguments or responses from api methods. Don't miss this point!

So, the example in appengine documentation is working. They serialize a Bar class which CAN have any kind of java serializable type (from Jackson marshaller point of view, not by the Java Serializable subclass meaning...) Jackson marshaller is used for this task. And, as already told, if I'm not happy how Jackson serialize a class (or to avoid Jackson marshaller exceptions or whatsoever) I will write my custom serializer.

This is the simple serializer I use. I just need a safeString rapresentation of a Key instead of the whole serialized class in my client code: 

package eu.novalike.diemmegi.web.server;


import com.google.api.server.spi.config.Serializer;
import com.googlecode.objectify.Key;


@SuppressWarnings("rawtypes")
public class KeySerializer implements Serializer<Key, String> {


  @Override
  public Key deserialize(String safeString) {
  return Key.create(safeString);
  }


  @Override
  public String serialize(Key key) {
  return key.getString();
  }


}

I hope that this better explains this topic :-)

[alesj]

So, what's the exact response you get when returning Key?

I guess this KeySerializer, after your endpoint returns Key, takes over, and converts this into (safe) String.

But what's the final output?

It must be something JSON-like, and I'm wondering what it is.

Probably StringResponse?

(or something like that, if we follow Long' example)

[gregor.sfiligoj]

You return the String rapresentation of the class or whatsoever you are serializing.

For example, this is the JSON response from AppEngine to one of my methods:

{

  "items" : [ {

  "id" : "4638839557586944",

  "nome" : "Nome 16",

  "cognome" : "Cognome 16",

  "azienda" : "Azienda 16 srl",

  "indirizzo" : "Via del Cliente, 16",

  "localita" : "Loc. del Cliente16",

  "telefono" : "[phone]",

  "email" : "[e-mail]",

  "updateTime" : "2013-10-04T19:01:19.692+02:00",

  "impianti" : [ "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4OIIDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4NILDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4IoJDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4OoKDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4LoIDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4IYLDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4OYJDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4LYKDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4M4IDA", "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4O4LDA" ]

  } ]

}

My class getter returns a List of Key-s, so the field "impianti" is the result of List<Key> serialized by my serializer. The Objectify Key class already have a getString() method which returns a String encoded Key rapresentation (i.e. a String like "aglhcHBlbmdpbmVyFQsSCEltcGlhbnRvGICAgICA4OIIDA") and a constructor I use to create a Key from this kind of String.

In the appengine documentation you see the output of the Bar class. Bar is a object so if an instance of Bar class have x=1 and y=2 in JSON you should have:

{"bar": {"x": 1, "y": 2}}

but using the example serializer insted of returning the serialized object  {"x": 1, "y": 2} you obtain a simpler

{"bar": "1,2"}

[alesj]

Yeah, I understand all of this.

But what's your return type on the @Api method?

From example, if one would return Bar, it would be illegal,

as Bar --> String, and String cannot be represented as JSON, right?

[gregor.sfiligoj]

There is a bit of confusion :-)

JSON IS STRING. JSON rapresent objects as strings.

Appengine do this work for you (using Jackson marshaller). But, it will not serialize returning objects (of methods) which are primitives or enums (they are allowed for input arguments).

Why this limitation is not known to me, because with Jackson, Gson and other JSON marshallers I think that it should be possible.

So, if you would return an Integer you will return a wrapper class instead which has a getter returning Integer instead. For collections Appengine returns a Collection object which has a getItems() getter for example.

But, and here is the point, don't confuse this limitation with the fact that the JSON response IS a string. All responses of REST Http requests are strings if we want say so.

Your custom serializer return a String because that string is embedded in the response. I will try to explain better. Take the example in appengine documentation, the well known Bar class:

public class Bar {

  private final int x;

  private final int y;

  public Bar(int x, int y) {

    this.x = x;

    this.y = y;

  }

  public int getX() {

    return x;

  }

  public int getY() {

    return y;

  }

}

At some point you have a method that looks like:

public void getBar(@Named("x") Integer x,  @Named("y") Integer y) { // "primitives" argument is allowed!

     return new Bar(x,y);

}

When GAE serialize the JSON RESPONSE passing x=1 and y=2 in the request will be:

{"x": 1, "y": 2}

This is a JSON rapresentation of Bar object.

Suppose that you are not happy with this rapresentation and instead you prefer this JSON rapresentation:

{"sum":3, "diff":1}

I know that "sum" is x+y and "diff" is y-x so I know also x and y (yes! I'm an engineer :-))

How to obtain this? Custom serializer!

I write something like that:

public class BarSerializer implements Serializer<Bar, String> {

  public String serialize(Bar in) {

    return "{\"sum\":"+in.getX()+in.getY() + ", \"diff\":" + (in.getY()-in.getX()) + "}";

  }

  public Bar deserialize(String in) {

     int x = ........;

     int y = ........;  

     // some parsing work :-)

     return new Bar(x,y);

  }

}

Thats'all! In this example I substitute JSON object with another JSON object rapresentation. The original example return a pure string and not a JSON object.

{"bar": {"x": 1, "y": 2}} --> {"bar": "1,2"}

Returning to my project the method which returns the response I posted before is:

@ApiMethod(name = "cliente.list", path = "cliente/{start}/{rows}", httpMethod = HttpMethod.GET)

   public ListCliente listCliente(@Named("start") Integer start,

         @Named("rows") Integer rows) {

      List<Cliente> response = ofy().load().type(Cliente.class).offset(start)

            .limit(rows).list();

      return new ListCliente(response);

   }

ListCliente is a collection wrapper class for List<Cliente>, Cliente is:

public class Cliente {

  @Id

  Long id;

  String nome;

  // other properties

  Date updateTime;

  List<Key> impianti;

   // getters/setters

}

My serialzer substitute the "impianti" property serializing my way the Key class.

Uff.... a long response, I hope that is also clear ;-)

[alesj]

Try this http://in.relation.to/Bloggers/FinallyNewCapeDwarfReleaseThisTimeOnWildFly

As I changed how we handle Endpoints -- we now simply use original resources.