Runtime container choice. vert.x vs WF core vs Karaf vs ???

[theute]

I see that we already have 2 implementation of RHQ Metrics server, one based on vert.x and one based on WF-core (JAX-RS), that' s probably one too many and this raise the question of which container to pick...

I don' t have a firm decision (leaning toward one but will try to keep it secret initially) but I see pros and cons:

vert.x

pros

• Current packaging makes it super easy to start with it... java -jar server-fat.jar and you are up and running (ignoring Cassandra for now)

cons

Wildfly Core

Here let's consider that Wildfly executes on the notion of profiles to add to a minimum Wildfly. So Wildfly is used as it's minimum

pros

• Wildly Core comes with a management layer which will come handy to configure RHQ metrics
• Standards

cons

• It currently requires to download WF separately and install the war. (We may be able to avoid that later and pick WF from Maven, to be confirmed)

Karaf

pros

• Any ?

cons

• OSGI madness

[john.sanda]

vert.x

pros

• Current packaging makes it super easy to start with it... java -jar server-fat.jar and you are up and running (ignoring Cassandra for now)
• Very easy to embed
• The async architecture and APIs are a good match for the DataStax driver which is also async
• Easy to test
• Good clustering support
• Nice deployment model with modules that can be downloaded and installed dynamically at runtime
• Modules and event bus promote modular designs
  • It will be easy for example to build a WebSockets server that uses the same core metrics-server that the REST server uses
• great Polyglot support

cons

• Logging
  • Built-in logging uses java.util.logging (that can be overridden with system properties though)
• Async programming can be difficult to reason about, debug, etc.

Wildfly Core

I have some questions more so that pros/cons.

• Does using Wildfly Core require a JEE container? If so, then I see that as a con.
• What is testability like. How easy is it to write integration tests?
• How easy/difficult to set up clustering?
• How easy/difficult will it be to embed in other runtimes, e.g., RHQ?

Karaf

I do not really have any experience with Karaf. A couple pros might be Fabric8 support/integration and clustering (I believing clustering support using Hazlecast). And I definitely see the OSGi madness as a big con

[theute]

• Does using Wildfly Core require a JEE container? --> No
• What is testability like. How easy is it to write integration tests? Not sure and how that compares to vert.x
• How easy/difficult to set up clustering? You will need to add clustering profile on top of Core
• How easy/difficult will it be to embed in other runtimes, e.g., RHQ? It would basically be a web archive

[pilhuhn]

I think vert.x brings a nice trait for the future and this is polyglot. We want that metrics server thingy (that exposes endpoints and talks to memory /C* backend) to be able to host plugins so that users / other developers can write their own code.

My canonical example here are baselines, where we may supply a default of what RHQ has today, but where users may want to use seasonal algorithms like Holt-Winters or an algorithm like our baselines, but which cuts out outliers that are outside the 99,99-percentile to not widen the band too much.

And to develop those plugins it is good when they can just use the language of choice.

What I do not like in the current vert.x implementation (but which may come handy later with plugins) is that the rest-api is not expressive like JAX-RS(2).

In JAX-RS you write

    @GET

    @Path("/{id}/data")

    @Produces({"application/json","text/json","application/xml"}) @GZIP

    public Response getDataForId(@PathParam("id") String id, @QueryParam("start") Long start, @QueryParam("end") Long end)

and it is clear from the annotations what data types can be returned, the names of the query parameters and the url format, while in the current implementation of

        routeMatcher.get("/rhq-metrics/:id/data", new Handler<HttpServerRequest>() {

            @Override

            public void handle(final HttpServerRequest request) {

                final String id = request.params().get("id");

you have to dig into the handler code to find the parameters.

How would one embed the vert.x based solution into e.g Wfly or also Fabric8? As Thomas mentioned, for the current JAX-RS based solution this is like dropping a WAR file.

[john.sanda]

I consider the Vert.x implementation to this point very much a working prototype. I was really focused on getting something done quickly for the purpose of discussion like this. With respect to the API being expressive, the docs I have written at rhq-metrics/metrics-rest at master · rhq-project/rhq-metrics · GitHub cover (or least strive to) the details you mention. I agree that the JAX-RS annotations are expressive. If the overall concern though is expressiveness/documentation, then certainly this can be addressed in a number of ways. I am also not sure how useful the expressiveness of JAX-RS is for someone who is coming from a non-Java/JEE environment.

Embedding Vert.x is really easy. Just look at the integration tests I have written as an example - https://github.com/rhq-project/rhq-metrics/blob/master/metrics-rest/src/test/java/org/rhq/metrics/core/MetricsITest.java. I have worked on another project in the past that embedded Vert.x in Cassandra. I would expect it to be equally easy to embed Vert.x in other environments as well.

[mithomps]

Clarification question: Are we picking a single container to go with or are we just starting with this container to make implementation simpler (more of a roadmap question and how open we should keep the architecture)?

What requirements are we asking of the container? REST, transactions, async, security, size/scalability(memory requirements, request/second, latency, number of threads)....or are other technology decisions more important?

Environmental constraints like PaaS might also be a large factor in limiting (or at least prioritizing) the requirements.

[theute]

Single container. (It doesn' t mean that we don' t use vert.x though, could be used with WF if WF was the choice).

Needs to run on PaaS for sure.

There are short term technological requirements (like REST, Tx, async...) but also longer term requirements like "Manageability", recently LiveOak moved from Netty to WF mainly for future management reasons.

[nstefan]

Thomas Heute wrote:

There are short term technological requirements (like REST, Tx, async...) but also longer term requirements like "Manageability", recently LiveOak moved from Netty to WF mainly for future management reasons.

I think the best way to plan for manageability is to have a simple container/framework that does not require that. If the plan is for RHQ Metrics to be a lightweight project that is the last thing we should have on the list.

Here is another aspect that was not considered. Cassandra servers are disposable. You set 10 servers in a cluster, 3 of them die, the cluster keeps going with little service disruption. Why not do the same with user facing portion of this project? You can set a bunch of these API/UI servers and they all work with the one Cassandra cluster. 5 servers die, well data can still be accessed, added, and aggregated. And what about manageability? That is easy, if it the server alive then we are good, if it goes down there is another one running while this one restarts. So we reduce the problem to just a binary state: is server up or down.

In my opinion the future is lightweight containers with minimalist frameworks that let you build up. Sure the vert.x code from John is just a prototype but I think he is not far from a what a production version would look like. You do not need documentation, annotations and 6 level deep inheritance when you can just say 'return {data: 'mydata'};' the code becomes obvious. Also with vert.x we open the polyglot and async doors at the same time.

The only I would say one container or framework is better than the rest is by measuring two things:

1) Performance: raw speed & scalability

2) Code complexity: number of lines of code

[theute]

Stefan,

what makes you say that WF Core is not lightweight (enough?). At some point you would need to add management capabilities to RHQ-Metrics (there are always things you want to configure)

The only I would say one container or framework is better than the rest is by measuring two things:

1) Performance: raw speed & scalability

2) Code complexity: number of lines of code

I partially agree here, but we won't develop each containers to get to this point so we need to take a decision early.

[ctomc]

Inline

Wildfly Core

I have some questions more so that pros/cons.

• Does using Wildfly Core require a JEE container? If so, then I see that as a con. <-- absolutly not, core has no EE functionality unless you want to add it.
• What is testability like. How easy is it to write integration tests? <-- arquillian, unit tests
• How easy/difficult to set up clustering? <-- just add infinispan subsystem and configure it
• How easy/difficult will it be to embed in other runtimes, e.g., RHQ? <-- just write your own extension and you can do whatever you want.

[ctomc]

Thomas Heute wrote:

• How easy/difficult will it be to embed in other runtimes, e.g., RHQ? It would basically be a web archive

web archive? no, if using vanilla wildfly core, it would not have support for deployment of web archives, unless you would add undertow subsystem for example.

You can embed any other runtime by writing extension that can add whatever functionality to server.

[ctomc]

Thomas Heute wrote:

Stefan,

what makes you say that WF Core is not lightweight (enough?). At some point you would need to add management capabilities to RHQ-Metrics (there are always things you want to configure)

The only I would say one container or framework is better than the rest is by measuring two things:

1) Performance: raw speed & scalability

2) Code complexity: number of lines of code

I partially agree here, but we won't develop each containers to get to this point so we need to take a decision early.

When it comes to code complexity based on number of lines, just make sure you are properly measuring it.

Currently WildFly Core code is part of whole WildFly code repository and you should really know what is what before you go to counting. We do plan to split code repositories up in coming months if that will help you with most irrelevant metric you can find.

On related topic you shouldn't be comparing apples to oranges especially as one of those would need to get many missing features of another to gain capabilities you need on the end.

[john.sanda]

ctomc thanks for the info. I am a little confused by your response to my question about embedding? Are you suggesting writing an extension to embed X into rhq-metrics? I am wondering about embedding rhq-metrics into X.

[ctomc]

John Sanda wrote:

Tomaz Cerar thanks for the info. I am a little confused by your response to my question about embedding? Are you suggesting writing an extension to embed X into rhq-metrics? I am wondering about embedding rhq-metrics into X.

By writing extension(subsystem) you get the most low level access to the server, which allows you to utilize what ever you want.

Only thing you are somewhat required to do is to expose attributes/resources/operations to management backbone which can than be manipulated by CLI, REST, Admin Console, ...

Writing extension is completely different than just dumping your application as deployment.

if you are interested you can take a look at my JUG presentation on this http://www.c2b2.co.uk/london_jbug_february_2014

[nstefan]

Tomaz Cerar can you give us a few details on the mechanics of implementing a REST interface along side a socket.io interface with WildFly Core? I am interested on what dependencies we would need to pull in and how the final structure of the project would look like.