Wire Protocol Definition Language

[jesper.pedersen]

Hi all,

I have created a tool for defining a wire protocol - which I call "Wire Protocol Definition Language" currently.

Goals:
* Ease the definition of a wire protocol
* Generate code to handle communication

Description:
WPDL is a tool that can define a wire protocol and can generate the necessary bindings for it in the specified programming language.

WPDL allows you to specify data structures using primitives (such as signed8, signed32, ...) that will form a packet to a target system. The idea is that all communication is done in old-school binary in order to reduce the overhead seen in modern protocols such as HTTP and WebServices.

The tool is designed such that any bindings can be generated such as Java, C, Ruby and so on. The Proof-of-Concept binding was java-io and a java-nio is in the works - others could be java-netty3, java-javassist, c-std, c++-std and ruby-std.

Technology:
* JAXB
* Code generator

Example:

 <packet name="Bind" frontend="true">
 <signed8 name="Identifier" content="B"/>
 <signed32 name="Length"/>
 <string name="DestinationPortal"/>
 <string name="PreparedStatement"/>
 <signed16 name="ParameterFormatCodesNumber"/>
 <signed16 name="ParameterFormatCodes" array="ParameterFormatCodesNumber"/>
 <signed16 name="ParameterNumber"/>
 <group name="Parameter" array="ParameterNumber">
 <signed32 name="ParameterLength"/>
 <signed8 name="ParameterValue" array="ParameterLength"/>
 </group>
 <signed16 name="ResultFormatCodesNumber"/>
 <signed16 name="ResultFormatCodes" array="ResultFormatCodesNumber"/>
 </packet>

Usage:
The binding generated will contain functionality to transmit or receive data from the server - this data is automatically parsed up as the packet structures defined.

Socket socket = new Socket(pgsqlServer);
Postgresql wire = new Postgresql(socket);

AuthenticationOk aok = wire.readAuthenticationOk();

Bind bind = new Bind();
bind.setDestinationPortal("");
...

wire.write(bind);

Let me know if there is an interested for such a project for Teiid.

[shawkins]

There's definitely interest in changing our JDBC driver's data access to a stable binary protocol to minimize forced client upgrades, improve performance, and provide a path for alternative language bindings and odbc access.

At this time though there's no internal bandwidth to make those changes in the 6.x releases. Also we haven't had users looking to implement connectors at the protocol level, but that may change with a broader open source audience.

I'd like to see us have more definitive ties to Teiid in the near term before we pull WPDL into the codebase and/or make it appear as a subproject. Perhaps someone following this thread will volunteer.

[jesper.pedersen]

The tool is not specific to JDBC protocols.

It is a tool where you can develop any binary protocol you want - there must be a lot of legacy systems out there which have their own binary protocol which could be interesting to hook up to Teiid.

The tool will help you with that - of course you have to implement the interaction with the packets on top of that, but having the wire layer abstracted to a f.ex. Java object model is a step in the right direction IMHO.

I see two major use-cases for the tool:

* Help the user to write a wire protocol to any application
* As a support library for the Teiid Connector framework

I'm more than willing to act as a project lead - of course the Teiid developers will have a lot more knowledge about how the tool could be integrated in the project.

[rareddy]

IMO this tool is lot more valuable as a stand alone project in JBoss tools area. By parking it as sub-project in Teiid it may lose its visibility out side the Teiid project. If we do not get further adoption by the community it will end up more of a liability for Teiid.

Certainly I am all in for making use of this project for Teiid JDBC protocol or any use cases that may come up with the custom connector development.