Skip navigation
2014

Teiid Designer Development

January 2014 Previous month Next month

Subsequent to my previous post, I was feeling really good about the teiid runtime client plugin. Having successfully created a QueryParser which constructs either a Teiid7Parser or a Teiid8Parser depending on a given TeiidServerVersion, one even purloined the Teiid unit tests in order to fully test-drive the parser and AST nodes. This was a great move given the number of bugs and omissions in the nodes.

 

Falling Down the Rabbit Hole!

After the parser, comes the SyntaxFactory (factory for creating AST Nodes in Designer). The inevitable conclusion of this is that the SyntaxFactory requires implementing the SPI interface ISyntaxFactory (otherwise Designer would have to have 768 problems to fix). One likes the SPI plugin given it has decoupled Designer's required interfaces from their implementations. Some of the interfaces are a little odd due to the use of generics but that has aided its decoupling and its flexibility. However, ISyntaxFactory demands that all of the AST nodes implement their respective interfaces, eg. Function implements IFunction. This has just been a laborious day-and-a-half of grunt-work. It could have been automated but doing it manually was of benefit in cleaning up some code.

 

One of the decisions taken a month or so back was to NOT depend on any Teiid library. One of the reasons was the DataTypeManager class which is used pervasively for dealing with types. This class added new types between versions 7.7 and 8.0 hence a fresh version was needed that could be decorated with annotations allowing for a TeiidServerVersion to dictate which types would be returned by a call from any other Designer/Teiid code. This seems quite a small thing but has become most fundamental to ensuring backward compatibility in a client able to understand the limitations of an older version over a newer one. Consequently, the plugin could not just depend on Teiid's Engine library as their internal DataTypeManager would get used by internal classes and the plugin would be back to square 1!

 

Turns out not depending on the Teiid library is potentially very beneficial from the perspective of really understanding what Teiid code is really required but at the same time inevitably involves porting over classes from the Teiid codebase and as one might expect one class tends to depend on another. For example, in resolving the Function AST node, a FunctionDescriptor field is initialised. In order to port over this field, it would require the FunctionMethod and FunctionParameter classes as well. Likewise, the QueryNode is going to require UpdateInfo which seems to require the UpdateValidator and a whole group of related scary classes. Am I just copying Teiid 1 class at a time?

 

Where Next?

The goal of this work was to construct a backward-compatible Teiid client that could parse, resolve and validate an SQL string. The parsing was the problem for current Teiid releases in that the parser just was never designed to be backward-compatible. That problem has at least a solution, while the resolver and validator are 'already backward-compatible' hence one should be able to plug straight into them? Well... not quite since the resolver takes a Command AST node created from the parser, one cannot just give Teiid's resolver any old AST node; it requires Teiid's version and again backward-compatibility disappears since a Teiid 8 Command has no idea what an HasCriteria node is anymore since that was only in Teiid 7.

 

The only possibility is to take the Resolver and its ancillary component classes and make them deal with the new parser's AST nodes. There seems little alternative?? ... still a bit of a break might shed some light on the subject!

 

Back to code ... (not before a little break!)


PGR

It has long been an issue in Designer that the development of the Teiid project has meant supporting multiple versions of Teiid when parsing, resolving and validating SQL transformations. At first, a single set of teiid client libraries was added to the Designer codebase, implying a 1-1 relationship with a version of Teiid. When Designer 8.1 came along, an attempt was made (by me) to support multiple versions of Teiid.

 

Broadly speaking, this has appeared to work (albeit a rather crude implementation). A user can choose a Teiid version synonymous with their Teiid instance and a default version is set in Designer. The user then has confidence that SQL validation in Designer has parity with what will happen on the . The whole idea being that the model validates in Designer and so will validate when deployed to Teiid.

 

The downside... (always is really!)

The development model used whereby the Teiid codebase is forked and a new submodule is generated in the Designer codebase is unsustainable. There are already 3 submodules (7.7, 8.3, 8.4)! Strictly speaking, 8.3 is unnecessary but at the time there was always some question over compatibility between Teiid 8.3 and 8.4 so for historical reasons it lives on.

 

With the advent of the next generation of Designer tool (codenamed Komodo), we wanna move totally away from submodules BUT we need a teiid client runtime that can still support multiple versions of Teiid. This is tricky since some objects in Teiid are backward-compatible, eg. the resolver, but other objects are not, eg. the parser, admin client (between 7 and 8). We need to support all of them.

 

So, what's the plan??

Jboss tools has the same issues in supporting multiple versions. One of that project's solutions is to provide generic interfaces and where objects have to differ, create internal version-specific implementations but present them under the one interface. Thus, one teiid-client-runtime plugin that is wholly backward-compatible between Teiid versions (back to 7.7.x) and where necessary implements v7, v8 ... objects that are used depending on a required Teiid version.

 

Simple really!

Indeed, but getting the framework up and running is a challenge and just getting 2 versions of the parser up to parity has taken quite an effort. Current work is maintained on my teiid-runtime branch (careful! I force push this branch with any work-in-progress so let me know if you pull from it).

 

Back to code...


PGR

The Teiid Designer team will be posting development comments here designed to aid in collaboration with each others' efforts as well as informing the community of development progress.

 

Feel free to follow along!

 

Back to code...

 

PGR