Is materializing entire models possible?

[m.ardito]

Hi,

I am learning materializazion benefits, eg: for some slow models, I can create views and with specific options, instruct Teiid to materialize those viewws, even with autorefresh, lazy or not.

Wonderful! This is really nice particularly when I have a view joining data between fast and slow datasources, because in this way all joined data is local to Teiid and applications response is faster, even if some data is old due to the refresh interval.

But sometimes I could neeed to have nearly all data on those slow datasources (eg: google spreadsheets, big csv files, etc) ready for queries.

But to do this, I need to create, for each of those models, a view that repeats all model fields, so that later the view can be configured for materialization...

in this case I don't need to alter the model DDL, or join other data, I need the exact model, as Teiid retrievs it, but local, with all the nice materialization refresh options.

So, what would be fantastic would be an option to materialize the entire model, if possible - note, there may be technical problems I don't know about. I'm just dreaming

Could this be already done, maybe with some query caching technique I don't know about (yet), or requested?

Marco

[rareddy]

Marco,

Yeah, we implemented a long time ago and did not work out as we expected with users and customers then we switched to using materializing on individual tables.

Ramesh..

[m.ardito]

Thanks.

but is there a way to materialize even a single table without having to create a view that basically just reads all table fields, in order to materialize it? I just know this method, I'll dig more through docs....

but please point me in useful directions, if you feel..

Marco

[shawkins]

> but is there a way to materialize even a single table without having to create a view that basically just reads all table fields, in order to materialize it?

Not currently as there are some implementation limitations that make it easier to have the view requirement.  Typically you do want to be selective about materialization as it's best applied as close as possible to the user queries - after federation and other processing.

It is possible to introduce a delegating or custom translator to pragmatically add materialized views at import time based upon the underlying metadata.

[m.ardito]

Steven Hawkins ha scritto:

>Not currently as there are some implementation limitations that make it easier to have the view requirement.

ok

>Typically you do want to be selective about materialization as it's best applied as close as possible to the user queries - after federation and other processing.

of course, I was thinking about corner cases, like simple one-file datasources like google spreadsheets (of course ) which are abit slow to load, on my internet connection

I thought that if they could be auto-materialized, when users may want to add/remove/rename a new column (I could lock the fundamental ones) , their change could be reflected automatically, somehow.

>It is possible to introduce a delegating or custom translator to pragmatically add materialized views at import time based upon the underlying metadata.

Interesting, any example around?

As always, thanks,

Marco

[rareddy]

Take look at teiid/BaseDelegatingExecutionFactory.java at master · rareddy/teiid · GitHub  and documentation at Delegating Translators | Teiid Documentation

Basically it is wrapper over any translator for cross cutting concerns, so here you would develop a class extending the BaseDelegatingExecutionFactory, and override the getMetadata method and create automatic view tables based on the metadata gathered from super.getMetadata method.