SOAP Webservice Pagination

[sgroucutt]

Hi,

We are using Teiid 8.8.1 and are experimenting with the SOAP webservice VDBs.

A target webservice of ours has a pagination mechanism via two supplied parameters to the call, "startIndex" and "endIndex".  We expect this source to occasionally provide lots of data, so we would like to configure Teiid to page through result sets to build the backing data for the VDB.  It is not clear whether Teiid would support this from documentation, or I haven't been able to find it.  Does Teiid support pagination of web service sources?  If so, is there an example project available (or is there a doc somewhere I missed that explains how to do this)?

Thanks very much,

Steve

[shawkins]

The basic web services layer would have no direct knowledge of pagination.  It's just a request / response procedure.  Higher level web translators are aware of pagination / batching, such as the salesforce and odata translators.

[rareddy]

Steve,

I do not know of any inherent pagination support from SOAP protocol, so if you need something like that then you need to build custom solution like you are proposing, with "startindex" and "endindex". This should be possible with Teiid. On the SOAP procedure that you are creating for this process, use the "startIndex", "length" as procedure parameters, then convert these parameters as LIMIT and OFFSET on the SQL query that is building your resultset. If your query is ordered, then it should get you results you are looking for. The assumption here is the data is not changing underneath the query. If the data is changing, then you need to materialize the result then use that as source for your query.

Ramesh..

[sgroucutt]

Thank you both - we will see if we can munge the start/end indexes into limit/offset as you suggest, Ramesh.  If we cannot, Stephen, your answer means we would explore writing our own translator that understands how to do this for this particular webservice source, correct?

[shawkins]

> our answer means we would explore writing our own translator that understands how to do this for this particular webservice source, correct?

Yes, especially if you want a table abstraction.

[rareddy]

Sorry, I missed the "target web service" part in original message. For custom scrolling of the results, an extended translator which knows the scrolling behavior is best option, ignore the LIMIT suggestion.

[sgroucutt]

OK, thank you both.  Would this also apply to the REST webservice area?  For instance, we have a different target service that has start/end index as query parameters.  Would that also be a translator on top that managed the pagination?

[rareddy]

Yes, equally applies to REST. The only only place this would not apply is defined protocols over SOAP or REST like SalesForce API and OData API, as they define the pagination in their API explicitly and they have been captured in their respective translators.