-
1. Re: Custom connector/translator for REST API
rareddy Mar 20, 2015 2:54 PM (in response to vagelis.pertsinis)Vagelis,
Teiid 8.9 introduced feature to include headers in the invokeHTTP. See [1] and documentation for it is at [2].
As per the writing your own translator, that is fine too. The best thing to do probably is to extend the WSExecutionFactory [3] or like it. If you see "getMetadata" method, there you can define another procedure like invokeHttp that can take headers and any other stuff you need. Then you can package and deploy the translator. For general translator development see [4]. Once this translator is deployed, you can use Teiid Designer's connection importer to import metadata, that will expose the invokeHttp variant function that you created. You that in your models and develop a VDB. But first I suggest using the Teiid 8.10 version then go through all these steps if that does satisfy your usecase.
[1] https://issues.jboss.org/browse/TEIID-3016
[2] Web Services Translator - Teiid 8.11 (draft) - Project Documentation Editor
[3] teiid/WSExecutionFactory.java at master · teiid/teiid · GitHub
[4] Developer's Guide - Teiid 8.11 (draft) - Project Documentation Editor
Ramesh..
-
2. Re: Custom connector/translator for REST API
vagelis.pertsinis Mar 24, 2015 1:31 PM (in response to rareddy)Hi Ramesh,
Thanks for the reply. Red Hat's data virtualization is using Teiid 8.7.1. Instead of updating the whole runtime, which will cause issues all over the place, we thought to copy the translator and webservice from another server which is running Teiid 8.9.1 runtime.
That said, the server starts fine and everything seems to be working fine but we still cant find a way to pass in the headers correctly.
InvokeHttp is expecting:
- action
- request
- endpoint
- stream
- result
- contentType
According to your post we should be able to doinvokeHttp(... headers=>jsonObject(
'application/json'
as ContentType, jsonArray(
'gzip'
,
'deflate'
) as
"Accept-Encoding"
))
To accomplish the above we added headers as child of invokeHttp with a type of clob. Is that correct?
Then what we have is
BEGIN DECLARE STRING VARIABLES.qp = ''; SELECT data AS data, successMessage AS successMessage FROM (EXEC NewAdaptorSourceModel.invokeHttp(action => 'POST', endpoint => qp, stream => 'TRUE', headers => JSONOBJECT('application/json' AS ContentType))) AS f, XMLTABLE('/response' PASSING JSONTOXML('response', result) COLUMNS data string PATH 'data/text()', successMessage string PATH 'successMessage/text()') AS A;
END
That throws the following error:
select * from ( exec "NewAdaptorViewModel"."getAPIDataNew"() ) AS X_X org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidException: null
Is there something obvious we're missing? This the first time we use Teiid and chances are we're not using something correctly.
Cheers
Vagelis
-
3. Re: Custom connector/translator for REST API
rareddy Mar 24, 2015 2:29 PM (in response to vagelis.pertsinis)What is exception in the server log?
-
4. Re: Custom connector/translator for REST API
vagos7 Mar 26, 2015 1:36 PM (in response to rareddy)Hi Ramesh,
After we started using Teiid 8.9.1 I'm not seeing exceptions in the logs. We still can't pass the headers to the request though. Do you see anything wrong with the following code:
BEGIN
DECLARE STRING VARIABLES.qp = '';
SELECT A.batchId AS batchId FROM (EXEC SourceModelAPI.invokeHttp(action => 'POST', endpoint => VARIABLES.qp, stream => 'TRUE', headers => JSONOBJECT('application/json' AS ContentType, '7004bbf10cff435f8fc961a0d20a9ced' AS Token)) AS f, XMLTABLE('/response' PASSING JSONTOXML('response', f.result) COLUMNS batchId string PATH 'data/text()') AS A;
END
In the SourceModel I added headers as a clob parameter. That correct, right?
-
5. Re: Custom connector/translator for REST API
rareddy Mar 26, 2015 1:44 PM (in response to vagos7)Yes, that is correct. The real definition of the procedure is defined in the "ws" translator [1] line #165, it then needs to match with the Designer version for modeling purposes. You can also use importer "Teiid Connection Importer" in the Designer, and point to the translator to retrieve invokeHttp metadata too.
So, now that you have done it manually and has supporting transformation, what is the result?
[1] teiid/WSExecutionFactory.java at 8.9.x · teiid/teiid · GitHub
-
6. Re: Custom connector/translator for REST API
vagos7 Mar 26, 2015 2:08 PM (in response to rareddy)You lost me there. Just so we're on the same page, do we need customise the 'ws' translator or else it won't work? We're using Teiid Deisgner Tools 9.0.1. The message from the server is:
select * from ( exec "ViewModelPOST"."getDataAPI"() ) AS X_X
org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidProcessingException: TEIID30504 SourceModelAPI: TEIID15005 Error Calling HTTP Service - 403 Forbidden
That essentially means the request headers are not passed correctly to the API.
-
7. Re: Custom connector/translator for REST API
shawkins Mar 26, 2015 2:33 PM (in response to vagos7)> You lost me there
Unfortunately Teiid Designer was not properly in sync with the actual metadata on the server. So what Ramesh is saying is that the server expects result to be the first parameter and marked as a return. There is a check specifically on the server side for legacy compatibility with the Designer such that if the first parameter is not a return, then we'll not look for the header value.
Steve
-
8. Re: Custom connector/translator for REST API
rareddy Mar 26, 2015 3:07 PM (in response to vagos7)1 of 1 people found this helpfulMay be this screencast will help to align the metadata
-
9. Re: Custom connector/translator for REST API
vagelis.pertsinis Mar 27, 2015 7:34 AM (in response to rareddy)Hi Ramesh,
I can't thank you enough for your help so far! I've set up the source model according to your screencast but now I'm getting the null value exception again:
org.teiid.runtime.client.TeiidClientException: java.lang.RuntimeException: Remote org.teiid.core.TeiidException: null [1]
I tried something very simple just to see if I can get a response back.
BEGIN
DECLARE STRING VARIABLES.qp = 'http://****/site.version';
SELECT A.data AS data FROM (EXEC APISource.invokeHttp(action => 'GET', endpoint => VARIABLES.qp, stream => 'TRUE')) AS f, XMLTABLE('/response' PASSING JSONTOXML('response', f.result) COLUMNS data string PATH 'data/text()') AS A;
END
[1] Console Output
10:56:50,409 ERROR [org.teiid.CONNECTOR] (Worker28_QueryProcessorQueue72) 8YBVhrPbIPV2 Connector worker process failed for atomic-request=8YBVhrPbIPV2.0.3.22: java.lang.NullPointerException
at org.teiid.translator.ws.BinaryWSProcedureExecution.execute(BinaryWSProcedureExecution.java:119) [translator-ws-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:327) [teiid-engine-8.9.1.jar:8.9.1]
at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) [:1.8.0_40]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_40]
at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_40]
at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:209) [teiid-engine-8.9.1.jar:8.9.1]
at com.sun.proxy.$Proxy47.execute(Unknown Source)
at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:298) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:110) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:107) [teiid-engine-8.9.1.jar:8.9.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_40]
at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:58) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:274) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.9.1.jar:8.9.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_40]
10:56:50,409 ERROR [org.teiid.PROCESSOR] (Worker27_QueryProcessorQueue73) 8YBVhrPbIPV2 TEIID30019 Unexpected exception for request 8YBVhrPbIPV2.0: java.lang.NullPointerException
at org.teiid.translator.ws.BinaryWSProcedureExecution.execute(BinaryWSProcedureExecution.java:119)
at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:327) [teiid-engine-8.9.1.jar:8.9.1]
at sun.reflect.GeneratedMethodAccessor90.invoke(Unknown Source) [:1.8.0_40]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_40]
at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_40]
at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:209) [teiid-engine-8.9.1.jar:8.9.1]
at com.sun.proxy.$Proxy47.execute(Unknown Source)
at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:298) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:110) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:107) [teiid-engine-8.9.1.jar:8.9.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_40]
at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:58) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:274) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119) [teiid-engine-8.9.1.jar:8.9.1]
at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210) [teiid-engine-8.9.1.jar:8.9.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_40]
-
10. Re: Custom connector/translator for REST API
shawkins Mar 27, 2015 7:47 AM (in response to vagelis.pertsinis)1 of 1 people found this helpfulMake sure that the result parameter is marked as the return and not just an out parameter.
-
11. Re: Custom connector/translator for REST API
vagelis.pertsinis Mar 27, 2015 9:12 AM (in response to shawkins)You sir are correct! Changing it to RESULT solved the issue.
I ll mark Ramesh's answer as correct and yours as helpful.
On another note, if you can have a look at this post and advise on the feasibility it would be amazing.
Thank you both,
Vagos