-
1. Re: Web Scrappping Teiid Translator
shawkins Oct 8, 2013 3:51 PM (in response to rokhmanov)After looking at their api could this be done as a UDF (passing in a blob and charset) and leave the html retrieval to the ws translator? Or does allowing jsoup to make the connection have benefits?
-
2. Re: Web Scrappping Teiid Translator
rokhmanov Oct 8, 2013 10:06 PM (in response to shawkins)I think it is possible to reuse ws translator for html retrieval purposes. My goal was simplicity - a whole logic is two independent classes with less than a dozen lines excluding Embedded Teiid boilerplate code, no other particular benefits. But having managed connector would be a better design I guess. I'll try to come up with udf approach sometime during this or next week.
-
3. Re: Web Scrappping Teiid Translator
shawkins Oct 11, 2013 1:22 PM (in response to rokhmanov)In case you missed it, Ramesh posted your work on the blog - http://teiid.blogspot.com/2013/10/bring-html-pages-into-relational-world.html
> But having managed connector would be a better design I guess.
It all depends upon whether someone cares about externalizing the urls - and on whether the web resource has httpbasic or other access concerns that are already handled by the resource adapter.
-
4. Re: Web Scrappping Teiid Translator
rokhmanov Oct 15, 2013 5:24 PM (in response to rokhmanov)I've added a new stored procedure to work with built-in Teiid WS translator (it seems simpler to retrieve a resultset using SP than a custom function). See the code and example updated on GitHub.
-
5. Re: Web Scrappping Teiid Translator
shawkins Oct 16, 2013 8:31 AM (in response to rokhmanov)Yes, we have not yet added the ability to directly return result sets from functions yet (an array of arrays would be a workaround - but may not be very memory safe). What you have looks good, thanks Andriy.
Steve