-
1. Re: need help getting JENA JDBC to work with Teiid
rareddy Oct 14, 2016 9:37 AM (in response to omarbkhan)Omar,
Currently Teiid does not have translator for the JENA. If this were any JDBC driver that supports SQL, there are some ansi compatible built-in translators that you could use, however this is SPARQL based query language it mandates the requirement to develop a custom translator where relational constructs to the RDF model need to be derived. When we looked at this feasibility of RDF to relational model, we were not completely satisfied with "a" solution. The issue is captured here [TEIID-3126] Add OWL / RDF translators to TeiiD - JBoss Issue Tracker
If you want to help contribute such translator, please let us know, as we could use a subject matter expert in RDF.
Ramesh.. -
2. Re: need help getting JENA JDBC to work with Teiid
omarbkhan Oct 15, 2016 11:34 AM (in response to rareddy)Ramesh,
I do have interest in your proposal. I'm very passionate about semantics/RDF/OWL
I hope to have some time next year to lightly be engaged in this, my bandwith is very limited. This may give me also a bit more time to understand the translator programming model & docs. No doubt, I probably will need some hand holding - but hopefully that will be a minimum as I get more adept with Teiid and actutally the Jena programming model to do the integration.
Thank you for the offer, with a bit of good fortune I'll be able to help.
Omar
-
3. Re: need help getting JENA JDBC to work with Teiid
omarbkhan Oct 31, 2016 1:48 PM (in response to rareddy)Ramesh,
I got a bit further with Jena, and partial success.I encountered an issue and reached out to the JDBC maintainers, and got the below with some helpful information - but don't understand what steps I should take next in Teiid ?
info from author in thread:
http://jena.markmail.org/thread/pxwjulkqiprttdhi
jira created to address issue:
[JENA-1255] jena-jdbc-bundle does not consolidate ServiceLoader files. - ASF JIRA
I believe I have all of the jars in the classpath and module.xml, so this is quite curious - the author believes it has to do with ServiceLoader if I'm interpreting it correctly and we should still be able to get it to work as is?
Any suggestions? I'm not sure what the author means by repackaging?
-
4. Re: need help getting JENA JDBC to work with Teiid
rareddy Oct 31, 2016 9:12 PM (in response to omarbkhan)It is not standard extension of the JDBC Driver load, so it is custom service loader specific to JENA. Ask him who is responsible for loading it, and how does one load it? They should have bootstrapped using the standard JDBC4 Driver service loader mechanism to initialize it, then we would not have issues.
I was going to ask on email thread but having some MarkMail issues.
-
5. Re: need help getting JENA JDBC to work with Teiid
omarbkhan Nov 1, 2016 12:36 PM (in response to rareddy)Hi Ramesh,
Andy responded to the questions I asked, can you review?
http://markmail.org/message/jivwjcv7uji7imro
I'm not sure how we would call ARQ.init() from module.xml?
-
6. Re: need help getting JENA JDBC to work with Teiid
rareddy Nov 1, 2016 8:23 PM (in response to omarbkhan)Omar,
Now I can not even get to MarkMail to read the response. Basically what they need to do, replace their ServiceLoader mechanism with standard JDBC4 ServiceLoader mechanism like
teiid/client/src/main/resources/META-INF/services at master · rareddy/teiid · GitHub
That will register their Driver class, and in their driver class possibly in a "static block" call ARQ.init() that will initialize every time one uses their JDBC driver. The issue is not with the JBoss modules, it is how JENA driver designed.
Ramesh..
-
7. Re: need help getting JENA JDBC to work with Teiid
andy.seaborne Nov 2, 2016 5:40 AM (in response to rareddy)Hi Ramesh,
Apache Jena JDBC does use java.sql.Driver. Omars stacktrace shows that Jena JDBC code has been loaded and executed.
at org.apache.jena.jdbc.statements.JenaStatement.execute(JenaStatement.java:273) <----- Jena JDBC code
at org.jboss.jca.adapters.jdbc.WrappedStatement.execute(WrappedStatement.java:182)
at org.teiid.translator.jdbc.JDBCDirectQueryExecution.execute(JDBCDirectQueryExecution.java:83)
... 17 more
So the issue is not about finding the driver as far as I can see.
Jena uses ServiceLoader itself and the report is indicative of that part not being configured properly for org.apache.jena.system.JenaSubsystemLifecycle.
Jena has several service files for the same interface - if combing jena jars, then that process needs to combine service files for org.apache.jena.system.JenaSubsystemLifecycle (for example, the maven shade plugin ServicesResourceTransformer does this)
There was an issue with the bundled jena-jdbc-driver-bundle (now fixed) but Omar reports an issue when using the driver in unbundled form via
<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.0" name="org.apache.jena">
<resources>
<resource-root path="jena-jdbc-core-3.1.0.jar"/>
<resource-root path="jena-jdbc-driver-remote-3.1.0.jar"/>
<resource-root path="jena-arq-3.1.0.jar"/>
<resource-root path="jena-core-3.1.0.jar"/>
<resource-root path="jena-base-3.1.0.jar"/>
<resource-root path="jena-cmds-3.1.0.jar"/>
...
and the stacktrace suggests incomplete Jena initialization.
Andy
-
8. Re: need help getting JENA JDBC to work with Teiid
omarbkhan Nov 2, 2016 9:56 AM (in response to rareddy)(Ramesh - permalink for more details (not MarkMail))
-
9. Re: need help getting JENA JDBC to work with Teiid
rareddy Nov 2, 2016 12:07 PM (in response to andy.seaborne)I read through the messages on the link now.
The JBoss module system (module.xml) does NOT bundle the jars. It is another modular class loader. There is no code it can automatically run (it can but its own extensions), it does not understand the JenaLifeCycle service loader. Does calling
JenaSystem.init() ;
enough? if I were to write a plain JAVA application, how is this code being initialized?
Ramesh..
-
10. Re: need help getting JENA JDBC to work with Teiid
andy.seaborne Nov 2, 2016 4:00 PM (in response to omarbkhan)JenaSystem.init() is called by various entry points into the code including NodeValue class initialization.
So (1) it is being called (2) it is not fully initializing Jena.
Do explicit calls would force the sequence outside of services files:
InitJenaCore.init();
RIOT.init();
ARQ.init();
before calling any Jena code or loading the JDBC driver.
-
11. Re: need help getting JENA JDBC to work with Teiid
andy.seaborne Nov 2, 2016 4:30 PM (in response to omarbkhan)Ah - "modular class loader"
Forgive my ignorance about the system but is there one classloader for all the jars in the module?
I checked - Jena's JDBC driver RemoteEndpointDriver does call jena system initialization in a class static initializer.
-
12. Re: need help getting JENA JDBC to work with Teiid
rareddy Nov 2, 2016 9:18 PM (in response to andy.seaborne)Andy,
Does JenaSystem.init() in turn execute
InitJenaCore.init();
RIOT.init();
ARQ.init();
or both separate initialization stacks and both are needed?
>Forgive my ignorance about the system but is there one classloader for all the jars in the module?
Yes, it is single class loader per module.xml, if it has <dependencies> section those are separate modules that this one depends upon
-
13. Re: need help getting JENA JDBC to work with Teiid
omarbkhan Nov 4, 2016 1:06 AM (in response to rareddy)Ramesh,
I was able to get a simple Java program to query a RDF triple store using the JDBC Jena driver to get a better understanding of the process, but not from Teiid using the driver bundle.
I tried to recreate the lack of initialization NodeValue in the simple Java program, but was unable to create the failure as in Teiid.
The above stated, I did notice some more details on the NodeValue error in Teiid (which did not occur in the simple Java program) :
Caused by: java.lang.ClassCastException: __redirected.__DatatypeFactory cannot be cast to javax.xml.datatype.DatatypeFactory
at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source)
at org.apache.jena.sparql.expr.NodeValue.getDatatypeFactory(NodeValue.java:201)
at org.apache.jena.sparql.expr.NodeValue.<clinit>(NodeValue.java:150)
... 52 more
Andy - I wonder if it's org.apache.xerces vs javax.xml? In that the way JBoss's module loader is invoking it. could it be this
The bundle jar I used was here (it may be helpful to download it to review it, I'm not sure if I can explain it's layout in an e-mail very well):
Index of /repositories/snapshots/org/apache/jena/jena-jdbc-driver-bundle/3.1.2-SNAPSHOT
I think/suspect after looking at the jars what could be going on is the following, perhaps (Andy please correct me) :
Teiid is picking up META-INF/services/java.sql.Driver which contents have : org.apache.jena.jdbc.remote.RemoteEndpointDriver
but... is not picking up META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle
Which contents has :
org.apache.jena.riot.system.InitRIOT
org.apache.jena.sparql.system.InitARQ
org.apache.jena.system.InitJenaCore
I wonder if there is something in Teiid we could leverage to call the org.apache.jena.system.JenaSubsystemLifecycle and other various initializations, maybe we need a bunch of dependencies loaded / initialized first, there are many other files in the META-INF/services folder that look like they are initializing ? If i take a look at the bundle, there are a bunch of other initializations taking place, so I'm guessing we need it all for it to work in Teiid?
Here is part of my program ...I tried different ways, manually included all of the jars , tried with and without the bundle, and didn't even need to do the direct initialization, it worked fine with and with out.- but did it anyway I'm guessing because the jars did it for me ???
I was going to try to query DBPedia , but I'm blocked by a proxy server right now to get there, so it's really not the best example (perhaps Andy has something better) ; it did work on our internal triple store remote endpoint fine.So I had to remove the queries unfortunately.
import java.sql.SQLException;
import java.sql.Driver;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;
import java.sql.Connection;
import java.util.ArrayList;
import java.util.List;
import org.apache.jena.jdbc.JdbcCompatibility;
import org.apache.jena.jdbc.connections.JenaConnection;
import org.apache.jena.jdbc.remote.connections.RemoteEndpointConnection;
import org.apache.jena.query.ARQ;
import org.apache.jena.riot.RIOT;
import org.apache.jena.system.InitJenaCore;
public class HelloJenaJDBC {
public static void main(String[] args) {
// I don't think we need to do the below 3 initializations because the jars do it anyhow???? It worked fine with and without.
InitJenaCore.init();
RIOT.init();
ARQ.init();
Statement stmt = null;
try {
Connection conn = DriverManager.getConnection("jdbc:jena:remote:query=http://<servername>","user","password");
// Need a statement
stmt = conn.createStatement();
// Make a query
ResultSet rset = stmt.executeQuery(" query here ");
// Iterate over results
while (rset.next()) {
// Print out type as a string
System.out.println(rset.getString(" sparql variable here without the ? "));
}
// Clean up
rset.close();
} catch (SQLException e) {
System.err.println("SQL Error - " + e.getMessage() );
e.printStackTrace();
} finally {
try {
stmt.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
-
14. Re: need help getting JENA JDBC to work with Teiid
rhn-engineering-rareddy Nov 4, 2016 1:39 AM (in response to omarbkhan)>>I wonder if it's org.apache.xerces vs javax.xml? In that the way JBoss's module loader is invoking it.
JBoss also uses xercess, and I believe the parent class loader already have this module loaded thus you may be seeing this issue, I would say this may be core of the issue.
>>but... is not picking up META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle
It will not, as this is JENA's own extension mechanism. But I see that this is being executed through Driver extension, but hitting above issue IMO.
Can you try replacing you module.xml like below (do not use bundle jar)
<module xmlns="urn:jboss:module:1.3" name="org.apache.jena"> <properties> <property name="jboss.api" value="private"/> </properties> <resources> <resource-root path="collection-0.6.jar"/> <resource-root path="commons-csv-1.0.jar"/> <resource-root path="httpclient-4.2.6.jar"/> <resource-root path="httpclient-cache-4.2.6.jar"/> <resource-root path="httpcore-4.2.5.jar"/> <resource-root path="jena-arq-3.1.0.jar"/> <resource-root path="jena-base-3.1.0.jar"/> <resource-root path="jena-cmds-3.1.0.jar"/> <resource-root path="jena-core-3.1.0.jar"/> <resource-root path="jena-iri-3.1.0.jar"/> <resource-root path="jena-shaded-guava-3.1.0.jar"/> <resource-root path="jena-tdb-3.1.0.jar"/> <resource-root path="jsonld-java-0.7.0.jar"/> </resources> <dependencies> <module name="javax.api"/> <module name="org.slf4j"/> <module name="org.apache.log4j"/> <module name="org.apache.commons.cli"/> <!--1.2 needs 1.3--> <module name="org.apache.commons.codec"/> <!-- 1.10 needs 1.9 --> <module name="org.apache.commons.io"/> <module name="org.apache.commons.lang" slot="3.3.2"/> <module name="com.fasterxml.jackson.core.jackson-core"/> <module name="com.fasterxml.jackson.core.jackson-annotations"/> <module name="com.fasterxml.jackson.core.jackson-databind"/> <module name="org.apache.thrift" /> <!--9.1 needs 9.2 --> <module name="org.apache.log4j" /> <module name="org.slf4j" /> <module name="org.slf4j.jcl-over-slf4j" /> <module name="org.slf4j.ext"/> <module name="org.apache.xerces"/> <module name="javax.xml.bind.api"/> <module name="javax.xml.rpc.api"/> <module name="javax.xml.soap.api"/> <module name="javax.xml.stream.api"/> <module name="javax.xml.ws.api"/> </dependencies> </module>