1 2 Previous Next 16 Replies Latest reply on Nov 4, 2016 1:30 PM by rareddy

    need help getting JENA JDBC to work with Teiid

    omarbkhan

      Hi - I've been trying to get the JENA JDBC driver Apache Jena - Jena JDBC - A SPARQL over JDBC driver framework  to work with Teiid.

       

      I'm able to perform a query in the SQL Scrapbook and see results return in the results window while in the database context, but modal dialog saying the query is still executing isn't dismissed? Any ideas how to get this to work with Teiid?

       

      Having a RDF triple-store remote endpoint datasource would really help us in our integration efforts, is there any info I could provide that would help clarify what I'm trying to do if the above isn't clear?

        • 1. Re: need help getting JENA JDBC to work with Teiid
          rareddy

          Omar,

           

          Currently Teiid does not have translator for the JENA. If this were any JDBC driver that supports SQL, there are some ansi compatible built-in translators that you could use, however this is SPARQL based query language it mandates the requirement to develop a custom translator where relational constructs to the RDF model need to be derived. When we looked at this feasibility of RDF to relational model, we were not completely satisfied with "a" solution. The issue is captured here [TEIID-3126] Add OWL / RDF translators to TeiiD - JBoss Issue Tracker

           

          If you want to help contribute such translator, please let us know, as we could use a subject matter expert in RDF.


          Ramesh..

          • 2. Re: need help getting JENA JDBC to work with Teiid
            omarbkhan

            Ramesh,

             

            I do have interest in your proposal.   I'm very passionate about semantics/RDF/OWL

             

            I hope to have some time next year to lightly be engaged in this, my bandwith is very limited. This may give me also a bit more time to understand the translator programming model & docs. No doubt, I probably will need some hand holding - but hopefully that will be a minimum as I get more adept with Teiid and actutally the Jena programming model to do the integration.

             

            Thank you for the offer, with a bit of good fortune I'll be able to help.

             

            Omar

            • 3. Re: need help getting JENA JDBC to work with Teiid
              omarbkhan

              Ramesh,


              I got a bit further with Jena, and partial success.

               

              I encountered an issue and reached out to the JDBC maintainers, and got the below with some helpful information - but don't understand what steps I should take next in Teiid ?

               

              info from author in thread:

              http://jena.markmail.org/thread/pxwjulkqiprttdhi

              jira created to address issue:

              [JENA-1255] jena-jdbc-bundle does not consolidate ServiceLoader files. - ASF JIRA

               

              I believe I have all of the jars in the classpath and module.xml, so this is quite curious - the author believes it has to do with ServiceLoader if I'm interpreting it correctly and we should still be able to get it to work as is?

               

              Any suggestions? I'm not sure what the author means by repackaging?

              • 4. Re: need help getting JENA JDBC to work with Teiid
                rareddy

                It is not standard extension of the JDBC Driver load, so it is custom service loader specific to JENA. Ask him who is responsible for loading it, and how does one load it? They should have bootstrapped using the standard JDBC4 Driver service loader mechanism to initialize it, then we would not have issues.

                 

                I was going to ask on email thread but having some MarkMail issues.

                • 5. Re: need help getting JENA JDBC to work with Teiid
                  omarbkhan

                  Hi Ramesh,

                   

                  Andy responded to the questions I asked, can you review?

                   

                  http://markmail.org/message/jivwjcv7uji7imro

                   

                  I'm not sure how we would call ARQ.init() from module.xml?

                  • 6. Re: need help getting JENA JDBC to work with Teiid
                    rareddy

                    Omar,

                     

                    Now I can not even get to MarkMail to read the response. Basically what they need to do, replace their ServiceLoader mechanism with standard JDBC4 ServiceLoader mechanism like

                     

                    teiid/client/src/main/resources/META-INF/services at master · rareddy/teiid · GitHub

                     

                    That will register their Driver class, and in their driver class possibly in a "static block" call ARQ.init() that will initialize every time one uses their JDBC driver. The issue is not with the JBoss modules, it is how JENA driver designed.

                     

                    Ramesh..

                    • 7. Re: need help getting JENA JDBC to work with Teiid
                      andy.seaborne

                      Hi Ramesh,

                       

                      Apache Jena JDBC does use java.sql.Driver. Omars stacktrace shows that Jena JDBC code has been loaded and executed.

                       

                      at org.apache.jena.jdbc.statements.JenaStatement.execute(JenaStatement.java:273)   <----- Jena JDBC code

                        at org.jboss.jca.adapters.jdbc.WrappedStatement.execute(WrappedStatement.java:182)

                        at org.teiid.translator.jdbc.JDBCDirectQueryExecution.execute(JDBCDirectQueryExecution.java:83)

                        ... 17 more

                       

                      So the issue is not about finding the driver as far as I can see.

                       

                      Jena uses ServiceLoader itself and the report is indicative of that part not being configured properly for org.apache.jena.system.JenaSubsystemLifecycle.

                       

                      Jena has several service files for the same interface - if combing jena jars, then that process needs to combine service files for org.apache.jena.system.JenaSubsystemLifecycle (for example, the maven shade plugin ServicesResourceTransformer does this)

                       

                      There was an issue with the bundled jena-jdbc-driver-bundle (now fixed) but Omar reports an issue when using the driver in unbundled form via

                       

                      <?xml version="1.0" encoding="UTF-8"?>

                      <module xmlns="urn:jboss:module:1.0" name="org.apache.jena">

                        <resources>

                        <resource-root path="jena-jdbc-core-3.1.0.jar"/>

                        <resource-root path="jena-jdbc-driver-remote-3.1.0.jar"/>

                        <resource-root path="jena-arq-3.1.0.jar"/>

                        <resource-root path="jena-core-3.1.0.jar"/>

                        <resource-root path="jena-base-3.1.0.jar"/>

                        <resource-root path="jena-cmds-3.1.0.jar"/>

                      ...

                       

                      and the stacktrace suggests incomplete Jena initialization.

                       

                          Andy

                      • 8. Re: need help getting JENA JDBC to work with Teiid
                        omarbkhan
                        • 9. Re: need help getting JENA JDBC to work with Teiid
                          rareddy

                          I read through the messages on the link now.

                           

                          The JBoss module system (module.xml) does NOT bundle the jars. It is another modular class loader. There is no code it can automatically run (it can but its own extensions), it does not understand the JenaLifeCycle service loader. Does calling

                           

                          JenaSystem.init() ;

                           

                          enough? if I were to write a plain JAVA application, how is this code being initialized?

                           

                          Ramesh..

                          • 10. Re: need help getting JENA JDBC to work with Teiid
                            andy.seaborne

                            JenaSystem.init() is called by various entry points into the code including NodeValue class initialization.

                             

                            So (1) it is being called (2) it is not fully initializing Jena.

                             

                            Do explicit calls would force the sequence outside of services files:

                             

                              InitJenaCore.init();

                              RIOT.init();

                              ARQ.init();

                             

                            before calling any Jena code or loading the JDBC driver.

                            • 11. Re: need help getting JENA JDBC to work with Teiid
                              andy.seaborne

                              Ah - "modular class loader"

                               

                              Forgive my ignorance about the system but is there one classloader for all the jars in the module?

                               

                              I checked - Jena's JDBC driver RemoteEndpointDriver does call jena system initialization in a class static initializer.

                              • 12. Re: need help getting JENA JDBC to work with Teiid
                                rareddy

                                Andy,

                                 

                                Does JenaSystem.init() in turn execute

                                 

                                  InitJenaCore.init();

                                  RIOT.init();

                                  ARQ.init();

                                 

                                or both separate initialization stacks and both are needed?

                                 

                                >Forgive my ignorance about the system but is there one classloader for all the jars in the module?

                                Yes, it is single class loader per module.xml, if it has <dependencies> section those are separate modules that this one depends upon

                                • 13. Re: need help getting JENA JDBC to work with Teiid
                                  omarbkhan

                                  Ramesh,

                                   

                                  I was able to get a simple Java program to query a RDF triple store using the JDBC Jena driver to get a better understanding of the process, but not from Teiid using the driver bundle.

                                   

                                  I tried to recreate the lack of initialization NodeValue in the simple Java program, but was unable to create the failure as in Teiid.

                                  The above stated, I did notice  some more details on the NodeValue error in Teiid (which did not occur in the simple Java program) :

                                  Caused by: java.lang.ClassCastException: __redirected.__DatatypeFactory cannot be cast to javax.xml.datatype.DatatypeFactory

                                    at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source)

                                    at org.apache.jena.sparql.expr.NodeValue.getDatatypeFactory(NodeValue.java:201)

                                    at org.apache.jena.sparql.expr.NodeValue.<clinit>(NodeValue.java:150)

                                    ... 52 more

                                   

                                  Andy - I wonder if it's org.apache.xerces vs javax.xml? In that the way JBoss's module loader is invoking it. could it be this

                                   

                                   

                                  The bundle jar I used was here (it may be helpful to download it to review it, I'm not sure if I can explain it's layout in an e-mail very well):

                                  Index of /repositories/snapshots/org/apache/jena/jena-jdbc-driver-bundle/3.1.2-SNAPSHOT

                                   

                                  I think/suspect after looking at the jars what could be going on is the following, perhaps (Andy please correct me) :

                                  Teiid is picking up META-INF/services/java.sql.Driver  which contents have : org.apache.jena.jdbc.remote.RemoteEndpointDriver

                                   

                                  but... is not picking up META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle

                                  Which contents has :

                                  org.apache.jena.riot.system.InitRIOT

                                  org.apache.jena.sparql.system.InitARQ

                                  org.apache.jena.system.InitJenaCore

                                   

                                  I wonder if there is something in Teiid we could leverage to call the org.apache.jena.system.JenaSubsystemLifecycle and other various initializations, maybe we need a bunch of dependencies loaded / initialized first, there are many other files in the META-INF/services folder that look like they are initializing ? If i take a look at the bundle, there are a bunch of other initializations taking place, so I'm guessing we need it all for it to work in Teiid?

                                   

                                  Here is part of my program ...I tried different ways, manually included all of the jars , tried with and without the bundle, and didn't even need to do the direct initialization, it worked fine with and with out.- but did it anyway I'm guessing because the jars did it for me ???

                                   

                                  I was going to try to query DBPedia , but I'm blocked by a proxy server right now to get there, so it's really not the best example (perhaps Andy has something better) ;  it did work on our internal triple store remote endpoint fine.So I had to remove the queries unfortunately.

                                  import java.sql.SQLException;

                                  import java.sql.Driver;

                                  import java.sql.DriverManager;

                                  import java.sql.ResultSet;

                                  import java.sql.Statement;

                                  import java.sql.Connection;

                                  import java.util.ArrayList;

                                  import java.util.List;

                                  import org.apache.jena.jdbc.JdbcCompatibility;

                                  import org.apache.jena.jdbc.connections.JenaConnection;

                                  import org.apache.jena.jdbc.remote.connections.RemoteEndpointConnection;

                                  import org.apache.jena.query.ARQ;

                                  import org.apache.jena.riot.RIOT;

                                  import org.apache.jena.system.InitJenaCore;

                                   

                                  public class HelloJenaJDBC {

                                  public static void main(String[] args) {

                                   

                                  // I don't think we need to do the below 3 initializations because the jars do it anyhow???? It worked fine with and without.

                                   

                                  InitJenaCore.init();

                                  RIOT.init();

                                  ARQ.init();

                                   

                                    Statement stmt = null;

                                     try {

                                   

                                    Connection conn = DriverManager.getConnection("jdbc:jena:remote:query=http://<servername>","user","password");

                                     // Need a statement

                                    stmt = conn.createStatement();

                                     // Make a query

                                     ResultSet rset = stmt.executeQuery(" query here ");

                                       // Iterate over results

                                     while (rset.next()) {

                                       // Print out type as a string

                                       System.out.println(rset.getString(" sparql variable here without the ? "));

                                     }

                                     // Clean up

                                     rset.close();

                                    } catch (SQLException e) {

                                     System.err.println("SQL Error - " + e.getMessage() );

                                     e.printStackTrace();

                                    } finally {

                                     try {

                                    stmt.close();

                                    } catch (SQLException e) {

                                    // TODO Auto-generated catch block

                                    e.printStackTrace();

                                    }

                                    }

                                    }

                                   

                                   

                                  }

                                  • 14. Re: need help getting JENA JDBC to work with Teiid
                                    rhn-engineering-rareddy

                                    >>I wonder if it's org.apache.xerces vs javax.xml? In that the way JBoss's module loader is invoking it.

                                     

                                    JBoss also uses xercess, and I believe the parent class loader already have this module loaded thus you may be seeing this issue, I would say this may be core of the issue.

                                     

                                    >>but... is not picking up META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle

                                    It will not, as this is JENA's own extension mechanism. But I see that this is being executed through Driver extension, but hitting above issue IMO.

                                     

                                    Can you try replacing you module.xml like below (do not use bundle jar)

                                    <module xmlns="urn:jboss:module:1.3" name="org.apache.jena">
                                        <properties>
                                            <property name="jboss.api" value="private"/>
                                        </properties>
                                         
                                        <resources>
                                            <resource-root path="collection-0.6.jar"/>
                                            <resource-root path="commons-csv-1.0.jar"/>
                                            <resource-root path="httpclient-4.2.6.jar"/>
                                          <resource-root path="httpclient-cache-4.2.6.jar"/>
                                            <resource-root path="httpcore-4.2.5.jar"/>
                                          <resource-root path="jena-arq-3.1.0.jar"/>
                                          <resource-root path="jena-base-3.1.0.jar"/>
                                          <resource-root path="jena-cmds-3.1.0.jar"/>
                                          <resource-root path="jena-core-3.1.0.jar"/>
                                          <resource-root path="jena-iri-3.1.0.jar"/>
                                          <resource-root path="jena-shaded-guava-3.1.0.jar"/>
                                          <resource-root path="jena-tdb-3.1.0.jar"/>
                                          <resource-root path="jsonld-java-0.7.0.jar"/>
                                        </resources>
                                    
                                    
                                        <dependencies>
                                            <module name="javax.api"/>
                                            <module name="org.slf4j"/>
                                            <module name="org.apache.log4j"/>
                                            <module name="org.apache.commons.cli"/> <!--1.2 needs 1.3-->
                                            <module name="org.apache.commons.codec"/> <!-- 1.10 needs 1.9 -->
                                            <module name="org.apache.commons.io"/>
                                            <module name="org.apache.commons.lang" slot="3.3.2"/>
                                            <module name="com.fasterxml.jackson.core.jackson-core"/>
                                            <module name="com.fasterxml.jackson.core.jackson-annotations"/>
                                            <module name="com.fasterxml.jackson.core.jackson-databind"/>
                                            <module name="org.apache.thrift" /> <!--9.1 needs 9.2 -->
                                            <module name="org.apache.log4j" />       
                                            <module name="org.slf4j" />
                                            <module name="org.slf4j.jcl-over-slf4j" />
                                            <module name="org.slf4j.ext"/>
                                            <module name="org.apache.xerces"/>
                                            <module name="javax.xml.bind.api"/>
                                            <module name="javax.xml.rpc.api"/>
                                            <module name="javax.xml.soap.api"/>
                                            <module name="javax.xml.stream.api"/>
                                            <module name="javax.xml.ws.api"/>
                                        </dependencies>
                                    </module>
                                    
                                    1 2 Previous Next