need help getting JENA JDBC to work with Teiid

[omarbkhan]

Hi - I've been trying to get the JENA JDBC driver Apache Jena - Jena JDBC - A SPARQL over JDBC driver framework  to work with Teiid.

I'm able to perform a query in the SQL Scrapbook and see results return in the results window while in the database context, but modal dialog saying the query is still executing isn't dismissed? Any ideas how to get this to work with Teiid?

Having a RDF triple-store remote endpoint datasource would really help us in our integration efforts, is there any info I could provide that would help clarify what I'm trying to do if the above isn't clear?

[rareddy]

Omar,

Currently Teiid does not have translator for the JENA. If this were any JDBC driver that supports SQL, there are some ansi compatible built-in translators that you could use, however this is SPARQL based query language it mandates the requirement to develop a custom translator where relational constructs to the RDF model need to be derived. When we looked at this feasibility of RDF to relational model, we were not completely satisfied with "a" solution. The issue is captured here [TEIID-3126] Add OWL / RDF translators to TeiiD - JBoss Issue Tracker

If you want to help contribute such translator, please let us know, as we could use a subject matter expert in RDF.

Ramesh..

[omarbkhan]

Ramesh,

I do have interest in your proposal.   I'm very passionate about semantics/RDF/OWL

I hope to have some time next year to lightly be engaged in this, my bandwith is very limited. This may give me also a bit more time to understand the translator programming model & docs. No doubt, I probably will need some hand holding - but hopefully that will be a minimum as I get more adept with Teiid and actutally the Jena programming model to do the integration.

Thank you for the offer, with a bit of good fortune I'll be able to help.

Omar

[omarbkhan]

Ramesh,

I got a bit further with Jena, and partial success.

I encountered an issue and reached out to the JDBC maintainers, and got the below with some helpful information - but don't understand what steps I should take next in Teiid ?

info from author in thread:

http://jena.markmail.org/thread/pxwjulkqiprttdhi

jira created to address issue:

[JENA-1255] jena-jdbc-bundle does not consolidate ServiceLoader files. - ASF JIRA

I believe I have all of the jars in the classpath and module.xml, so this is quite curious - the author believes it has to do with ServiceLoader if I'm interpreting it correctly and we should still be able to get it to work as is?

Any suggestions? I'm not sure what the author means by repackaging?

[rareddy]

It is not standard extension of the JDBC Driver load, so it is custom service loader specific to JENA. Ask him who is responsible for loading it, and how does one load it? They should have bootstrapped using the standard JDBC4 Driver service loader mechanism to initialize it, then we would not have issues.

I was going to ask on email thread but having some MarkMail issues.

[omarbkhan]

Hi Ramesh,

Andy responded to the questions I asked, can you review?

http://markmail.org/message/jivwjcv7uji7imro

I'm not sure how we would call ARQ.init() from module.xml?

[rareddy]

Omar,

Now I can not even get to MarkMail to read the response. Basically what they need to do, replace their ServiceLoader mechanism with standard JDBC4 ServiceLoader mechanism like

teiid/client/src/main/resources/META-INF/services at master · rareddy/teiid · GitHub

That will register their Driver class, and in their driver class possibly in a "static block" call ARQ.init() that will initialize every time one uses their JDBC driver. The issue is not with the JBoss modules, it is how JENA driver designed.

Ramesh..

[andy.seaborne]

Hi Ramesh,

Apache Jena JDBC does use java.sql.Driver. Omars stacktrace shows that Jena JDBC code has been loaded and executed.

at org.apache.jena.jdbc.statements.JenaStatement.execute(JenaStatement.java:273)   <----- Jena JDBC code

  at org.jboss.jca.adapters.jdbc.WrappedStatement.execute(WrappedStatement.java:182)

  at org.teiid.translator.jdbc.JDBCDirectQueryExecution.execute(JDBCDirectQueryExecution.java:83)

  ... 17 more

So the issue is not about finding the driver as far as I can see.

Jena uses ServiceLoader itself and the report is indicative of that part not being configured properly for org.apache.jena.system.JenaSubsystemLifecycle.

Jena has several service files for the same interface - if combing jena jars, then that process needs to combine service files for org.apache.jena.system.JenaSubsystemLifecycle (for example, the maven shade plugin ServicesResourceTransformer does this)

There was an issue with the bundled jena-jdbc-driver-bundle (now fixed) but Omar reports an issue when using the driver in unbundled form via

<?xml version="1.0" encoding="UTF-8"?>

<module xmlns="urn:jboss:module:1.0" name="org.apache.jena">

  <resources>

  <resource-root path="jena-jdbc-core-3.1.0.jar"/>

  <resource-root path="jena-jdbc-driver-remote-3.1.0.jar"/>

  <resource-root path="jena-arq-3.1.0.jar"/>

  <resource-root path="jena-core-3.1.0.jar"/>

  <resource-root path="jena-base-3.1.0.jar"/>

  <resource-root path="jena-cmds-3.1.0.jar"/>

...

and the stacktrace suggests incomplete Jena initialization.

    Andy

[omarbkhan]

(Ramesh - permalink for more details (not MarkMail))

Pony Mail!

https://lists.apache.org/thread.html/a227a41fb68d4ba6d2564bbf17acdab035853ddf4ff62bad7048ca42@%3Cusers.jena.apache.org%3E

[rareddy]

I read through the messages on the link now.

The JBoss module system (module.xml) does NOT bundle the jars. It is another modular class loader. There is no code it can automatically run (it can but its own extensions), it does not understand the JenaLifeCycle service loader. Does calling

JenaSystem.init() ;

enough? if I were to write a plain JAVA application, how is this code being initialized?

Ramesh..

[andy.seaborne]

JenaSystem.init() is called by various entry points into the code including NodeValue class initialization.

So (1) it is being called (2) it is not fully initializing Jena.

Do explicit calls would force the sequence outside of services files:

  InitJenaCore.init();

  RIOT.init();

  ARQ.init();

before calling any Jena code or loading the JDBC driver.

[andy.seaborne]

Ah - "modular class loader"

Forgive my ignorance about the system but is there one classloader for all the jars in the module?

I checked - Jena's JDBC driver RemoteEndpointDriver does call jena system initialization in a class static initializer.

[rareddy]

Andy,

Does JenaSystem.init() in turn execute

  InitJenaCore.init();

  RIOT.init();

  ARQ.init();

or both separate initialization stacks and both are needed?

>Forgive my ignorance about the system but is there one classloader for all the jars in the module?

Yes, it is single class loader per module.xml, if it has <dependencies> section those are separate modules that this one depends upon

[omarbkhan]

Ramesh,

I was able to get a simple Java program to query a RDF triple store using the JDBC Jena driver to get a better understanding of the process, but not from Teiid using the driver bundle.

I tried to recreate the lack of initialization NodeValue in the simple Java program, but was unable to create the failure as in Teiid.

The above stated, I did notice  some more details on the NodeValue error in Teiid (which did not occur in the simple Java program) :

Caused by: java.lang.ClassCastException: __redirected.__DatatypeFactory cannot be cast to javax.xml.datatype.DatatypeFactory

  at javax.xml.datatype.DatatypeFactory.newInstance(Unknown Source)

  at org.apache.jena.sparql.expr.NodeValue.getDatatypeFactory(NodeValue.java:201)

  at org.apache.jena.sparql.expr.NodeValue.<clinit>(NodeValue.java:150)

  ... 52 more

Andy - I wonder if it's org.apache.xerces vs javax.xml? In that the way JBoss's module loader is invoking it. could it be this

The bundle jar I used was here (it may be helpful to download it to review it, I'm not sure if I can explain it's layout in an e-mail very well):

Index of /repositories/snapshots/org/apache/jena/jena-jdbc-driver-bundle/3.1.2-SNAPSHOT

I think/suspect after looking at the jars what could be going on is the following, perhaps (Andy please correct me) :

Teiid is picking up META-INF/services/java.sql.Driver  which contents have : org.apache.jena.jdbc.remote.RemoteEndpointDriver

but... is not picking up META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle

Which contents has :

org.apache.jena.riot.system.InitRIOT

org.apache.jena.sparql.system.InitARQ

org.apache.jena.system.InitJenaCore

I wonder if there is something in Teiid we could leverage to call the org.apache.jena.system.JenaSubsystemLifecycle and other various initializations, maybe we need a bunch of dependencies loaded / initialized first, there are many other files in the META-INF/services folder that look like they are initializing ? If i take a look at the bundle, there are a bunch of other initializations taking place, so I'm guessing we need it all for it to work in Teiid?

Here is part of my program ...I tried different ways, manually included all of the jars , tried with and without the bundle, and didn't even need to do the direct initialization, it worked fine with and with out.- but did it anyway I'm guessing because the jars did it for me ???

I was going to try to query DBPedia , but I'm blocked by a proxy server right now to get there, so it's really not the best example (perhaps Andy has something better) ;  it did work on our internal triple store remote endpoint fine.So I had to remove the queries unfortunately.

import java.sql.SQLException;

import java.sql.Driver;

import java.sql.DriverManager;

import java.sql.ResultSet;

import java.sql.Statement;

import java.sql.Connection;

import java.util.ArrayList;

import java.util.List;

import org.apache.jena.jdbc.JdbcCompatibility;

import org.apache.jena.jdbc.connections.JenaConnection;

import org.apache.jena.jdbc.remote.connections.RemoteEndpointConnection;

import org.apache.jena.query.ARQ;

import org.apache.jena.riot.RIOT;

import org.apache.jena.system.InitJenaCore;

public class HelloJenaJDBC {

public static void main(String[] args) {

// I don't think we need to do the below 3 initializations because the jars do it anyhow???? It worked fine with and without.

InitJenaCore.init();

RIOT.init();

ARQ.init();

  Statement stmt = null;

   try {

  Connection conn = DriverManager.getConnection("jdbc:jena:remote:query=http://<servername>","user","password");

   // Need a statement

  stmt = conn.createStatement();

   // Make a query

   ResultSet rset = stmt.executeQuery(" query here ");

     // Iterate over results

   while (rset.next()) {

     // Print out type as a string

     System.out.println(rset.getString(" sparql variable here without the ? "));

   }

   // Clean up

   rset.close();

  } catch (SQLException e) {

   System.err.println("SQL Error - " + e.getMessage() );

   e.printStackTrace();

  } finally {

   try {

  stmt.close();

  } catch (SQLException e) {

  // TODO Auto-generated catch block

  e.printStackTrace();

  }

  }

  }

}

[rhn-engineering-rareddy]

>>I wonder if it's org.apache.xerces vs javax.xml? In that the way JBoss's module loader is invoking it.

JBoss also uses xercess, and I believe the parent class loader already have this module loaded thus you may be seeing this issue, I would say this may be core of the issue.

>>but... is not picking up META-INF/services/org.apache.jena.system.JenaSubsystemLifecycle

It will not, as this is JENA's own extension mechanism. But I see that this is being executed through Driver extension, but hitting above issue IMO.

Can you try replacing you module.xml like below (do not use bundle jar)

<module xmlns="urn:jboss:module:1.3" name="org.apache.jena">
    <properties>
        <property name="jboss.api" value="private"/>
    </properties>
     
    <resources>
        <resource-root path="collection-0.6.jar"/>
        <resource-root path="commons-csv-1.0.jar"/>
        <resource-root path="httpclient-4.2.6.jar"/>
      <resource-root path="httpclient-cache-4.2.6.jar"/>
        <resource-root path="httpcore-4.2.5.jar"/>
      <resource-root path="jena-arq-3.1.0.jar"/>
      <resource-root path="jena-base-3.1.0.jar"/>
      <resource-root path="jena-cmds-3.1.0.jar"/>
      <resource-root path="jena-core-3.1.0.jar"/>
      <resource-root path="jena-iri-3.1.0.jar"/>
      <resource-root path="jena-shaded-guava-3.1.0.jar"/>
      <resource-root path="jena-tdb-3.1.0.jar"/>
      <resource-root path="jsonld-java-0.7.0.jar"/>
    </resources>


    <dependencies>
        <module name="javax.api"/>
        <module name="org.slf4j"/>
        <module name="org.apache.log4j"/>
        <module name="org.apache.commons.cli"/> <!--1.2 needs 1.3-->
        <module name="org.apache.commons.codec"/> <!-- 1.10 needs 1.9 -->
        <module name="org.apache.commons.io"/>
        <module name="org.apache.commons.lang" slot="3.3.2"/>
        <module name="com.fasterxml.jackson.core.jackson-core"/>
        <module name="com.fasterxml.jackson.core.jackson-annotations"/>
        <module name="com.fasterxml.jackson.core.jackson-databind"/>
        <module name="org.apache.thrift" /> <!--9.1 needs 9.2 -->
        <module name="org.apache.log4j" />       
        <module name="org.slf4j" />
        <module name="org.slf4j.jcl-over-slf4j" />
        <module name="org.slf4j.ext"/>
        <module name="org.apache.xerces"/>
        <module name="javax.xml.bind.api"/>
        <module name="javax.xml.rpc.api"/>
        <module name="javax.xml.soap.api"/>
        <module name="javax.xml.stream.api"/>
        <module name="javax.xml.ws.api"/>
    </dependencies>
</module>