Custom translator that depends on Xalan

[mdfspiff]

Teiid 8.9.0

I'm developing a custom Teiid translator that depends on geo-tools.  A particular module in geo-tools requires xalan for XSLT processing and unfortunately, does not work with saxon. 

I've created a JBoss module for geo-tools described by module.xml below.  My translator depends on this module (org.geotools).  The problem is that saxon always seems to be loaded first regardless of how I configure the geo-tools module or the Jboss deployment structure file for the translator.  It has something to do with the fact that the main teiid module includes Saxon-HE-9.5.1-6.jar.

module.xml

<?xml version="1.0" encoding="UTF-8"?>

<module xmlns="urn:jboss:module:1.1" name="org.geotools">

  <resources>

      <resource-root path="gt-wfs-ng-13-SNAPSHOT.jar"/>

      <resource-root path="gt-api-13-SNAPSHOT.jar"/>

...

     <resource-root path="gt-xsd-wfs-13-SNAPSHOT.jar"/>

   </resources>

   <dependencies>

        <module name="org.jboss.teiid">

            <imports>

                <exclude-set>

                    <path name="net.sf.saxon"/>

                </exclude-set>

            </imports>

        </module>

        <module name="org.apache.xalan"/>

        <module name="org.apache.xerces"/>

        <module name="javax.api"/>

    </dependencies>

</module>

jboss-deployment-structure.xml

<?xml version="1.0" encoding="UTF-8"?>

<jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.1">

    <deployment>

        <dependencies>

            <module name="org.jboss.teiid">

            <imports>

                <exclude-set>

                    <path name="net.sf.saxon"/>

                </exclude-set>

            </imports>

            </module>           

            <module name="javax.api"/>

            <module name="org.apache.xerces" services="import"/>

            <module name="org.apache.xalan" services="import"/>

            <module name="org.geotools"/>

        </dependencies>

    </deployment>

</jboss-deployment-structure>

Regardless of how these files are configured, the JBoss classloader loads saxon instead of xalan.  Any idea what I'm doing wrong?

[shawkins]

Why is there a dependency to all of teiid? Normally you'd just want org.jboss.teiid.api.

Otherwise I'm not familiar with module exclusion, perhaps the AS folks could be of some assistance there if there is some issue with things loaded as services.

[mdfspiff]

Thanks for the reply Steven.  I've moved this question to the Jboss forum.  Jboss application that depends on Xalan

Would it be possible to split the Saxon jar into it's own module?  That may make it easier to deal with.  Also, as you recommended above, I tried to specify teiid-api only, but that didn't work.

[shawkins]

What didn't work about just including the teiid-api dependency?

> Would it be possible to split the Saxon jar into it's own module?

That may be possible, but I'd want to understand first what the translator is using from the engine.

[mdfspiff]

Steve,

Thanks for your reply.  When I change the dependencies from teiid main (org.jboss.teiid) to teiid api (org.jboss.teiid.api) and teiid common-core(org.jboss.teiid.common-core), I still have the Saxon issue.  The translator seems to load and run as expected, but it fails to process the metadata because of Saxon.

I thought the module system would allow me to specify a dependency on xalan and xalan would be loaded as the jaxp implementation for my application.  Should I be able to do this or do I have a fundamental misunderstanding of the Jboss class loader and module system?

MIke

[shawkins]

Is the scenario something like a service loader is using the thread context classloader from the engine thread in the translator code and thus using what's visible to the engine?

[rareddy]

JBoss module system is much like OSGI, driven by the module.xml file. You can define resources and dependent modules. Each module classloader is isolated from others, and load dependencies automatically.

When you say "fails to process metadata because of saxon" what do you see? The stacktrace should show you path it is trying to use Teiid module.

[mdfspiff]

Steve,

I'm certainly not a JBOSS classloader or TCCL expert, so I'm not exactly sure what's going on.

Ramesh,

I'm receiving a Saxon exception.  Essentially, GeoTools is making a call that is out of spec.  Xalan is a little more flexible than Saxon.  I've looked fixing the Saxon issues in GeoTools, and it's quite extensive.

So, as an experiment, I separated Saxon into its own JBoss module and it seems to work with my translator.  I created a new module (net.sf.saxon).  I added the Saxon jar file to the main folder and the following module.xml file:

<?xml version="1.0" encoding="UTF-8"?>

<module xmlns="urn:jboss:module:1.0" name="net.sf.saxon">

    <resources>

        <resource-root path="Saxon-HE-9.5.1-6.jar" />

    </resources>

    <dependencies>

        <module name="javax.api" />

    </dependencies>

</module>

Then I modified the org.jboss.teiid module.  I removed the Saxon jar file as a resource and added a dependency on the net.sf.saxon module. 

<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.0" name="org.jboss.teiid">
    <resources>

        ...
        <!-- <resource-root path="Saxon-HE-9.5.1-6.jar" /> -->

        ...

    </resources>

    <dependencies>

        ...
        <module name="net.sf.saxon"/>      

        ...
    </dependencies>
</module>

This allows me to exclude the net.sf.saxon module in my translator's jboss deployment structure.

<?xml version="1.0" encoding="UTF-8"?>

<jboss-deployment-structure xmlns="urn:jboss:deployment-structure:1.1">

    <deployment>

        <exclusions>

            <module name="net.sf.saxon"/>

        </exclusions>

        <dependencies>

            <module name="org.geotools"/>           

            <module name="org.jboss.teiid.api"/>

            <module name="org.jboss.teiid.common-core"/>

        </dependencies>

    </deployment>

</jboss-deployment-structure>

As a result, I no longer have the Saxon errors while parsing the metadata.

Would it be possible to move Saxon into its own module?

[rareddy]

But the question still is why are you depending upon the "teiid" module for your translator? For developing a translator you never need to "teiid", and you only need "api", "common-core" and may be "client" that it. That is the core question Steve was driving towards. If you remove dependency on "teiid", then Saxon is mute issue. We will handle saxon issue as separate.

It could be you are using it for testing, but you do not need the dependency for runtime.

Ramesh..

[mdfspiff]

Ramesh,

Thank you so much for your help.  I really don't know why I continue to see the Saxon issue when I remove the dependency on teiid main.  As you can see from my new jboss deployment structure, my translator does not hava a dependency on teiid main.  Only on teiid api and common core.  Yet the Saxon issues remain until Saxon is separated into its own module. I wish I can give you an answer, but I don't understand the internals of Jboss class loading.

I just looked at all of the modules.  The only modules that have a dependency on org.jboss.teiid are the following:

org.jboss.teiid.translator.infinispan.dsl

org.jboss.teiid.translator.object

org.jboss.teiid.translator.ws

I didn't see any dependencies on these modules.

However, I did find org.jboss.teiid listed as an extension in standalone.xml

<extension module="org.jboss.teiid"/>

Any idea how this would affect class loading of the applications?

Mike

[rareddy]

<extension> has nothing to do with the class loading. We do have a fair knowledge of how class loading works in JBoss EAP, thus the questions. As Steve mentioned couple comments ago, if you were using ThreadContextClassLoader somewhere in your translator then above *could* happen, as TCCL should not be used. I would like to see the stacktrace when the Saxon issue occurs. If you can attach that stack, we can take look and see, who is trying to load the Saxon libraries.

Ramesh..

[mdfspiff]

Thanks Ramesh!

java.lang.IllegalArgumentException: Invalid value for serialization method: must be xml, html, xhtml, text, or a QName in '{uri}local' form

at net.sf.saxon.Controller.setOutputProperty(Controller.java:412)

at org.geotools.xml.Encoder.encode(Encoder.java:594)

at org.geotools.data.wfs.internal.AbstractWFSStrategy.getPostContents(AbstractWFSStrategy.java:703)

at org.geotools.data.wfs.internal.WFSRequest.performPostOutput(WFSRequest.java:179)

at org.geotools.data.ows.AbstractOpenWebService.internalIssueRequest(AbstractOpenWebService.java:415)

at org.geotools.data.wfs.internal.WFSClient.internalIssueRequest(WFSClient.java:280)

at org.geotools.data.wfs.internal.WFSClient.issueRequest(WFSClient.java:323)

at org.geotools.data.wfs.WFSDataStore.getRemoteFeatureType(WFSDataStore.java:147)

at org.geotools.data.wfs.WFSDataStore.getRemoteSimpleFeatureType(WFSDataStore.java:160)

at org.geotools.data.wfs.WFSFeatureSource.buildFeatureType(WFSFeatureSource.java:341)

at org.geotools.data.wfs.WFSFeatureStore.buildFeatureType(WFSFeatureStore.java:108)

at org.geotools.data.store.ContentFeatureSource.getAbsoluteSchema(ContentFeatureSource.java:340)

at org.geotools.data.store.ContentFeatureSource.getSchema(ContentFeatureSource.java:309)

at org.geotools.data.store.ContentFeatureSource.getSchema(ContentFeatureSource.java:109)

at hydra.teiid.translator.wfs.WFSMetadataProcessor.process(WFSMetadataProcessor.java:49)

at hydra.teiid.translator.wfs.WFSMetadataProcessor.process(WFSMetadataProcessor.java:27)

at org.teiid.translator.ExecutionFactory.getMetadata(ExecutionFactory.java:915)

at org.teiid.query.metadata.NativeMetadataRepository.loadMetadata(NativeMetadataRepository.java:73)

at org.teiid.query.metadata.ChainingMetadataRepository.loadMetadata(ChainingMetadataRepository.java:55)

at org.teiid.jboss.VDBService$6.run(VDBService.java:395)

at org.teiid.jboss.VDBService$7.run(VDBService.java:442)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)

at org.jboss.threads.JBossThread.run(JBossThread.java:122)

[rareddy]

OK, I believe the geoTools is using the TCCL to load the schema, thus the error. However, the error is on the Teiid side not correctly removing/setting the correct TCCL before we call "getMetadata" call on the translator. You can add JIRA for this we will fix it.

Ramesh..

[mdfspiff]

Wow, thanks for sticking with me on this one!  I did confirm that Geotools is using TCCL to load the schema.