[WF-9] Speeding up and changing BuilderFactory

[massimiliano.masi]

Hi All,

As in a previous post, I am facing strange problems with wildfly9 and XML DOM implementations.

I actually found here: https://access.redhat.com/solutions/97793 that adding to the java opts

   JAVA_OPTS="$JAVA_OPTS -Djavax.xml.parsers.DocumentBuilderFactory=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl"

It sets an unique implementation to the FactoryFinder (without falling back into com.sun.org.apache.xerces, sometimes to saaj, etc), and it also increases the performance. But by setting this value in wildfly 9, I have:

=========================================================================

  JBoss Bootstrap Environment

  JBOSS_HOME: /opt/bin/jboss1

  JBOSS_OPTS:  '-c' 'standalone.xml' '-b' '0.0.0.0' '-bmanagement' '0.0.0.0' '-Djboss.socket.binding.port-offset=0' '-Djboss.node.name=jboss1' '-Djboss.instance.name=jboss1' '-Djboss.server.config.dir=/opt/conf/jboss1/configuration' '-Djboss.server.base.dir=/opt/bin/jboss1/standalone' '-Duser.dir=/opt/conf/jboss1' '-DspiritCfgDir=/opt/conf/jboss1/spiritCfg' '-Djboss.server.deployment.dir=/opt/conf/jboss1/deployments' '-Djboss.server.log.dir=/opt/logs/jboss1' '-Djboss.Log4j.file=/opt/conf/jboss1/configuration/log4j.xml' '-Djboss.Logs.dir=/opt/logs/jboss1' '-Djboss.STL.dir=/opt/logs/STL' '-Djava.endorsed.dirs=/opt/bin/jboss1/java.endorsed.dirs' '-Djboss.Key.file=/opt/Certificates/eb2.tiani-spirit.int.jks' '-Djboss.Key.alias=eb2.tiani-spirit.int' '-Djboss.Key.pwd=spirit' '-Djboss.Trust.file=/opt/Certificates/BeS_Truststore.jks' '-Djboss.Trust.pwd=spirit'

  JAVA: /opt/bin/java/bin/java

  JAVA_OPTS: -server -d64 -Xms1303m -Xmx1303m -Djava.net.preferIPv4Stack=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000  -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.modules.policy-permissions=true -Dhttps.protocols=TLSv1.2 -Djavax.xml.parsers.DocumentBuilderFactory=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n

=========================================================================

Listening for transport dt_socket at address: 8787

Exception in thread "main" javax.xml.parsers.FactoryConfigurationError: Provider org.apache.xerces.jaxp.DocumentBuilderFactoryImpl not found

  at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:200)

  at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:152)

  at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:232)

  at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:120)

  at __redirected.__DocumentBuilderFactory.<clinit>(__DocumentBuilderFactory.java:58)

  at __redirected.__JAXPRedirected.initAll(__JAXPRedirected.java:72)

  at org.jboss.modules.Module$1.run(Module.java:129)

  at org.jboss.modules.Module$1.run(Module.java:116)

  at java.security.AccessController.doPrivileged(Native Method)

  at org.jboss.modules.Module.<clinit>(Module.java:116)

  at org.jboss.modules.Main.main(Main.java:375)

Caused by: java.lang.ClassNotFoundException: org/apache/xerces/jaxp/DocumentBuilderFactoryImpl

  at java.lang.Class.forName0(Native Method)

  at java.lang.Class.forName(Class.java:348)

  at javax.xml.parsers.FactoryFinder.getProviderClass(FactoryFinder.java:124)

  at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:188)

  ... 10 more

Any idea on where to add the xerces API?

I have the following module in org/apache/xerces/main/module.xml

<module xmlns="urn:jboss:module:1.3" name="org.apache.xerces">

    <resources>

        <resource-root path="xercesImpl-2.11.0.SP3.jar"/>

    </resources>

    <dependencies>

        <module name="javax.api"/>

    </dependencies>

</module>

And I have another instance for debug in another module directory, of my application.

Any hints?

Thanks

[ctomc]

xerces is already used as default. not sure what are you trying to achive

[massimiliano.masi]

Hi,

I have exactly that trouble. I would like to have xerces as default, but it is not. See, e.g., what I asked in Endorsing XML implementations, and in 'Re: XML Implementation issues' - MARC . It seems that saaj is using one implementation, opensaml another one, and wildfly another one. They all use the FindFactory, of course.

I just would like to be sure that they all use the same XML implementation, and this switch is the only thing that I have been able to find.

[massimiliano.masi]

Hi,

I added the property into jre/lib/jaxp.properties, as you can see from the DEBUG.

Listening for transport dt_socket at address: 8787

JAXP: find factoryId =javax.xml.parsers.DocumentBuilderFactory

JAXP: Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

JAXP: found in $java.home/jaxp.properties, value=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl

Exception in thread "main" javax.xml.parsers.FactoryConfigurationError: Provider org.apache.xerces.jaxp.DocumentBuilderFactoryImpl not found

  at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:200)

  at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:152)

  at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:259)

  at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:120)

  at __redirected.__DocumentBuilderFactory.<clinit>(__DocumentBuilderFactory.java:58)

  at __redirected.__JAXPRedirected.initAll(__JAXPRedirected.java:72)

  at org.jboss.modules.Module$1.run(Module.java:129)

  at org.jboss.modules.Module$1.run(Module.java:116)

  at java.security.AccessController.doPrivileged(Native Method)

  at org.jboss.modules.Module.<clinit>(Module.java:116)

  at org.jboss.modules.Main.main(Main.java:375)

Caused by: java.lang.ClassNotFoundException: org/apache/xerces/jaxp/DocumentBuilderFactoryImpl

  at java.lang.Class.forName0(Native Method)

  at java.lang.Class.forName(Class.java:348)

  at javax.xml.parsers.FactoryFinder.getProviderClass(FactoryFinder.java:124)

  at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:188)

  ... 10 more

22624

I saw __DocumentBuilderFactory (jboss-modules/__DocumentBuilderFactory.java at master · jboss-modules/jboss-modules · GitHub) in the static block it has a very nice comment:

// Unfortunately we can not use null because of a stupid bug in the jdk JAXP factory finder.
	// Lack of tccl causes the provider file discovery to fallback to the jaxp loader (bootclasspath)
	// which is correct. However, after parsing it, it then disables the fallback for the loading of the class.
	// Thus, the class can not be found.
	//
	// Work around the problem by using the System CL, although in the future we may want to just "inherit"
	// the environment's TCCL

and then it is setting the thread classloader as the system classloader. Thus, I added the xercesImpl, xml-apis, and xalan to the $JAVA_HOME/lib/endorsed and I added -Djava.endorsed.lib=$JAVA_HOME/lib/endorsed. The JAXP now seems to run nicely,

Listening for transport dt_socket at address: 8787

JAXP: find factoryId =javax.xml.parsers.DocumentBuilderFactory

JAXP: Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

JAXP: found in $java.home/jaxp.properties, value=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl

JAXP: created new instance of class org.apache.xerces.jaxp.DocumentBuilderFactoryImpl using ClassLoader: sun.misc.Launcher$AppClassLoader@14dad5dc

JAXP: find factoryId =javax.xml.parsers.SAXParserFactory

JAXP: found jar resource=META-INF/services/javax.xml.parsers.SAXParserFactory using ClassLoader: sun.misc.Launcher$AppClassLoader@14dad5dc

JAXP: found in resource, value=org.apache.xerces.jaxp.SAXParserFactoryImpl

JAXP: created new instance of class org.apache.xerces.jaxp.SAXParserFactoryImpl using ClassLoader: sun.misc.Launcher$AppClassLoader@14dad5dc

JAXP: find factoryId =javax.xml.transform.TransformerFactory

JAXP: Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

JAXP: found jar resource=META-INF/services/javax.xml.transform.TransformerFactory using ClassLoader: sun.misc.Launcher$AppClassLoader@14dad5dc

JAXP: found in resource, value=org.apache.xalan.processor.TransformerFactoryImpl

JAXP: created new instance of class org.apache.xalan.processor.TransformerFactoryImpl using ClassLoader: sun.misc.Launcher$AppClassLoader@14dad5dc

JAXP: using thread context class loader (sun.misc.Launcher$AppClassLoader@14dad5dc) for search

JAXP: Looking up system property 'javax.xml.xpath.XPathFactory:http://java.sun.com/jaxp/xpath/dom'

JAXP: The property is undefined.

JAXP: Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

JAXP: found null in $java.home/jaxp.properties

JAXP: looking into jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xalan-2.7.1.jar!/META-INF/services/javax.xml.xpath.XPathFactory

JAXP: Reading jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xalan-2.7.1.jar!/META-INF/services/javax.xml.xpath.XPathFactory

JAXP: instanciating org.apache.xpath.jaxp.XPathFactoryImpl

JAXP: loaded it from jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xalan-2.7.1.jar!/org/apache/xpath/jaxp/XPathFactoryImpl.class

JAXP: factory 'org.apache.xpath.jaxp.XPathFactoryImpl' was found for http://java.sun.com/jaxp/xpath/dom

JAXP: find factoryId =javax.xml.stream.XMLEventFactory

JAXP: Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

JAXP: loaded from fallback value: com.sun.xml.internal.stream.events.XMLEventFactoryImpl

JAXP: created new instance of class com.sun.xml.internal.stream.events.XMLEventFactoryImpl using ClassLoader: null

JAXP: find factoryId =javax.xml.stream.XMLInputFactory

JAXP: loaded from fallback value: com.sun.xml.internal.stream.XMLInputFactoryImpl

JAXP: created new instance of class com.sun.xml.internal.stream.XMLInputFactoryImpl using ClassLoader: null

JAXP: find factoryId =javax.xml.stream.XMLOutputFactory

JAXP: loaded from fallback value: com.sun.xml.internal.stream.XMLOutputFactoryImpl

JAXP: created new instance of class com.sun.xml.internal.stream.XMLOutputFactoryImpl using ClassLoader: null

javax.xml.datatype.FactoryFinder:Using context class loader: sun.misc.Launcher$AppClassLoader@14dad5dc

javax.xml.datatype.FactoryFinder:Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

javax.xml.datatype.FactoryFinder:found null in $java.home/jaxp.properties

javax.xml.datatype.FactoryFinder:found jar resource=META-INF/services/javax.xml.datatype.DatatypeFactory using ClassLoader: sun.misc.Launcher$AppClassLoader@14dad5dc

javax.xml.datatype.FactoryFinder:found in resource, value=org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl

javax.xml.datatype.FactoryFinder:Loaded org.apache.xerces.jaxp.datatype.DatatypeFactoryImpl from jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xercesImpl-2.9.1.jar!/org/apache/xerces/jaxp/datatype/DatatypeFactoryImpl.class

JAXP: using thread context class loader (sun.misc.Launcher$AppClassLoader@14dad5dc) for search

JAXP: Looking up system property 'javax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema'

JAXP: The property is undefined.

JAXP: Read properties file /opt/bin/jdk1.8.0_60/jre/lib/jaxp.properties

JAXP: found null in $java.home/jaxp.properties

JAXP: looking into jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xercesImpl-2.9.1.jar!/META-INF/services/javax.xml.validation.SchemaFactory

JAXP: Reading jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xercesImpl-2.9.1.jar!/META-INF/services/javax.xml.validation.SchemaFactory

JAXP: instanciating org.apache.xerces.jaxp.validation.XMLSchemaFactory

JAXP: loaded it from jar:file:/opt/bin/jdk1.8.0_60/lib/endorsed/xercesImpl-2.9.1.jar!/org/apache/xerces/jaxp/validation/XMLSchemaFactory.class

JAXP: factory 'org.apache.xerces.jaxp.validation.XMLSchemaFactory' was found for http://www.w3.org/2001/XMLSchema

but saaj is now forcing somehow to use com.sun.org.apache. I am still checking why. 

[massimiliano.masi]

FYI: I solved it, after all the investigations above.

The idea is the following. Setting is wildfly 9, opensaml, 2.6, 3.x, java8.

When an Element is unmarshalled by the Xmltooling in the context above, it gets its DOM associated using the SAAJ DOM Implementation (which is used).

Here, there is a funny thing: they overcome for whatever reason the FactoryFinder and they always instantiate the com.sun.org.apache.xerces,

using a plain new().

Results: when marshalling, the opensaml’s ParserPool gets the implementation from wildfly (the BasicParserPool uses the FactoryFinder)

but then, in the XMLHelper, in the adoptNode, the underlying wildfly's xerces implementation returns null, and thus XMLHelper throws the “DOM Element Node Adoption Failed”.

Note that setting the builderfactory either in $JAVA_HOME/jre/lib/jaxp.properties, or in the -D in the wildfly JAVA_OPTS, does not work as expected,

as per the TCCL issue above.

The workaround that I found is the following. The the Configuration.getParserPool().getBuilder().getDOMImplementation(), then get the DOM

implementation from the xmlObject.getDOM(), and check if they are the same. If not, take the DOM from the XML Object, create a new

DocumentBuilder out of the Configuration.getParserPool().getBuilder(), dump the xmlObject’s DOM to a byte array, parse it, and call the

xmlObject.setDOM(newDOM). Then marshall it.