Problem compiling an XSLT stylesheet file

[shorero2011]

SUMMARY OF PROBLEM: I have an .xsl stylesheet file that compiles fine when I run a standalone program outside the JBoss environment. The file also works fine when I run under JBoss 5 or 6. When my JMX MBean inside JBoss 7 tries to compile the same file, it gets an error that ends with the following exception:

Caused by: javax.xml.transform.TransformerConfigurationException: Could not compile stylesheet

      at org.apache.xalan.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:832)

      at __redirected.__TransformerFactory.newTemplates(__TransformerFactory.java:137)

      at com.cst.chetx.core.bean.transform.TransformDataSupport.loadFromUrl(TransformDataSupport.java:215)

      at com.cst.chetx.core.bean.transform.TransformDataSupport.loadTemplateSequence(TransformDataSupport.java:269)

... 14 more

Experience with previous versions of JBoss suggests that this error is caused by an undocumented limit on the number of template elements in the single .xsl file; if so, the limit has been dropped down in JBoss 7 from maybe 1000 or so to under 500, which is beginning to be a real problem for me. I can provide more details on exactly how the MBean is getting loaded etc if that would help with answers.

QUESTIONS:

1. Can anybody confirm a limit on the number of templates in an .xsl file? If so, is there any way I can modify or control it from inside the EAR file to allow this stylesheet to compile?

2. Is there any way for a single .ear to use a TransformerFactory other than the one returned by TransformerFactory.getInstance() when I'm running under JBoss 7? I tried using the TransformerFactory.getInstance(className, null) method and plugged in the name of the transformer factory class that I get outside JBoss (com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl), but I get a NoClassDefFound exception.

3. Any other suggestions re how to work around this problem?

ENVIROMENT: (from boot.log)

17:54:23,715 INFO  [org.jboss.modules] JBoss Modules version 1.0.2.GA

17:54:25,410 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA

17:54:25,548 INFO  [org.jboss.as] JBoss AS 7.0.2.Final "Arc" starting

17:54:25,550 DEBUG [org.jboss.as.config] Configured system properties:

...

        java.runtime.name = Java(TM) SE Runtime Environment

        java.runtime.version = 1.7.0-b147

...

        java.vm.name = Java HotSpot(TM) 64-Bit Server VM

        java.vm.specification.name = Java Virtual Machine Specification

        java.vm.specification.vendor = Oracle Corporation

        java.vm.specification.version = 1.7

        java.vm.vendor = Oracle Corporation

        java.vm.version = 21.0-b17

...

uname -a

Linux splitter 2.6.38-11-server #50-Ubuntu SMP Mon Sep 12 21:34:27 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

[jason.greene]

We are using a slightly patched version of xalan as the default provider, with default settings. You can also bundle your own Transformer impl in your application. One way to do this is to include it in ear/lib. Another way to do this is to create a module and add an import on the impl (be sure to use the services option when importing the dep).

[shorero2011]

Thanks. Let me poke around and see if there's some way I can get my ear to "see" the TransformerFactory et al bundled with the JDK/JRE.

[shorero2011]

OK. I added a Maven dependency on Xalan 2.7.1 to my EAR builder, put the jar into the lib directory, and configured the code to use the org.apache.xalan.processor.TransformerFactoryImpl class. The stylesheet now compiles. I get a half-dozen or so complaints from JBoss about missing jar files from Class-Path entries in the xalan jar's manifest, but at least the stylesheet now compiles. I'm hoping that the bean will work properly even in the face of the missing classpath items, but I'll need to finish my J2EE6/Java7 conversion before I can be completele sure.

I am a bit uneasy about this situation, though. I've basically got to grab a private version of xalan (TransformerFactory #3) to work around a problem introduced by the JBoss version (TransformerFactory #2) that gets used in place of the working JDK/JRE version (TransformerFactory #1). I'm sure you had your reasons for using a TransformerFactory other than that included with the JDK/JRE (probably to ensure a consistent runtime environment under pre-Java7 versions of the JRE), but it sure seems like there should be an easier way to simply default back to the included JDK/JRE version rather than having to drag my own TransformerFactory around.

You also might want to check in your undoubtedly copious spare time ;-) to see what your "slightly patched" version did to make large stylesheets uncompilable.

[jason.greene]

shorero2011 wrote:

OK. I added a Maven dependency on Xalan 2.7.1 to my EAR builder, put the jar into the lib directory, and configured the code to use the org.apache.xalan.processor.TransformerFactoryImpl class. The stylesheet now compiles. I get a half-dozen or so complaints from JBoss about missing jar files from Class-Path entries in the xalan jar's manifest, but at least the stylesheet now compiles. I'm hoping that the bean will work properly even in the face of the missing classpath items, but I'll need to finish my J2EE6/Java7 conversion before I can be completele sure.

Those are just warnings and can be ignored, with the exception of serializer.jar, i believe that one is needed.

I am a bit uneasy about this situation, though. I've basically got to grab a private version of xalan (TransformerFactory #3) to work around a problem introduced by the JBoss version (TransformerFactory #2) that gets used in place of the working JDK/JRE version (TransformerFactory #1). I'm sure you had your reasons for using a TransformerFactory other than that included with the JDK/JRE (probably to ensure a consistent runtime environment under pre-Java7 versions of the JRE), but it sure seems like there should be an easier way to simply default back to the included JDK/JRE version rather than having to drag my own TransformerFactory around.

We have our own for two reasons:

1. The JDK version is full of bugs and as you mention inconsistent between VM revs
2. Unpatched Xalan is incompatible with a modular classloading environment. Unpatched will however work for EE deployments since TCCL happens to be the deployment CL. A patch for this was submitted to the Xalan project upstream, but the project is not actively maintained.

If you want to use the jdk version, you can add an import on the com.sun classes, and add a META-INF/services/javax.xml.transform.TransformerFactory that points to the jdk class.

You also might want to check in your undoubtedly copious spare time ;-) to see what your "slightly patched" version did to make large stylesheets uncompilable.

You can see the delta here. It's a set of very specific changes on top of the same version you are using:

https://github.com/jboss/xalan-j/compare/jboss_2_7_1

One difference that is in common with jdk6 and our version is that we use the bytecode generation version of xsltc, which is quite a bit faster than the default xalan interpreted one. Maybe this is the cause and perhaps permgen is too small to account for a large template?> I'm just grasping at straws, I would need a lot more detail on exactly what failed to know what went wrong.

[shorero2011]

OK, I know what's going on. Everybody (Sun/Oracle, JBoss, me, ...) is using some sort of variant of the Apache Xalan XSLT implementation. It turns out that there are two TransformerFactory implementations in Xalan:

• org.apache.xalan.processor.TransformerFactoryImpl: The Javadoc for this class says "The TransformerFactoryImpl, which implements the TRaX TransformerFactory interface, processes XSLT stylesheets into a Templates object (a StylesheetRoot)." What it doesn't say is that this factory builds Templates objects that are interpreted to process incoming XML documents.
• org.apache.xalan.xsltc.trax.TransformerFactoryImpl: The Javadoc for this class says "Implementation of a JAXP1.1 TransformerFactory for Translets." What it doesn't say is that this class builds compiled Templates objects. I didn't dig down far enough in the code to verify, but I strongly suspect (given the error msg below) that this class builds Java source that is dynamically compiled to process incoming XML documents.

Looking at the stack trace at the start of this discussion, it appears that JBoss is providing the compiling factory. Spelunking through the source for this factory (DL'ed from the Apache web site), I noticed that the class supports an attribute named "debug" that can be set to a Boolean via the standard factory setAttribute() method. When I turned on "debug" and ran my code using the JBoss-provided TransformerFactory (via TransformerFactory.getInstance()), I got the following expanded error:

org.apache.xalan.xsltc.compiler.util.InternalError: Internal XSLTC error:  a method in the translet exceeds the Java Virtual Machine limitation on the length of a method of 64 kilobytes.  This is usually caused by templates in a stylesheet that are very large.  Try restructuring your stylesheet to use smaller templates.

      at org.apache.xalan.xsltc.compiler.util.MethodGenerator.outlineChunks(MethodGenerator.java:1197)

...

      at org.apache.xalan.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:799)

      at __redirected.__TransformerFactory.newTemplates(__TransformerFactory.java:137)

...

Now, in my case it's not a single big template that's causing the problem but a plethora of little ones. The result is the same: this xsltc compiler must be building a single Java method to handle a single file of templates, and too much generated code (either a few big templates or a lot of little ones) exceeds the 64K limit and makes the stylesheet un-compilable.

The next thing I tried was to specify the interpretative TransformerFactory without included a private copy of Xalan in my ear. This throws an exception: javax.xml.transform.TransformerFactoryConfigurationError: Provider org.apache.xalan.processor.TransformerFactoryImpl not found. However JBoss has modified/installed Apache Xalan, it's blocking or masking my access to the alternative TransformerFactory. SUGGESTION: allow access to the interpretative TransformerFactory, so others with my problem don't have to drag around a private copy of Xalan.

So, my final fix is the following:

1. Add the following Maven dependencies to my ear's pom.xml:

	<dependency>
	<groupId>xalan</groupId>
	<artifactId>xalan</artifactId>
	<version>2.7.1</version>
	<scope>runtime</scope>
	</dependency>
	<dependency>
	<groupId>xalan</groupId>
	<artifactId>serializer</artifactId>
	<version>2.7.1</version>
	<scope>runtime</scope>
	</dependency>

2. Add the following jar modules to the ear-builder plugin

	<jarModule>
	<groupId>xalan</groupId>
	<artifactId>xalan</artifactId>
	<bundleDir>lib</bundleDir>
	<bundleFileName>xalan.jar</bundleFileName>
	</jarModule>
	<jarModule>
	<groupId>xalan</groupId>
	<artifactId>serializer</artifactId>
	<bundleDir>lib</bundleDir>
	<bundleFileName>serializer.jar</bundleFileName>
	</jarModule>

3. At runtime, use TransformerFactory.getInstance(className, null), where className is the full class name of the interpretator org.apache.xalan.processor.TransformerFactoryImpl.

With these changes, the stylesheet file compiles and, I suspect, will run properly. Next step (for me, at least) is to try to restructure my stylesheet builder so I can safely use the compiling factory via a series of stylesheets.

Thanks again for your help.

[jason.greene]

shorero2011 wrote:

• org.apache.xalan.xsltc.trax.TransformerFactoryImpl: The Javadoc for this class says "Implementation of a JAXP1.1 TransformerFactory for Translets." What it doesn't say is that this class builds compiled Templates objects. I didn't dig down far enough in the code to verify, but I strongly suspect (given the error msg below) that this class builds Java source that is dynamically compiled to process incoming XML documents.

Right, that's exactly what I was referring to when I mentioned byte code generation to be the most likely problem.

The next thing I tried was to specify the interpretative TransformerFactory without included a private copy of Xalan in my ear. This throws an exception: javax.xml.transform.TransformerFactoryConfigurationError: Provider org.apache.xalan.processor.TransformerFactoryImpl not found. However JBoss has modified/installed Apache Xalan, it's blocking or masking my access to the alternative TransformerFactory. SUGGESTION: allow access to the interpretative TransformerFactory, so others with my problem don't have to drag around a private copy of Xalan.

That should work provided that you add the META-INF/services/javax.xml.transform.TransformerFactory file with that reference, and that you also add "Dependency: org.apache.xalan" to your MANIFEST.MF of the deployments which use xalan (includes subdeployments). It's very important though that in this case you  you DO NOT import services. This ensures that your copy of the TransformerFactory file is given preference over the dependency. Note if you don't like teh manifest approach we also have an xml descriptor that does the same (and more). See the following for more information:

https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7

3. At runtime, use TransformerFactory.getInstance(className, null), where className is the full class name of the interpretator org.apache.xalan.processor.TransformerFactoryImpl.

With these changes, the stylesheet file compiles and, I suspect, will run properly. Next step (for me, at least) is to try to restructure my stylesheet builder so I can safely use the compiling factory via a series of stylesheets.

This might not work on some jdks, as the behavior on passing null isn't specified. You could however use Thread.currentThread().getContextClassloader() as the argument to that parameter. The other thing to look out for though is that you are always calling this factory method within an EE context of some sort (somewhere in ejb, servlet code, not in a custom thread). I assume that you are. Also as mentioned above you MUST have a dependency on org.apache.xalan listed in every deployment. Otherwise the classes will not be visible unless you decided to include xalan in your deployment like you mentioned. Using this approach of the two arg constructor means you do not need the META-INF/services/javax.xml.transform.TransformerFactory file.

[shorero2011]

re Right... byte code generation: Makes sense now that I know what the problem is but was too subtle for me to understand at the time ;-)

re That should work provided...: Hmmm - all I can say is that I'm getting the proper class without any services entry or any dependency entries. All I did was change the code to use the getInstance(className, null) method rather than getInstance().

re behavior on passing null: Remember that I'm using 1.7. The Javadocs indicate that a null class loader is interpreted as Thread.currentThread().getContextClassloader().

re EE context: Everything I'm talking about is inside a JMX bean. JBoss starts -> ear deploys -> loader war deploys w/ load-at-startup servlet -> loader EJB -> create and register a bunch of JMX beans.

re do not need...: Ah, that's why it works. I did in fact include xalan (and serialize.jar as you suggested) in the ear's lib directory. In other words, I'm dragging along a private copy of the apache stuff rather than relying on somebody else's copy. At some point I may revisit this with your suggestion about the service/manifest changes, but at the moment I'd rather look more closely at my XSLT file generator to see if I can safely break it into a series of transforms that can be compiled individually and executed sequentially with the same overall effect as the single large transform. It's "just" a matter of arranging the templates so I can "draw a line" in the sequence without changing the final result. If I can get the individual files small enouth, then I can use the compiling version of the TransformerFactory, which I suspect is significantly faster than the interpreted version (hmmm - I should probably test this - another todo entry)

[yunshi]

You helped me ! Thanks.