JAX-WS Client JBoss Modules/jboss-deployment-structure.xml Configuration

[jfisherdev]

I have an EAR application, which is deployed to a standalone WildFly 9.0.2.Final server, in which I want to use a JAX-WS client.

The JAX-WS client is written using standard JAX-WS APIs, so it does not target a specific implementation, and it works as expected when used from a standalone client application. However, when I try to use it in the EAR application, there are class loading and linkage errors when making the client call.

The error information that results when making the client call is given below, but needs a bit of explanation.

The "deployment.wsapp.wsapp-ws.war" module referenced in the error is a different EAR application, which has a web module with JAX-WS endpoints, than the one trying to use the JAX-WS client. It is NOT NECESSARILY the web module with the endpoints the client is targeting; however, whether the client is targeting that application or another one, even one that's on a different server [not necessarily a WildFly server, but a third party web server in some cases], the same error referencing that particular deployment module occurs.

java.lang.LinkageError: Failed to link com/sun/xml/messaging/saaj/soap/SOAPDocumentImpl (Module "deployment.wsapp.wsapp-ws.war:main" from Service Module Loader)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:437)
at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:269)
at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:77)
at org.jboss.modules.Module.loadModuleClass(Module.java:560)
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:197)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:455)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:404)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:385)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:130)
at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.<init>(SOAPPartImpl.java:106)
at com.sun.xml.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.<init>(SOAPPart1_1Impl.java:70)
at com.sun.xml.messaging.saaj.soap.ver1_1.Message1_1Impl.getSOAPPart(Message1_1Impl.java:96)

Caused by: java.lang.NoClassDefFoundError: com/sun/org/apache/xerces/internal/dom/DocumentImpl
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:353)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:432)
... 90 more
Caused by: java.lang.ClassNotFoundException: com.sun.org.apache.xerces.internal.dom.DocumentImpl from [Module "deployment.wsapp.ear.wsapp-ws.war:main" from Service Module Loader]
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:205)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:455)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:404)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:385)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:130)

The strange thing to me is that it's referencing the other deployment, even when that deployment isn't the target of the JAX-WS client call. Furthermore, when the EAR isn't present and I'm making a call to a different application or I put the EAR on a different server and have the client target that application it works fine.

One thing that may be worth mentioning is that the deployment, which has JAX-WS endpoints, referenced has references to com.sun.xml.ws.transport.http.servlet.WSServlet and com.sun.xml.ws.transport.http.servlet.WSServletContextListener in the WAR's web.xml, but no com.sun.xml.* references in any code, as well as a sun-jaxws.xml descriptor.

It also contains the following JARs:

• gmbal-api-only.jar
• ha-api.jar
• jaxb-api.jar
• jaxb-core.jar
• jaxb-impl.jar
• jaxb-jxc.jar
• jaxb-xjc.jar
• jaxws-rt.jar
• management-api.jar
• policy.jar
• resolver.jar
• saaj-impl.jar
• stax-ex.jar
• streambuffer.jar

We have been in the process of migrating to WildFly 9 from JBoss AS 4.2.2, and I believe this was done to work around the fact that the JBoss AS 4 configuration we had didn't support container-provided JAX-WS deployment, so it was necessary to basically bring a JAX-WS runtime with each deployment. I would like to get away from bringing these libraries and referencing the com.sun.xml libraries as soon as possible, but I am trying to hold off on making that change for now if possible since we are in a transitional state where backward compatibility with JBoss 4 needs to be maintained for the short term, though I will do it if necessary.

Looking at this guide JBoss Modules and WS applications - WildFly 9 - Project Documentation Editor , my guess is this will come down to JBoss Modules/jboss-deployment-structure.xml configuration or application library packaging, but I'm not sure what exactly needs to be done.

My main questions right now are:

- Is JAX-WS client functionality supported by default or does it require extra configuration? Looking at the article, it sounds like it is supported by default and that module dependency configuration is only necessary when you want to use a specific implementation, but I'm not sure.

- Why is a deployment that isn't necessarily involved in the client call referenced in a Linkage error and possibly causing it? Does this have something to do with the way the endpoints are deployed [use of com.sun.xml classes] that somehow affects which JAX-WS implementation is used?

Any information on this would be very much appreciated.

[jaikiran]

Can you post the entire exception stacktrace? The one in the post appears to be just part of the complete stacktrace. Also, paste a snippet of your client code which is triggering that exception.

[jfisherdev]

Thank you for responding.

I have included a more detailed stack trace below.

To keep things concise, I've omitted the portion of the stack trace leading up to the client call, since it's simply leading up to a Guice/Jersey JAX-RS service invocation that ultimately makes the client call.

java.lang.LinkageError: Failed to link com/sun/xml/messaging/saaj/soap/SOAPDocumentImpl (Module "deployment.wsapp.ear.wsapp-ws.war:main" from Service Module Loader)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:437)
at org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:269)
at org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:77)
at org.jboss.modules.Module.loadModuleClass(Module.java:560)
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:197)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:455)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:404)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:385)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:130)
at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.<init>(SOAPPartImpl.java:106)
at com.sun.xml.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.<init>(SOAPPart1_1Impl.java:70)
at com.sun.xml.messaging.saaj.soap.ver1_1.Message1_1Impl.getSOAPPart(Message1_1Impl.java:96)
at org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor.handleMessage(SAAJOutInterceptor.java:126)
at org.apache.cxf.jaxws.handler.soap.SOAPHandlerInterceptor.handleMessage(SOAPHandlerInterceptor.java:120)
at org.apache.cxf.jaxws.handler.soap.SOAPHandlerInterceptor.handleMessage(SOAPHandlerInterceptor.java:71)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307)
at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:516)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:425)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:326)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:279)
at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:138)
at com.sun.proxy.$Proxy440.getWorkInfo(Unknown Source)
at clientapp.services.WorkService.getWorkInfo(WorkService.java:30)
Caused by: java.lang.NoClassDefFoundError: com/sun/org/apache/xerces/internal/dom/DocumentImpl
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:353)
at org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:432)
... 90 more
Caused by: java.lang.ClassNotFoundException: com.sun.org.apache.xerces.internal.dom.DocumentImpl from [Module "deployment.wsapp.ear.wsapp-ws.war:main" from Service Module Loader]
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:205)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:455)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:404)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:385)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:130)
... 94 more

A few other things I wanted to note:

• No modifications have been made to the jboss-deployment-structure.xml as described in the previously mentioned documentation, so I'm not specifying any implementation modules.
• The webservices subsystem is active on the server and is NOT excluded from the client application deployment. The only configuration that's been done to the subsystem is to enable WSDL host rewrite.
• I see differences in behavior depending on whether the client call happens before or after the wsapp.ear being deployed. The cases, in which I deployed the apps individually, I tried and the results:
  • Deploy clientapp.ear, make client call which works, deploy wsapp.ear, make client call again and it still works.
  • Deploy clientapp.ear but don't make any client calls, deploy wsapp.ear, make a client call and it fails with the linkage error.

With that last point, it does seem that whatever happens first influences other behaviors.

[jaikiran]

It also contains the following JARs:

• gmbal-api-only.jar
• ha-api.jar
• jaxb-api.jar
• jaxb-core.jar
• jaxb-impl.jar
• jaxb-jxc.jar
• jaxb-xjc.jar
• jaxws-rt.jar
• management-api.jar
• policy.jar
• resolver.jar
• saaj-impl.jar
• stax-ex.jar
• streambuffer.jar

One thing to definitely consider is that many of these API/impl libraries including saaj do _not_ work correctly when it comes to classloading in a modular environment. That's specifically the reason why WildFly has forked those libraries and used it in WildFly. For example, saaj-impl has its own fork for WildFly here GitHub - asoldano/saaj-impl: Fork of saaj-impl  and is available as a Maven artifact here Index of /groups/public/com/sun/xml/messaging/saaj/saaj-impl/1.3.16-jbossorg-1. What that ultimately means is, the application that deployed on WildFly and packaging these libraries, would have to use the forked versions of these libraries to have them behave correctly in the (WildFly) modular environment. saaj-impl isn't the only one that needs to be taken care of, jaxb is another GitHub - jboss/jaxb: JBoss Fork of Java.net JAXB Project .

[jfisherdev]

Thank you for sharing that information.

jaikiran pai wrote:

One thing to definitely consider is that many of these API/impl libraries including saaj do _not_ work correctly when it comes to classloading in a modular environment.

Would you be able to elaborate on that or point me to something that does? I am curious about that and would like to understand it better.

I'm not sure if this is a proper comparison, but it reminds me of this situation:

One deployed application uses java.security.Security.addProvider() to add Provider implementation to the JVM-wide list of providers and the Provider implementation is only bundled by the application and isn't provided by the server. When another application tries to obtain a security API class instance without specifying a provider [e.g. KeyPairGenerator.getInstance(String algorithm)] you can run into class loading issues because the application can't load the particular provider/implementation that gets chosen.

One thing I did was use the jboss.modules:type=ModuleLoader,name=LocalModuleLoader-2 queryLoadedModuleNames() operation result to compare the set of loaded modules in the various scenarios I described earlier. What I found was in the case when the client app was able to make the call, the org.jboss.ws.saaj-impl:main module was present and absent when it could not.

If I'm understanding this correctly, it sounds like those libraries need to be removed from the deployments, unless you use the WildFly forks of them. I would prefer to get rid of them and that will likely be the direction we go. Would removing the webservices subsystem [or just excluding it from the deployments] and having the applications continue to bring their own implementations be another option? It isn't what I would want to do long term, but I am curious if that's a possibility.

[jfisherdev]

After experimenting further, it looks like the solution we will pursue is to remove these libraries and let the container handle JAX-WS service deployment as it probably should have been from the beginning. It will be much easier to manage and the deployments will be lighter by not packaging these libraries. I suppose there may be some challenges with migrating things that were using Sun implementation features, such as the sun-jaxws.xml for URL mapping, but we will deal with those issues as we come to them.

I am still curious about how exactly these cause issues with modular class loading to understand the issue better and to avoid other problems in the future. Any information on that would be appreciated, but my main question has been answered.