JBoss5 classloader issue after changing the classloader strategy

[fharms]

Hi

I’m struggling with the problem below after I had to change the classloader strategy. When the other nodes receive a message and marshalling try to load the class it get’s a “java.lang.ClassNotFoundException”. The JAR file containing the class is packed in the EAR file inside “lib\”.

java.lang.ClassNotFoundException: com.product.Rack
  at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:249)
  at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:300)
  at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1138)
  at org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:862)
  at org.jboss.classloader.spi.base.BaseClassLoader.doLoadClass(BaseClassLoader.java:502)
  at org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:447)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:249)
  at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:602)
  at org.jboss.invocation.MarshalledValueInputStream.resolveClass(MarshalledValueInputStream.java:109)
  at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1589)
  at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1494)
  at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1460)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1310)
  at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1969)
  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1775)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1327)
  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
  at java.util.ArrayList.readObject(ArrayList.java:593)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
  at java.lang.reflect.Method.invoke(Method.java:597)
  at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:969)
  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1775)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1327)
  at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1683)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1321)
  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
  at org.jgroups.blocks.MethodCall.readExternal(MethodCall.java:488)
  at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1814)
  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1773)
  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1327)
  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:349)
  at org.jboss.ha.framework.server.ClusterPartition.objectFromByteBufferInternal(ClusterPartition.java:1898)
  at org.jboss.ha.framework.server.ClusterPartition$RpcHandler.handle(ClusterPartition.java:2264)
  at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
  at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
  at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
  at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
  at org.jgroups.JChannel.up(JChannel.java:1336)
  at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
  at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:490)
  at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:153)
  at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
  at org.jgroups.protocols.FC.up(FC.java:473)
  at org.jgroups.protocols.pbcast.GMS.up(GMS.java:821)
  at org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:192)
  at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
  at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
  at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
  at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
  at org.jgroups.protocols.BARRIER.up(BARRIER.java:136)
  at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
  at org.jgroups.protocols.FD.up(FD.java:284)
  at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:328)
  at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
  at org.jgroups.protocols.Discovery.up(Discovery.java:264)
  at org.jgroups.protocols.PING.up(PING.java:273)
  at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2319)
  at org.jgroups.protocols.TP.passMessageUp(TP.java:1253)
  at org.jgroups.protocols.TP.access$100(TP.java:50)
  at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1830)
  at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1809)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
  at java.lang.Thread.run(Thread.java:680)

I have added following configuration file “jboss-classloading.xml” to the EAR file under “META-INF\”

<classloading xmlns="urn:jboss:classloading:1.0"
    domain="ISXO_Core.ear"
    export-all="NON_EMPTY"
    import-all="true"
    parent-first="false">
    </classloading>

And the jboss-app.xml file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE jboss-app PUBLIC
  "-//JBoss//DTD Java EE Application 5.0//EN"
  "http://www.jboss.org/j2ee/dtd/jboss-app_5_0.dtd">
<jboss-app>
  <loader-repository>com.apc:loader=ISXO_Core.ear<loader-repository-config>java2ParentDelegation=false</loader-repository-config>
  </loader-repository>
</jboss-app>

I didn't have this problem before I changed the classloader strategy, so how do I export my classes inside my EAR so a core service like HA ClusterPartition and marshalling is able to load the class?

/Flemming

[fharms]

Found out this was caused by a classloader issue when we registered a custom RPC handler, because it was invoked by the "DefaultDomain" classloader it didn't know anything about the classes inside the EAR file. The trick seems to be registred a RPC handler with a classloader.

partition.registerRPCHandler(serviceName, handler,Thread.currentThread().getContextClassLoader());

But that will lead to another issue in the org.jboss.ha.framework.server.ClusterPartition.handle(Message req), it fail because of the check in line 32. I suspect this is because the assignment of body happen in registred classloader and the check "!(body instanceof MethodCall)" happens in the “DefaultDomain” classloader and sees it as two different classes although it's the same type.

try
{
// If client registered the service with a classloader, override the thread classloader here
  WeakReference<ClassLoader> weak = ClusterPartition.this.clmap.get(service);
  if (weak != null)
  {
    if( trace )
    {
      this.log.trace("overriding Thread ContextClassLoader for RPC service " + service);
    }
    previousCL = Thread.currentThread().getContextClassLoader();
    ClassLoader loader = weak.get();
    overrideCL = true;
    Thread.currentThread().setContextClassLoader(loader);
  }
  body = ClusterPartition.this.objectFromByteBufferInternal(request_bytes);
}
catch (Exception e)
{
  this.log.warn("Partition " + ClusterPartition.this.getPartitionName() + " failed extracting message body from request bytes", e);
  return null;
}
finally
{
  if (overrideCL)
  {
     this.log.trace("resetting Thread ContextClassLoader");
      Thread.currentThread().setContextClassLoader(previousCL);
   }
 }
         
 if(body == null || !(body instanceof MethodCall))
 {
     this.log.warn("Partition " + ClusterPartition.this.getPartitionName() + " message does not contain a MethodCall object!");
            return null;
 }

The solution seem pretty easy by moving the check inside the set class loader context block

if(body == null || !(body instanceof MethodCall))
{
   this.log.warn("Partition " + ClusterPartition.this.getPartitionName() + " message does not contain a MethodCall object!");
   return null;
}