Classloading failure in a new thread started from the cluste

[chtimi2]

Here is what i do:

1/ I'm notified of a topology change in my MembershipListener (which is an inner class of an MBean)

2/ I call asynchronously (so that i'm no longer in the JGroups thread) a method that notifies the nodes (via the clustered JMX notification mechanism)

3/ When a node's NotificationListener receives the event, it calls a SLSB entity finder method (findAllCentreExploitation).
The SLSB is looked up as remote but the call happens on the same node as the NotificationListener-->the classloading issue doesn't seem to come from a remoting issue?

It doesn't change anything whether i call the SLSB in a new thread or not, it still fails loading the entity class.
I'm really stuck here..

Something that could be (is probably) related, I have had classloading issues from a JGroups thread before, that i worked around by converting the problematic application object to and from an array of J2SE objects (so no application classloading necessary).
Do i have to tell JBoss to use the right classloader, or what else can i do?


Something else that seems strange that i thought i should mention; it doesn't fail on all topology changes, only certain ones.
For instance, i have 2 nodes A and B; the successive topologies are:
(A)-->(A,B)-->(B)-->(A,B)
IT ONLY FAILS ON THE LAST TOPOLOGY CHANGE! But maybe that's not important and the real solution is doing something about the classloading?

Here is the stacktrace:

17:29:36,315 INFO [STDOUT] [com.navineo.sa.jmx.ha.taches.ComEmbarquesHAS][IncomingPacketHandler (channel=PartitionNavineo)] - DEBUT demarrer
17:29:36,325 ERROR [STDERR] java.lang.reflect.UndeclaredThrowableException
17:29:36,325 ERROR [STDERR] at $Proxy411.findAllCentreExploitation(Unknown Source)
17:29:36,325 ERROR [STDERR] at com.navineo.sa.jmx.ha.taches.DummyTacheEgoiste1HAS$PrintTotoRunnable.run(DummyTacheEgoiste1HAS.java:57)
17:29:36,326 ERROR [STDERR] at java.lang.Thread.run(Thread.java:619)
17:29:36,326 ERROR [STDERR] Caused by: java.io.IOException
17:29:36,326 ERROR [STDERR] at org.jboss.serial.persister.RegularObjectPersister.readSlotWithMethod(RegularObjectPersister.java:107)
17:29:36,326 ERROR [STDERR] at org.jboss.serial.persister.RegularObjectPersister.defaultRead(RegularObjectPersister.java:269)
17:29:36,326 ERROR [STDERR] at org.jboss.serial.persister.RegularObjectPersister.readData(RegularObjectPersister.java:241)
17:29:36,326 ERROR [STDERR] at org.jboss.serial.objectmetamodel.ObjectDescriptorFactory.readObjectDescriptionFromStreaming(ObjectDescriptorFactory.java:412)
17:29:36,326 ERROR [STDERR] at org.jboss.serial.objectmetamodel.ObjectDescriptorFactory.objectFromDescription(ObjectDescriptorFactory.java:82)
17:29:36,326 ERROR [STDERR] at org.jboss.serial.objectmetamodel.DataContainer$DataContainerInput.readObject(DataContainer.java:845)
17:29:36,326 ERROR [STDERR] at org.jboss.serial.io.MarshalledObjectForLocalCalls.get(MarshalledObjectForLocalCalls.java:60)
17:29:36,326 ERROR [STDERR] at org.jboss.ejb3.remoting.IsLocalInterceptor.invokeLocal(IsLocalInterceptor.java:96)
17:29:36,326 ERROR [STDERR] at org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor.invoke(ClusteredIsLocalInterceptor.java:53)
17:29:36,326 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
17:29:36,326 ERROR [STDERR] at org.jboss.ejb3.stateless.StatelessClusteredProxy.invoke(StatelessClusteredProxy.java:112)
17:29:36,326 ERROR [STDERR] ... 3 more
17:29:36,326 ERROR [STDERR] Caused by: java.lang.reflect.InvocationTargetException
17:29:36,326 ERROR [STDERR] at sun.reflect.GeneratedMethodAccessor130.invoke(Unknown Source)
17:29:36,326 ERROR [STDERR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
17:29:36,327 ERROR [STDERR] at java.lang.reflect.Method.invoke(Method.java:597)
17:29:36,327 ERROR [STDERR] at org.jboss.serial.persister.RegularObjectPersister.readSlotWithMethod(RegularObjectPersister.java:103)
17:29:36,327 ERROR [STDERR] ... 13 more
17:29:36,327 ERROR [STDERR] Caused by: java.io.IOException: No ClassLoaders found for: com.navineo.dock.commande.unitaire.CentreExploitation
17:29:36,327 ERROR [STDERR] at org.jboss.serial.classmetamodel.ClassMetamodelFactory.getClassMetaData(ClassMetamodelFactory.java:332)
17:29:36,327 ERROR [STDERR] at org.jboss.serial.classmetamodel.StreamingClass.readStream(StreamingClass.java:72)
17:29:36,327 ERROR [STDERR] at org.jboss.serial.objectmetamodel.ObjectDescriptorFactory.readObjectDescriptionFromStreaming(ObjectDescriptorFactory.java:381)
17:29:36,327 ERROR [STDERR] at org.jboss.serial.objectmetamodel.ObjectDescriptorFactory.objectFromDescription(ObjectDescriptorFactory.java:82)
17:29:36,327 ERROR [STDERR] at org.jboss.serial.objectmetamodel.DataContainer$DataContainerInput.readObject(DataContainer.java:845)
17:29:36,327 ERROR [STDERR] at org.jboss.serial.persister.ObjectInputStreamProxy.readObjectOverride(ObjectInputStreamProxy.java:68)
17:29:36,327 ERROR [STDERR] at java.io.ObjectInputStream.readObject(ObjectInputStream.java:345)
17:29:36,327 ERROR [STDERR] at java.util.ArrayList.readObject(ArrayList.java:593)
17:29:36,327 ERROR [STDERR] ... 17 more
17:29:36,327 ERROR [STDERR] Caused by: java.lang.ClassNotFoundException: No ClassLoaders found for: com.navineo.dock.commande.unitaire.CentreExploitation
17:29:36,327 ERROR [STDERR] at org.jboss.mx.loading.LoadMgr3.beginLoadTask(LoadMgr3.java:212)
17:29:36,327 ERROR [STDERR] at org.jboss.mx.loading.RepositoryClassLoader.loadClassImpl(RepositoryClassLoader.java:521)
17:29:36,327 ERROR [STDERR] at org.jboss.mx.loading.RepositoryClassLoader.loadClass(RepositoryClassLoader.java:415)
17:29:36,328 ERROR [STDERR] at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
17:29:36,328 ERROR [STDERR] at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
17:29:36,328 ERROR [STDERR] at java.lang.Class.forName0(Native Method)
17:29:36,328 ERROR [STDERR] at java.lang.Class.forName(Class.java:247)
17:29:36,328 ERROR [STDERR] at org.jboss.serial.classmetamodel.ClassMetamodelFactory.resolveClassByName(ClassMetamodelFactory.java:274)
17:29:36,328 ERROR [STDERR] at org.jboss.serial.classmetamodel.ClassMetamodelFactory.getClassMetaData(ClassMetamodelFactory.java:289)
17:29:36,328 ERROR [STDERR] ... 24 more

[chtimi2]

When it works, the ClassLoader used is:

getClassLoader: org.jboss.mx.loading.UnifiedClassLoader3@1d2300e{ url=file:/opt/jboss-4.2.3.GA/server/all/tmp/deploy/tmp7477Navineo.ear ,addedOrder=45}

When it fails, the ClassLoader used is:

DummyTacheEgoiste1HAS/getClassLoader: org.jboss.mx.loading.UnifiedClassLoader3@183d59c{ url=null ,addedOrder=20}

The problem is obviously that url=null so the ear classes are unknown. But what can i do about it?

[chtimi2]

I have found a way to make it work, but i still don't understand what happens.
Our ear has a jboss-app.xml, which contained the following:

<loader-repository>
 seam.jboss.org:loader=navineo.ear
 <loader-repository-config>
 java2ParentDelegation=false
 </loader-repository-config>
 </loader-repository>

(we have a small Seam application in the ear)

If i comment this it works but the UCL's url property is still null, so i was wrong that it was the cause.
I have read the doc and wiki about classloader isolation but i still don't get why it changes anything in my case since i only have 1 ear and i cold-deploy everything.

[brian.stansberry]

Your problem is the thread doesn't have the correct classloader assigned to it via Thread.setContextClassLoader(). The clustered JMX notification is going to be delivered by a thread coming up from JGroups:

17:29:36,315 INFO [STDOUT] [com.navineo.sa.jmx.ha.taches.ComEmbarquesHAS][IncomingPacketHandler (channel=PartitionNavineo)] - DEBUT demarrer

So it's not going to have your ear's classloader assigned to it. If you spawn a thread to handle the findAllCentreExploitation call, it's going to inherit whatever TCCL is bound to the JGroups thread as it's TCCL. You need to assign the correct TCCL to that thread.

Commenting out the loader-repository element makes the problem go away because it means your application classes are available to the thread's classloader. That's a band-aid though.

[chtimi2]

Thread.setContextClassLoader()

What is the correct way to get a handle to the application ClassLoader (ear ClassLoader)? I couldn't find the info anywhere. I also tried to use

UnifiedClassLoader ucl = (UnifiedClassLoader) Thread.currentThread().getContextClassLoader();
String earPath = "file:/opt/jboss-4.2.3.GA/server/all/deploy/[MyApplication].ear";
URL earURL = new URL ( earPath );
ucl.addURL ( earURL );

but to no avail (the UCl's URL property was still null no matter what).

Also is there an intrinsic problem with not using the <loader-repository> parameter? Outside of the fact you can't specify isolation of course but if you don't is it a bad practice?

Maybe it doesn't do what i think it does?
From the wiki this specifies Isolation with Overriding Server Classes so it seems to me to mean that the Seam application overrides server classes, that is classes that are in the ear are always loaded from there even if they exist in "upper" classloaders. But why does it do anything to the classloading of classes not in the Seam application?

[brian.stansberry]

A couple techniques to get a ref to a classloader:

1) Call getClass().getClassLoader() on some class that you know was loaded from the ear. From your stack trace com.navineo.sa.jmx.ha.taches.DummyTacheEgoiste1HAS seems like a candidate.

2) You can call Thread.currentThread().getContextClassLoader() when you know the correct classloader is in place. Best time for that is during some call that happens during application deployment. So, say class com.navineo.sa.jmx.ha.taches.DummyTacheEgoiste1HAS for whatever reason isn't packaged in the ear, but you know a particular instance of that class is constructed as part of the deployment of the ear. So you could call Thread.currentThread().getContextClassLoader() in the constructor.

Re: loader-repository in general, see the links under

https://www.jboss.org/community/docs/DOC-9287

particularly https://www.jboss.org/community/docs/DOC-10290.

Re: using the loader-repository element in a Seam app, if the Seam docs are saying to do that, you'll have to ask the Seam folks why. (I'm not implying they are wrong to say to do it; they very much know what they are doing.)

[chtimi2]

Thanks for your help, i tried using

Thread.currentThread().setContextClassLoader ( DummyTacheEgoiste1HAS.class.getClassLoader() );

and it works even if put again the isolation parametrer.

Can't believe i didn't think about it with the more complicated stuff i tried, oh well.

I have already read those looks but now that i know it works like this i'm going to read it again.

cheers