java.lang.ClassNotFoundException Module "org.jgroups:main" from local module loader

[niteshmehta]

Hi

We are running JBoss 7.1.1 Final. We run Jgroup channel and send Messages over it. In org.jgroups.Message we set our custom objects which are present inside jars in lib of our ear.

our ear depends on module called pam "Dependencies: pam" and pam module add dependacy of jgroups  <dependencies><module name="org.jgroups" export="true"/></dependencies>

Now when we try to send and receive messages we are getting following exceptions

INFO   | jvm 1    | 2013/04/22 14:17:42 | 14:17:42,447 SEVERE [org.jgroups.protocols.pbcast.NAKACK] (Incoming-8,PAM_Cluster_009bab72-bd07-4a97-993f-307f0161cd93,6ea3721c-4e05-4d4c-b0de-d0e17acdbaf0) couldn't deliver message [dst: <null>, src: ba1438dc-c17d-4dca-b30d-62bd4d755b3d (3 headers), size=859 bytes]: java.lang.IllegalArgumentException: java.lang.ClassNotFoundException: com.pam.automation.server.GlobalDBNotification from [Module "org.jgroups:main" from local module loader @6f991ccf (roots: C:\Jboss7\node1\standalone\..\modules)]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.Message.getObject(Message.java:379) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at com.pam.automation.server.ClusterViewChangeListener.processMessagage(ClusterViewChangeListener.java:119) [clusterservices-snapshot.jar:4.2]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at com.pam.automation.server.ClusterViewChangeListener$1.receive(ClusterViewChangeListener.java:84) [clusterservices-snapshot.jar:4.2]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.JChannel.invokeCallback(JChannel.java:758) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.JChannel.up(JChannel.java:719) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:178) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.UNICAST.up(UNICAST.java:332) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:730) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:559) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.BARRIER.up(BARRIER.java:126) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.FD.up(FD.java:273) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:282) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.Discovery.up(Discovery.java:355) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.TP.passMessageUp(TP.java:1174) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1699) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 | Caused by: java.lang.ClassNotFoundException: com.pam.automation.server.GlobalDBNotification from [Module "org.jgroups:main" from local module loader @6f991ccf (roots: C:\Jboss7\node1\standalone\..\modules)]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190)

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468)

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456)

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398)

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120)

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.lang.Class.forName0(Native Method) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.lang.Class.forName(Class.java:264) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:622) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1593) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1514) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1750) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:498) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at javax.management.Notification.readObject(Notification.java:370) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:991) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1866) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369) [rt.jar:1.7.0_01]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.util.Util.objectFromByteBuffer(Util.java:416) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           at org.jgroups.Message.getObject(Message.java:376) [jgroups-3.0.6.Final.jar:3.0.6.Final]

INFO   | jvm 1    | 2013/04/22 14:17:42 |           ... 26 more

Currently to solve this problem we have added jgroup jar inside lib of our ear.

Is there any other way to solve this type of serialization problem?

[vikrammurugesan]

Nitesh,

I am having the same issue as the Rsp objects are usually our custom objects. I did not try putting the jgroups jar into the ear library yet. But just wanted to ask you if you figured out an alternate solution to this.

Did you guys fix this or you still have jgroups jar in your ear's library?

[andilog]

You can add global module to your server.

This will be available for all applications on your server.

Do that like follows:

If you have "yourExample.jar", which contains "org.what.ever" package.

1-

Create following directory:

{$jboss-home-directory}/modules/org/what/ever/main/

and copy yourExample.jar here.

2.

Create module.xml with following content:

<module xmlns="urn:jboss:module:1.1" name="org.what.ever">
    <resources>
        <resource-root path="yourExample.jar"/>
    </resources>

    <dependencies>
        <module name="com.other.example.module"/>
    </dependencies>
</module>

Note: add <dependencies> if know that there are some, else delete it.

3.

Add in your standalone.xml (search for "urn:jboss:domain:ee"):

...
<subsystem xmlns="urn:jboss:domain:ee:1.0">
     <ear-subdeployments-isolated>false</ear-subdeployments-isolated>
     <global-modules>
...
               <module name="org.what.ever" slot="main"/>
     </global-modules>
</subsystem>
...

4.

Restart server.

I am looking for alternative solutions also...

[vikrammurugesan]

Andreas - Appreciate your response. and thanks for the solution.

I was thinking dropping the jar into my ear should fix the issue. I am going to try that, which shouldnt take too long.

Right now I'm kinda stuck not able to proceed withouth getting this fixed. So I will reply to this post with my findings and solution. that way it will help someone else who has the same issue.

[jaikiran]

Can you try this against a latest released WildFly version http://wildfly.org/download/

[vikrammurugesan]

Jaikiran - I am using EAP version 6.0.1 which uses AS 7.1.3.Final. Do you know if there is any EAP version that might have this fix. It would be easy for me to try this in an EAP rather than an AS.

[vikrammurugesan]

Quick update on this issue. The issue is gone after dropping jgroups jar inside my ear's lib  directory. Just in case if some one is looking for how to do that, read this http://pic.dhe.ibm.com/infocenter/radhelp/v9/index.jsp?topic=%2Fcom.ibm.etools.maven.doc%2Ftopics%2Fadd_libs_to_ear_lib_…

Jaikiran - Using a different version of EAP is a very big process in my company. Each version is certified by the company before the teams get to use it. Appreciate if you can tell us if the fix is available in any of the EAP versions after 6.0.1. Thanks for your time.

[jaikiran]

Vikram, if you are using EAP version then you are entitled to one-off patches and other similar means to get access to fixes which are feasible. Please get in touch with the support team via the support portal so that they can help you appropriately.

[vikrammurugesan]

Thanks Jaikiran. I will reach out to the support team.