EJB client class loader memory leaks
pref Apr 26, 2013 8:13 AMHi all,
I'm using JBoss EJB client in a web application deployed on Tomcat 6/7 to interact with remote JBoss server. There are multiple issues with class loader leaks when the application undeploys. I discovered them a bit and found the causes of these leaks. Maybe it will be helpful to somebody because there is no documentation on this question at all.
First, you must manually close EJB client context to stop Remoting threads:
EJBClientContext.getCurrent().close();
There is a problem in Remoting3 library. RemotingConnectionProvider registers MBean but nobody seems going to unregister it. https://issues.jboss.org/browse/REM3-171 So, you must also add this ugly code to your application:
MBeanServer mbeanServer = ManagementFactory.getPlatformMBeanServer(); for (ObjectName name : mbeanServer.queryNames(new ObjectName("jboss.remoting.handler:name=*"), null)) { if (mbeanServer.getClassLoaderFor(name).equals(thisClassLoader)) { mbeanServer.unregisterMBean(name); } }
There is another problem in EJB Client library that registers shutdown hooks but never unregisters them. https://issues.jboss.org/browse/EJBCLIENT-78 To cope with it you must add this:
Field targetField = Thread.class.getDeclaredField("target"); targetField.setAccessible(true); Class<?> clazz = Class.forName("java.lang.ApplicationShutdownHooks"); Field hooksField = clazz.getDeclaredField("hooks"); field.setAccessible(true); Map<Thread, Thread> hooks = (Map<Thread, Thread>) field.get(null); for (Thread thread : hooks.keySet()) { Runnable target = (Runnable) targetField.get(t); if (target != null && target.getClass().getClassLoader().equals(thisClassLoader)) { Runtime.getRuntime().removeShutdownHook(thread); break; } }
Finally there are two old known Java problems with soft references preventing your classes to be unloaded.
- http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6543126
- http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6232010, it marked resolved however there are still some problems in ObjectStreamClass. (see clearClassLoaderCaches() in http://svn.apache.org/viewvc/tomee/tomee/trunk/container/openejb-core/src/main/java/org/apache/openejb/ClassLoaderUtil.java?revision=1457337&view=markup for example).
To handle with them you must add this:
Field field = Level.class.getDeclaredField("known"); field.setAccessible(true); List<Level> levels = (List<Level>) field.get(null); synchronized (Level.class) { Iterator<Level> it = levels.iterator(); while (it.hasNext()) { Level level = it.next(); if (level.getClass().getClassLoader().equals(thisClassLoader)) { it.remove(); } } }
and this:
private static void clearCache(Class<?> clazz, String fieldName) { Field field = clazz.getDeclaredField(fieldName); field.setAccessible(true); Map<?, ?> cache = (Map<?, ?>) field.get(null); synchronized (cache) { cache.clear(); } } ... Class<?> clazz = Class.forName("java.io.ObjectStreamClass$Caches"); clearCaches(clazz, "localDescs"); clearCaches(clazz, "reflectors");
After adding this code to ServletContextListener.contextDestroyed Tomcat is happy when application undeploys (there are still some problems with unremoved ThreadLocals in org.jboss.marshalling.UTFUtils.BytesHolder and in org.jboss.remoting3.remote.ProtocolUtils but they are some hunders of bytes in sum, and they look harmless if you use Tomcat 7, because it periodically renew threads)