Sorry for slow reply; I got e-mail messages for other postings on this "Design of POJO Server" forum yesterday, but none for this topic. :(

Thanks for the quick patch; will get it tested out.

Dominik is running the EJB3 perf tests and while it's early, he reports that "but the app is running, and it's running much much faster :)"

The "but" above is because there are CNFE problems at server start; look to be due to webapps being unable to load classes from the domain. Key server.log snippet:

2009-04-28 10:43:38,715 DEBUG [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) injectionContainer enabled and processing beginning
2009-04-28 10:43:38,716 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/admin-console]] (main) Error configuring application listener of class org.jboss.web.tomcat.security.SecurityFlushSessionListener
java.lang.ClassNotFoundException: org.jboss.web.tomcat.security.SecurityFlushSessionListener
 at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
 at org.jboss.web.tomcat.service.TomcatInjectionContainer.newInstance(TomcatInjectionContainer.java:262)
 at org.jboss.web.tomcat.service.TomcatInjectionContainer.newInstance(TomcatInjectionContainer.java:256)
 at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3859)
 at org.apache.catalina.core.StandardContext.start(StandardContext.java:4393)
 at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeployInternal(TomcatDeployment.java:310)
 at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeploy(TomcatDeployment.java:142)
 at org.jboss.web.deployers.AbstractWarDeployment.start(AbstractWarDeployment.java:461)
 at org.jboss.web.deployers.WebModule.startModule(WebModule.java:118)
 at org.jboss.web.deployers.WebModule.start(WebModule.java:97)

The webapps that fail are declaring a loader-repository in jboss-web.xml. The CNFE is coming from:

Thread [main] (Suspended)
 ClassLoaderDomain(BaseClassLoaderDomain).checkClassBlackList(BaseClassLoader, String, String, boolean) line: 1477
 ClassLoaderDomain(BaseClassLoaderDomain).checkClassCacheAndBlackList(BaseClassLoader, String, String, boolean) line: 1503
 BaseClassLoader.checkCacheAndBlackList(String, boolean) line: 365
 BaseClassLoader.loadClass(String, boolean) line: 429
 WebCtxLoader$ENCLoader(ClassLoader).loadClass(String, boolean) line: 299
 WebCtxLoader$ENCLoader(ClassLoader).loadClass(String) line: 251
 TomcatInjectionContainer.newInstance(String, ClassLoader) line: 262

Just to experiment I wrapped the call to BaseClassLoader.checkCacheAndBlackList in a try/catch and caught the CNFE. The AS starts clean and the smoke tests pass. A mvn install in the jboss-cl codebase shows 3 tests with errors, but those were there before I made the change as well.

Looking at the actual usage, the catch could also be inside checkCacheAndBlackList itself, but that's a protected method that declares the exception in its signature.

I'm not committing this as you may have other thoughts and if not it's trivial.

Now running classloader leak tests...

The classloader leak tests pass.

Comparable, but TPS is still only 63% of what I see for EAP 4.3 at http://hudson.qa.jboss.com/hudson/view/EAP%20Perf+Fail/job/eap-4.3-perf-ejb3-buddy-async/lastSuccessfulBuild/artifact/report/report.html. So we still have a long way to go. But the slowdown could be for many reasons.

I went ahead and fixed the CNFE issue. https://jira.jboss.org/jira/browse/JBCL-101

Done. That would be this:

Index: BaseClassLoader.java
===================================================================
--- BaseClassLoader.java (revision 88165)
+++ BaseClassLoader.java (working copy)
@@ -351,18 +351,22 @@
 * Check the cache and blacklist
 *
 * @param name the name of the class
+ * @param failIfBlackListed <code>true</code> if a blacklisted class should
+ * result in ClassNotFoundException; <code>false</code>
+ * if a <code>null</code> return value is acceptable
 * @param trace whether trace is enabled
 * @return the class is if it is already loaded, null otherwise
- * @throws ClassNotFoundException when blacklisted
+ * @throws ClassNotFoundException when the class is blacklisted and
+ * <code>failIfBlackListed</code> is <code>true</code>
 */
- protected Class<?> checkCacheAndBlackList(String name, boolean trace) throws ClassNotFoundException
+ protected Class<?> checkCacheAndBlackList(String name, boolean failIfBlackListed, boolean trace) throws ClassNotFoundException
 {
 BaseClassLoaderPolicy basePolicy = policy;
 BaseClassLoaderDomain domain = basePolicy.getClassLoaderDomain();
 if (domain == null)
 return null;

- return domain.checkClassCacheAndBlackList(this, name, null, basePolicy.isImportAll());
+ return domain.checkClassCacheAndBlackList(this, name, null, basePolicy.isImportAll(), false);
 }

 /**
@@ -426,17 +430,9 @@
 if (result != null)
 return result;

- try
- {
- result = checkCacheAndBlackList(name, trace);
- if (result != null)
- return result;
- }
- catch (ClassNotFoundException blacklisted)
- {
- if (trace)
- log.trace(name + " has been blacklisted; cannot load from domain cache");
- }
+ result = checkCacheAndBlackList(name, false, trace);
+ if (result != null)
+ return result;

 synchronized (this)
 {
Index: BaseClassLoaderDomain.java
===================================================================
--- BaseClassLoaderDomain.java (revision 88158)
+++ BaseClassLoaderDomain.java (working copy)
@@ -1487,10 +1487,14 @@
 * @param name the name
 * @param path the path of the class resource
 * @param allExports whether to look at all exports
+ * @param failIfBlackListed <code>true</code> if a blacklisted class should
+ * result in ClassNotFoundException; <code>false</code>
+ * if a <code>null</code> return value is acceptable
 * @return the class when found in the cache
- * @throws ClassNotFoundException when the class is blacklisted
+ * @throws ClassNotFoundException when the class is blacklisted and
+ * <code>failIfBlackListed</code> is <code>true</code>
 */
- protected Class<?> checkClassCacheAndBlackList(BaseClassLoader classLoader, String name, String path, boolean allExports) throws ClassNotFoundException
+ protected Class<?> checkClassCacheAndBlackList(BaseClassLoader classLoader, String name, String path, boolean allExports, boolean failIfBlackListed) throws ClassNotFoundException
 {
 if (path == null)
 path = ClassLoaderUtils.classNameToPath(name);
@@ -1498,8 +1502,11 @@
 Class<?> result = checkClassCache(classLoader, name, path, allExports);
 if (result != null)
 return result;
-
- checkClassBlackList(classLoader, name, path, allExports);
+
+ if (failIfBlackListed)
+ {
+ checkClassBlackList(classLoader, name, path, allExports);
+ }
 return null;
 }