3 Replies Latest reply on Apr 24, 2009 9:02 AM by brian.stansberry

    BaseClassLoader blocks.

    pbmwg

      I have an application where I am seeing a lot of blocks on the synchronized block of BaseClassloader (when I look at thread dumps) which I would not normally expect (I would expect most of the time to see the stack traces doing database IO or some-such, but mostly I see thread blocked on classloading). The thread that is holding the lock (I believe) has the stack trace below. It seems to me that a lot of work is going on with the classloading when I would expect most classes to be cached (this stack trace is after the app has been running several hours). Do I have something mis-configured, or am I simply ignorant in some fashion (ok, that is a given, I know :-) ) ? An extra bit of information: I am using EJB3's over http. Thanks for any help

      java.net.URLClassLoader.findClass(URLClassLoader.java:188)
      java.lang.ClassLoader.loadClass(ClassLoader.java:306)
      - locked <0x52cc95d> (a org.jboss.system.NoAnnotationURLClassLoader)
      java.lang.ClassLoader.loadClass(ClassLoader.java:251)
      java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
      - locked <0x52cc95d> (a org.jboss.system.NoAnnotationURLClassLoader)
      java.lang.Class.forName0(Native Method)
      java.lang.Class.forName(Class.java:247)
      org.jboss.classloader.plugins.loader.ClassLoaderToLoaderAdapter.loadClass(ClassLoaderToLoaderAdapter.java:173)
      org.jboss.classloader.spi.ClassLoaderDomain.loadClassFromParent(ClassLoaderDomain.java:352)
      org.jboss.classloader.spi.ClassLoaderDomain.loadClassBefore(ClassLoaderDomain.java:307)
      org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:243)
      org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1099)
      org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:735)
      org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:378)
      - locked <0x611c3b04> (a org.jboss.classloader.spi.base.BaseClassLoader)
      java.lang.ClassLoader.loadClass(ClassLoader.java:251)
      org.jboss.aop.util.MarshalledValueInputStream.resolveClass(MarshalledValueInputStream.java:58)
      java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1575)
      java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
      java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1583)
      java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
      java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
      java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
      java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
      java.util.HashMap.readObject(HashMap.java:1030)
      sun.reflect.GeneratedMethodAccessor252.invoke(Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      java.lang.reflect.Method.invoke(Method.java:597)
      java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
      java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849)
      java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
      java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
      java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
      java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
      java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
      java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
      java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
      org.jboss.aop.util.MarshalledValue.get(MarshalledValue.java:87)
      org.jboss.aop.metadata.SimpleMetaData$MetaDataValue.get(SimpleMetaData.java:88)
      org.jboss.aop.metadata.SimpleMetaData.getMetaData(SimpleMetaData.java:211)
      - locked <0x725b298b> (a org.jboss.aop.metadata.SimpleMetaData)
      org.jboss.aop.metadata.SimpleMetaData.resolve(SimpleMetaData.java:283)
      - locked <0x725b298b> (a org.jboss.aop.metadata.SimpleMetaData)
      org.jboss.aop.joinpoint.InvocationBase.getMetaData(InvocationBase.java:251)
      org.jboss.aop.joinpoint.MethodInvocation.getMetaData(MethodInvocation.java:240)
      org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:78)
      org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
      org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:41)
      org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
      org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)
      org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
      org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:65)
      org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
      org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67)
      org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
      org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke(StatelessContainer.java:486)
      org.jboss.ejb3.session.InvokableContextClassProxyHack._dynamicInvoke(InvokableContextClassProxyHack.java:56)
      org.jboss.aop.Dispatcher.invoke(Dispatcher.java:91)
      org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
      org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:908)
      org.jboss.remoting.transport.servlet.ServletServerInvoker.processRequest(ServletServerInvoker.java:309)
      org.jboss.remoting.util.ServletSecurityUtility.processRequest(ServletSecurityUtility.java:53)
      org.jboss.remoting.transport.servlet.web.ServerInvokerServlet.processRequest(ServerInvokerServlet.java:128)
      org.jboss.remoting.transport.servlet.web.ServerInvokerServlet.doPost(ServerInvokerServlet.java:157)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
      javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
      org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
      org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
      org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
      org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
      org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
      org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
      org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
      org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
      org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
      org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
      org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
      org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:601)
      org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
      java.lang.Thread.run(Thre

        • 1. Re: BaseClassLoader blocks.
          pbmwg

          I have stepped this in the debugger. There are several jboss classes that are unmarshalled (security/auth classes in jbosssx.jar) that are never found in *any* classloader, and I can see in the debugger that these classes are being loaded from the jars *for every remote ejb invocation*. Some of the classes I have identified are JBossSecurityContext, JBossCallbackHandler, CredentialIdentityFactory and DefaultSecurityManagement.

          I removed authorization from the ejb3 interceptors and my load has decreased by 66%, incredibly sizable for default auth.

          Is this a bug?

          • 2. Re: BaseClassLoader blocks.
            anil.saldhana

             

            "pbmwg" wrote:
            I have stepped this in the debugger. There are several jboss classes that are unmarshalled (security/auth classes in jbosssx.jar) that are never found in *any* classloader, and I can see in the debugger that these classes are being loaded from the jars *for every remote ejb invocation*. Some of the classes I have identified are JBossSecurityContext, JBossCallbackHandler, CredentialIdentityFactory and DefaultSecurityManagement.

            I removed authorization from the ejb3 interceptors and my load has decreased by 66%, incredibly sizable for default auth.

            Is this a bug?


            All these security classes should be available in client/jbosssx-client.jar
            They are needed for security.

            • 3. Re: BaseClassLoader blocks.
              brian.stansberry

              pbmwg, thanks for the analysis and stack trace. Please see discussion I just opened at http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4226629#4226629 .