10 Replies Latest reply on Mar 30, 2009 4:28 PM by flavia.rainone

    Method Advised class on getInterceptorChainReadWriteLock wit

      I been searching the JIRA bugs related to AOP but not finding anything so I thought I might see if anyone has any suggestions.

      We have upgraded from AOP 1.5.6 in JBoss 4.2.2 GA to JBoss 5.0.1 using AOP 2.0.0 GA and are using the new instrumentor. All of the code was working in the older version of AOP.

      Here is the issue we are having now in JBoss 5.0.1. We have a set of objects that are access from different threads located in a thread safe map. These objects have Pointcuts defined on these objects for both fields and methods. During execution we get in a state when a read is occuring on the object with the MethodInfo interceptor chain locked by a ReadLock and another Object in the map write lock during a GeneratedClassAdvisor checkVersion call. It seems like somewhere in the advised classes the system is getting in to a state where the unlock never happens. Whenever another thread that attempts to use the object and or maybe even the advised class in question it will get BLOCKED. The methods have a single PER_VM instance aspect and the fields have a PER_INSTANCE aspect attached to each object in the Map. We are using mixins with the object in the map.


      jboss-aop.xml -- example aop configuration.

      <aop xmlns="urn:jboss:aop-beans:1.0">
       <aspect class="com.company.common.item.aspect.MethodAspect"/>
       <aspect class="com.company.common.item.aspect.FieldAspect" scope="PER_INSTANCE"/>
      
       <bind pointcut="execution(* *->@com.company.common.item.aspect.ObjectMethod(..)) OR
       execution(* @com.company.common.item.aspect.ObjectMethod->*(..)) OR
       execution(* @com.company.common.item.aspect.ObjectItem->*(..))">
      
       <advice name="trace"
       aspect="com.company.common.item.aspect.MethodAspect"/>
       </bind>
      
       <bind pointcut="field(* @com.company.common.item.aspect.ObjectItem->*)">
       <advice name="check"
       aspect="com.company.common.cache.aspect.FieldAspect"/>
       </bind>
      
       <introduction class="@com.company.common.item.aspect.ObjectItem">
       <mixin>
       <interfaces>com.company.common.item.aspect.ILoadable</interfaces>
       <class>com.company.common.item.aspect.ObjectLoader</class>
       <construction>new com.company.common.item.aspect.ObjectLoader(this)</construction>
       </mixin>
       </introduction>
      </aop>
      



      JStack output during a locked situations. All locks were waiting no running threads were active in our code it was all locked in the Advisors.

      Read Lock -- case
      condition [0x000000004aa6a000..0x000000004aa6eb00]
       java.lang.Thread.State: WAITING (parking)
       at sun.misc.Unsafe.park(Native Method)
       - parking to wait for <0x00007f30d52a4868> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:877)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1197)
       at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:594)
       at com.company.common.item.common.UserInfo$UserInfoAdvisor.setEmail_N_4117224176847888539(UserInfo$UserInfoAdvisor.java)
       at com.company.common.item.common.UserInfo$UserInfoInstanceAdvisor.setEmail_N_4117224176847888539(UserInfo$UserInfoInstanceAdvisor.java)
       at com.company.common.item.common.UserInfo.setEmail(UserInfo.java)
      



      Write Lock -- case
      condition [0x000000004ac6e000..0x000000004ac70a00]
       java.lang.Thread.State: WAITING (parking)
       at sun.misc.Unsafe.park(Native Method)
       - parking to wait for <0x00007f30d51a5858> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
       at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
       at com.company.common.item.common.UserInfo$UserInfoInstanceAdvisor.lockWriteInterceptorChains(UserInfo$UserInfoInstanceAdvisor.java)
       at com.company.common.item.common.UserInfo$UserInfoInstanceAdvisor.doRebuildForInstance(UserInfo$UserInfoInstanceAdvisor.java)
       at org.jboss.aop.GeneratedClassAdvisor$InstanceAdvisorStrategy.checkVersion(GeneratedClassAdvisor.java:1616)
       at org.jboss.aop.GeneratedClassAdvisor.checkVersion(GeneratedClassAdvisor.java:268)
       at com.company.common.item.common.UserInfo$UserInfoInstanceAdvisor.getUID_N_622210694221713302(UserInfo$UserInfoInstanceAdvisor.java)
       at com.company.common.item.common.UserInfo.getUID(UserInfo.java)
      



      I decompiled the advised class to see the structure of the implementation for the Pointcuts and it seems to have all the appropriate unlock statements in finally blocks and all exception cases from the best i can tell.
      I dont understand why it is trying to doRebuildForInstance().


      //DECOMPILED Advised Class
      
      public class UserInfo
       implements IUserInfo, Advised, ILoadable
      {
       public static class UserInfoAdvisor extends GeneratedClassAdvisor
       implements Untransformable
       {
      
       public Domain getDomain()
       {
       return domain;
       }
      
       protected void initialiseMethods()
       {
       Object obj = null;
       aop$MethodInfo_getEmail99631339274670319 = new MethodInfo(Class.forName("com.company.common.item.common.UserInfo"), 0x161f62cd9f690efL, 0xefeef414804be60eL, this);
       addMethodInfo(aop$MethodInfo_getEmail99631339274670319);
       Object obj1 = null;
       aop$MethodInfo_getStartPage_N_8828064419336332239 = new MethodInfo(Class.forName("com.company.common.item.common.UserInfo"), 0x857c6a2f07315c31L, 0xf71910379dc11de3L, this);
       addMethodInfo(aop$MethodInfo_getStartPage_N_8828064419336332239);
       Object obj2 = null;
       aop$MethodInfo_getHome4315750976580329194 = new MethodInfo(Class.forName("com.company.common.item.common.UserInfo"), 0x3be4a0a5ba1d9eeaL, 0xb7a3ea639c9f4219L, this);
       addMethodInfo(aop$MethodInfo_getHome4315750976580329194);
      
       .... truncated ....
       }
      
       protected void initialiseConstructors(Collection collection)
       {
       }
      
       protected void initialiseConstructions(Collection collection)
       {
       }
      
       protected void initialiseFieldReads(Collection collection)
       {
       Object obj = null;
       aop$FieldInfo_r_email = new FieldInfo(Class.forName("com.company.common.item.common.UserInfo"), 0, "email", 0xf2a8295678139d21L, this, true);
       addFieldReadInfo(aop$FieldInfo_r_email, collection);
       Object obj1 = null;
       aop$FieldInfo_r_home = new FieldInfo(Class.forName("com.company.common.item.common.UserInfo"), 1, "home", 0x5537394f14d81f41L, this, true);
       addFieldReadInfo(aop$FieldInfo_r_home, collection);
       Object obj2 = null;
       aop$FieldInfo_r_pageSize = new FieldInfo(Class.forName("com.company.common.item.common.UserInfo"), 2, "pageSize", 0x8f5f147f40d31e6dL, this, true);
       addFieldReadInfo(aop$FieldInfo_r_pageSize, collection);
       Object obj3 = null;
       aop$FieldInfo_r_startPage = new FieldInfo(Class.forName("com.company.common.item.common.UserInfo"), 3, "startPage", 0x423337f6036663e8L, this, true);
      
      
       ... truncated ...
      
       }
      
      
       // **** Example of Read Method pointcut ****
       // this is from JAD so finally block are not shown but can be determined by the goto statements
      
       protected String getEmail99631339274670319(UserInfo userinfo) throws MMSException
       {
       aop$MethodInfo_getEmail99631339274670319.getInterceptorChainReadWriteLock().readLock().lock();
       String s;
       if(joinpoint_getEmail99631339274670319 == null && aop$MethodInfo_getEmail99631339274670319 != null && aop$MethodInfo_getEmail99631339274670319.hasAdvices())
       super.generateJoinPointClass(aop$MethodInfo_getEmail99631339274670319);
       if(joinpoint_getEmail99631339274670319 == null)
       {
       s = (String)userinfo.com$company$common$item$common$UserInfo$getEmail$aop();
       } else
       {
       s = (String)joinpoint_getEmail99631339274670319.invokeJoinpoint(userinfo);
       break MISSING_BLOCK_LABEL_123;
       }
       goto _L1
      _L2:
       return s;
      _L3:
       return s;
       Exception exception;
       exception;
       aop$MethodInfo_getEmail99631339274670319.getInterceptorChainReadWriteLock().readLock().unlock();
       throw exception;
      _L1:
       aop$MethodInfo_getEmail99631339274670319.getInterceptorChainReadWriteLock().readLock().unlock();
       goto _L2
       aop$MethodInfo_getEmail99631339274670319.getInterceptorChainReadWriteLock().readLock().unlock();
       goto _L3
       }
      
      
      
       // **** Example of Write Method Pointcut ****
       // this is from JAD so finally block are not shown but can be determined by the goto statements
      
       protected void setEmail_N_4117224176847888539(UserInfo userinfo, Strings) throws MMSException
       {
       aop$MethodInfo_setEmail_N_4117224176847888539.getInterceptorChainReadWriteLock().readLock().lock();
       if(joinpoint_setEmail_N_4117224176847888539 == null && aop$MethodInfo_setEmail_N_4117224176847888539 != null && aop$MethodInfo_setEmail_N_4117224176847888539.hasAdvices())
       super.generateJoinPointClass(aop$MethodInfo_setEmail_N_4117224176847888539);
       if(joinpoint_setEmail_N_4117224176847888539 == null)
       userinfo.com$company$common$item$common$UserInfo$setEmail$aop(s);
       else
       joinpoint_setEmail_N_4117224176847888539.invokeJoinpoint(userinfo, s);
       break MISSING_BLOCK_LABEL_97;
       Exception exception;
       exception;
       aop$MethodInfo_setEmail_N_4117224176847888539.getInterceptorChainReadWriteLock().readLock().unlock();
       throw exception;
       aop$MethodInfo_setEmail_N_4117224176847888539.getInterceptorChainReadWriteLock().readLock().unlock();
       return;
       }
      
      
      
       //other set / get methods
       ... truncated ...
      
      



      Any help would be great, I dont know how to get this condition to happen it just happens so any help or direction would be great. Like I said before we did not see this occur in AOP 1.5.6 in JBoss 4.2.2 and our code did not change -- yet :) .

      -Pete

        • 1. Re: Method Advised class on getInterceptorChainReadWriteLock
          stalep

          hi, just to be sure that the new instrumentor might be to blame could you try to use the classic instrumentor and let us know if that works?
          you can set the instrumentor with this property -Djboss.aop.instrumentor=org.jboss.aop.instrument.ClassicInstrumentor

          • 2. Re: Method Advised class on getInterceptorChainReadWriteLock
            mike03

             

            "stale.pedersen@jboss.org" wrote:
            hi, just to be sure that the new instrumentor might be to blame could you try to use the classic instrumentor and let us know if that works?
            you can set the instrumentor with this property -Djboss.aop.instrumentor=org.jboss.aop.instrument.ClassicInstrumentor


            Hi, I'm working with Pete on this. I've got our app compile-time weaved using the Classic intrumentor but the ClassLoader can't find our aspects. We're using the same package structure that we use with GeneratedAdvisor instrumentation which doesn't have any problem finding classes.

            The ear is packaged as:

            was.ear
            was.ear/META-INF/application.xml
            was.ear/lib -> 3rd party jars
            was.ear/was.aop
            was.ear/was.aop/META-INF/jboss-aop.xml -> our aspect/joinpoint def file
            was.ear/was.aop/com/** -> our classes including aspects
            was.ear/was-web.war -> JSPs, web resources, etc

            Here's our application.xml:

            <?xml version="1.0" encoding="UTF-8"?>
            <!DOCTYPE application PUBLIC "-//Sun Microsystems, Inc.//DTD J2EE Application 1.3//EN" "http://java.sun.com/dtd/application_1_3.dtd">
            <application>
             <display-name>WAS :: Marble</display-name>
             <module>
             <java>was.aop</java>
             </module>
             <module id="marble-web.iml">
             <web>
             <web-uri>was-web.war</web-uri>
             <context-root>server</context-root>
             </web>
             </module>
            </application>
            


            From server.log on startup:

            2009-02-26 10:54:10,884 DEBUG [org.jboss.aop.microcontainer.beans.GenericBeanAspectFactory] (main) Creating advice com.company.common.cache.aspect.CacheAspect with loader org.jboss.system.NoAnnotationURLClassLoader@663e89c3
            2009-02-26 10:54:10,886 WARN [com.company.policy.impl.BootstrapPhase] (main) Error initializing component: unknown component
            java.lang.RuntimeException: java.lang.ClassNotFoundException: com.company.common.cache.aspect.CacheAspect
             at org.jboss.aop.microcontainer.beans.GenericBeanAspectFactory.doCreate(GenericBeanAspectFactory.java:151)
             at org.jboss.aop.microcontainer.beans.GenericBeanAspectFactory.createPerInstance(GenericBeanAspectFactory.java:105)
             at org.jboss.aop.InstanceAdvisorDelegate.initializeAspects(InstanceAdvisorDelegate.java:109)
             at org.jboss.aop.InstanceAdvisorDelegate.initialize(InstanceAdvisorDelegate.java:71)
             at org.jboss.aop.ClassInstanceAdvisor.setAdvisorAndInitialise(ClassInstanceAdvisor.java:88)
             at org.jboss.aop.ClassInstanceAdvisor.<init>(ClassInstanceAdvisor.java:72)
             at com.company.common.item.dbcache.DbCacheItem._getInstanceAdvisor(DbCacheItem.java)
             at com.company.common.item.dbcache.DbCacheItem.dbCacheKey_w_$aop(DbCacheItem.java)
             at com.company.common.item.dbcache.DbCacheItem.setDbCacheKey(DbCacheItem.java:29)
             at com.company.common.item.dbcache.DBCacheService.getItem(DBCacheService.java:161)
             at com.company.system.executive.alias.impl.AliasExecutive.createAlias(AliasExecutive.java:280)
             at com.company.system.executive.alias.impl.AliasExecutive.createAlias(AliasExecutive.java:155)
             at com.company.policy.command.system.xml.XMLImportAcl.importNode(XMLImportAcl.java:73)
             at com.company.policy.command.system.xml.XMLImport.xmlImportList(XMLImport.java:182)
             at com.company.policy.command.system.xml.XMLImport.xmlImport(XMLImport.java:152)
             at com.company.policy.biz.system.impl.SystemPolicy.xmlImport(SystemPolicy.java:44)
             at com.company.policy.impl.Bootstrap.init(Bootstrap.java:56)
             at com.company.framework.init.impl.DefaultPhase.createComponentData(DefaultPhase.java:87)
             at com.company.policy.impl.BootstrapPhase.createComponentData(BootstrapPhase.java:99)
             at com.company.framework.impl.BaseProvider.initComponents(BaseProvider.java:450)
             at com.company.framework.impl.BaseProvider.init(BaseProvider.java:99)
             at com.company.framework.init.impl.DefaultPhase.init(DefaultPhase.java:46)
             at com.company.policy.impl.BootstrapPhase.init(BootstrapPhase.java:51)
             at com.company.framework.init.impl.DefaultPhaseProvider.initComponent(DefaultPhaseProvider.java:157)
             at com.company.framework.impl.BaseProvider.initComponents(BaseProvider.java:458)
             at com.company.framework.impl.BaseProvider.init(BaseProvider.java:99)
             at com.company.framework.init.impl.DefaultPhaseProvider.initializePhases(DefaultPhaseProvider.java:80)
             at com.company.framework.CoreSystem.init(CoreSystem.java:686)
             at com.company.framework.CoreSystem.main(CoreSystem.java:645)
             at com.company.framework.impl.CoreServlet.init(CoreServlet.java:83)
             at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1048)
             at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:950)
             at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4122)
             at org.apache.catalina.core.StandardContext.start(StandardContext.java:4421)
             at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeployInternal(TomcatDeployment.java:312)
             at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeploy(TomcatDeployment.java:144)
             at org.jboss.web.deployers.AbstractWarDeployment.start(AbstractWarDeployment.java:461)
             at org.jboss.web.deployers.WebModule.startModule(WebModule.java:118)
             at org.jboss.web.deployers.WebModule.start(WebModule.java:97)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)
             at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)
             at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
             at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
             at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)
             at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:206)
             at $Proxy36.start(Unknown Source)
             at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:42)
             at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:37)
             at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
             at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
             at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
             at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
             at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:286)
             at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1598)
             at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
             at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1062)
             at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
             at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
             at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
             at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
             at org.jboss.system.ServiceController.start(ServiceController.java:460)
             at org.jboss.system.deployers.ServiceDeployer.start(ServiceDeployer.java:163)
             at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:99)
             at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:46)
             at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
             at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
             at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
             at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
             at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
             at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
             at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1210)
             at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
             at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
             at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1598)
             at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
             at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1062)
             at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
             at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
             at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
             at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
             at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:698)
             at org.jboss.system.server.profileservice.ProfileServiceBootstrap.loadProfile(ProfileServiceBootstrap.java:304)
             at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:205)
             at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:405)
             at org.jboss.Main.boot(Main.java:209)
             at org.jboss.Main$1.run(Main.java:547)
             at java.lang.Thread.run(Thread.java:619)
            Caused by: java.lang.ClassNotFoundException: com.company.common.cache.aspect.CacheAspect
             at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
             at java.security.AccessController.doPrivileged(Native Method)
             at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
             at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
             at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
             at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
             at java.lang.Class.forName0(Native Method)
             at java.lang.Class.forName(Class.java:247)
             at org.jboss.reflect.plugins.introspection.IntrospectionTypeInfoFactoryImpl.resolveComplexTypeInfo(IntrospectionTypeInfoFactoryImpl.java:419)
             at org.jboss.reflect.plugins.introspection.IntrospectionTypeInfoFactoryImpl.getTypeInfo(IntrospectionTypeInfoFactoryImpl.java:388)
             at org.jboss.reflect.plugins.introspection.IntrospectionTypeInfoFactory.getTypeInfo(IntrospectionTypeInfoFactory.java:54)
             at org.jboss.classadapter.plugins.BasicClassAdapterFactory.getClassAdapter(BasicClassAdapterFactory.java:61)
             at org.jboss.config.plugins.AbstractConfiguration.getBeanInfo(AbstractConfiguration.java:86)
             at org.jboss.kernel.plugins.config.AbstractKernelConfig.getBeanInfo(AbstractKernelConfig.java:80)
             at org.jboss.kernel.plugins.config.AbstractKernelConfigurator.getBeanInfo(AbstractKernelConfigurator.java:77)
             at org.jboss.beans.metadata.plugins.factory.GenericBeanFactory.createBean(GenericBeanFactory.java:209)
             at org.jboss.aop.microcontainer.beans.ClassLoaderAwareGenericBeanFactory.createBean(ClassLoaderAwareGenericBeanFactory.java:70)
             at org.jboss.aop.microcontainer.beans.GenericBeanAspectFactory.doCreate(GenericBeanAspectFactory.java:140)
             ... 90 more
            


            Thanks for the help!


            • 3. Re: Method Advised class on getInterceptorChainReadWriteLock
              mike03

              I've got a repro of our problem using the ClassicInstrumentor using a modified injboss example here: https://jira.jboss.org/jira/browse/JBAOP-706

              thanks for your help!

              • 4. Re: Method Advised class on getInterceptorChainReadWriteLock

                Just to circle back to the orginal problem with the new introspector as well it seems that this ReadLock - WriteLock deadlock issue only happens when using PER_INSTANCE aspects as well since it is occuring in the generated InstanceAdvisor for the advised class.

                What causes the version of the parent to the advisor to change, to cause the doRebuildForInstance method to be called?

                public void checkVersion()
                 {
                 if (needsRebuild || parent.version != GeneratedClassAdvisor.this.version)
                 {
                 doRebuildForInstance();
                 needsRebuild = false;
                 }
                 }
                


                -Pete

                • 5. Re: Method Advised class on getInterceptorChainReadWriteLock
                  stalep

                  hi, again. thanks for the provided reprodusable example attached to the jira issue, we'll take a look at it.
                  regarding the last question i dont have any immediate answer for you, and most of the aop team is on http://aosd.net/2009/ this week.
                  - but you'll get an answer when they have time to get online i guess :)

                  • 6. Re: Method Advised class on getInterceptorChainReadWriteLock
                    mike03

                    Regarding our issue with the ClassicInstrumentor and class loading:

                    We found a potential solution for this problem with a tweak to microcontainer/aop-mc-int/src/main/java/org/jboss/aop/microcontainer/beans/GenericBeanAspectFactory.java and updated https://jira.jboss.org/jira/browse/JBAOP-706 with a diff against a 2.0.4.GA checkout.

                    However, another issue came up when running our app with compile-time using the ClassicInstrumentor and optimized=true - we're getting the following ClassCastException during FieldWriteInvocation of one of our aspectized objects:

                    2009-03-05 12:21:57,221 DEBUG [com.company.common.item.aspect.TransactionalAspect] (main) Line: 582 main org.jboss.aop.ClassAdvisor cannot be cast to com.company.common.item.common.Item$ItemAdvisor
                    java.lang.ClassCastException: org.jboss.aop.ClassAdvisor cannot be cast to com.company.common.item.common.Item$ItemAdvisor
                     at com.company.common.item.common.Item.itemKey_w_$aop(Item.java)
                     at com.company.common.item.common.Item.setItemKey(Item.java:44)
                     at com.company.common.item.type.factory.ItemFactory.instantiate(ItemFactory.java:41)
                     at com.company.common.cache.impl.LRUCache.getItem(LRUCache.java:236)
                     at com.company.common.item.service.impl.ItemService.getItemFromItemKey(ItemService.java:679)
                     at com.company.common.item.common.Container.createChildRelationship(Container.java:88)
                     at com.company.common.item.common.ContainerItem.createChildRelationship(ContainerItem.java:72)
                     at com.company.common.item.service.impl.ItemService.com$company$common$item$service$impl$ItemService$_createItem$aop(ItemService.java:268)
                     at com.company.common.item.service.impl.ItemService.access$1(ItemService.java)
                     at com.company.common.item.service.impl.ItemService$JoinPoint__createItem_N_6418346684073351391.dispatch(ItemService$JoinPoint__createItem_N_6418346684073351391.java)
                     at com.company.common.item.aspect.TransactionalAspect.com$company$common$item$aspect$TransactionalAspect$trace$aop(TransactionalAspect.java:182)
                     at com.company.common.item.aspect.TransactionalAspect$JoinPoint_trace5044930021793041279.dispatch(TransactionalAspect$JoinPoint_trace5044930021793041279.java)
                     at com.company.rtl.perf.PerfMonitorAspect.trace(PerfMonitorAspect.java:56)
                     at com.company.common.item.aspect.TransactionalAspect$TransactionalAspectAdvisor.trace5044930021793041279(TransactionalAspect$TransactionalAspectAdvisor.java)
                     at com.company.common.item.aspect.TransactionalAspect.trace(TransactionalAspect.java)
                     at com.company.rtl.perf.PerfMonitorAspect.trace(PerfMonitorAspect.java:56)
                     at com.company.common.item.service.impl.ItemService$ItemServiceAdvisor._createItem_N_6418346684073351391(ItemService$ItemServiceAdvisor.java)
                     at com.company.common.item.service.impl.ItemService$ItemServiceInstanceAdvisor._createItem_N_6418346684073351391(ItemService$ItemServiceInstanceAdvisor.java)
                     at com.company.common.item.service.impl.ItemService._createItem(ItemService.java)
                     at com.company.common.item.service.impl.ItemService.com$company$common$item$service$impl$ItemService$createItem$aop(ItemService.java:235)
                     at com.company.common.item.service.impl.ItemService$JoinPoint_createItem6251048042645984084.dispatch(ItemService$JoinPoint_createItem6251048042645984084.java)
                     at com.company.common.item.aspect.TransactionalAspect.com$company$common$item$aspect$TransactionalAspect$trace$aop(TransactionalAspect.java:182)
                     at com.company.common.item.aspect.TransactionalAspect$JoinPoint_trace5044930021793041279.dispatch(TransactionalAspect$JoinPoint_trace5044930021793041279.java)
                     at com.company.rtl.perf.PerfMonitorAspect.trace(PerfMonitorAspect.java:56)
                     at com.company.common.item.aspect.TransactionalAspect$TransactionalAspectAdvisor.trace5044930021793041279(TransactionalAspect$TransactionalAspectAdvisor.java)
                     at com.company.common.item.aspect.TransactionalAspect.trace(TransactionalAspect.java)
                     at com.company.rtl.perf.PerfMonitorAspect.trace(PerfMonitorAspect.java:56)
                     at com.company.common.item.service.impl.ItemService$ItemServiceAdvisor.createItem6251048042645984084(ItemService$ItemServiceAdvisor.java)
                     at com.company.common.item.service.impl.ItemService$ItemServiceInstanceAdvisor.createItem6251048042645984084(ItemService$ItemServiceInstanceAdvisor.java)
                     at com.company.common.item.service.impl.ItemService.createItem(ItemService.java)
                     at com.company.system.executive.storage.impl.BasicExecutive.create(BasicExecutive.java:1635)
                     at com.company.system.executive.storage.impl.ContainerExecutive.createChild(ContainerExecutive.java:760)
                     at com.company.framework.impl.ItemComponentProvider.com$company$framework$impl$ItemComponentProvider$createComponentDataID$aop(ItemComponentProvider.java:334)
                     at com.company.framework.impl.ItemComponentProvider$JoinPoint_createComponentDataID_N_1216227476944167699.dispatch(ItemComponentProvider$JoinPoint_createComponentDataID_N_1216227476944167699.java)
                     at com.company.common.item.aspect.TransactionalAspect.com$company$common$item$aspect$TransactionalAspect$trace$aop(TransactionalAspect.java:195)
                     at com.company.common.item.aspect.TransactionalAspect$JoinPoint_trace5044930021793041279.dispatch(TransactionalAspect$JoinPoint_trace5044930021793041279.java)
                     at com.company.rtl.perf.PerfMonitorAspect.trace(PerfMonitorAspect.java:56)
                     at com.company.common.item.aspect.TransactionalAspect$TransactionalAspectAdvisor.trace5044930021793041279(TransactionalAspect$TransactionalAspectAdvisor.java)
                     at com.company.common.item.aspect.TransactionalAspect.trace(TransactionalAspect.java)
                     at com.company.framework.impl.ItemComponentProvider$ItemComponentProviderAdvisor.createComponentDataID_N_1216227476944167699(ItemComponentProvider$ItemComponentProviderAdvisor.java)
                     at com.company.framework.impl.ItemComponentProvider$ItemComponentProviderInstanceAdvisor.createComponentDataID_N_1216227476944167699(ItemComponentProvider$ItemComponentProviderInstanceAdvisor.java)
                     at com.company.framework.impl.ItemComponentProvider.createComponentDataID(ItemComponentProvider.java)
                     at com.company.framework.impl.ItemComponentProvider.upgradeComponentDataFromBootstrapComponentData(ItemComponentProvider.java:148)
                     at com.company.policy.impl.UpgradeComponentData.init(UpgradeComponentData.java:42)
                     at com.company.framework.init.impl.DefaultPhase.createComponentData(DefaultPhase.java:87)
                     at com.company.policy.impl.BootstrapPhase.createComponentData(BootstrapPhase.java:99)
                     at com.company.framework.impl.BaseProvider.initComponents(BaseProvider.java:450)
                     at com.company.framework.impl.BaseProvider.init(BaseProvider.java:99)
                     at com.company.framework.init.impl.DefaultPhase.init(DefaultPhase.java:46)
                     at com.company.policy.impl.BootstrapPhase.init(BootstrapPhase.java:51)
                     at com.company.framework.init.impl.DefaultPhaseProvider.initComponent(DefaultPhaseProvider.java:157)
                     at com.company.framework.impl.BaseProvider.initComponents(BaseProvider.java:458)
                     at com.company.framework.impl.BaseProvider.init(BaseProvider.java:99)
                     at com.company.framework.init.impl.DefaultPhaseProvider.initializePhases(DefaultPhaseProvider.java:80)
                     at com.company.framework.CoreSystem.init(CoreSystem.java:686)
                     at com.company.framework.CoreSystem.main(CoreSystem.java:645)
                     at com.company.framework.impl.CoreServlet.init(CoreServlet.java:83)
                     at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1048)
                     at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:950)
                     at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4122)
                     at org.apache.catalina.core.StandardContext.start(StandardContext.java:4421)
                     at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeployInternal(TomcatDeployment.java:312)
                     at org.jboss.web.tomcat.service.deployers.TomcatDeployment.performDeploy(TomcatDeployment.java:144)
                     at org.jboss.web.deployers.AbstractWarDeployment.start(AbstractWarDeployment.java:461)
                     at org.jboss.web.deployers.WebModule.startModule(WebModule.java:118)
                     at org.jboss.web.deployers.WebModule.start(WebModule.java:97)
                     at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)
                     at $Proxy36.start(Unknown Source)
                     at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:42)
                     at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:37)
                     at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
                     at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
                     at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
                     at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                     at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:286)
                     at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1598)
                     at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                     at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1062)
                     at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                     at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                     at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                     at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
                     at org.jboss.system.ServiceController.start(ServiceController.java:460)
                     at org.jboss.system.deployers.ServiceDeployer.start(ServiceDeployer.java:163)
                     at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:99)
                     at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:46)
                     at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
                     at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
                     at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
                     at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
                     at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
                     at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
                     at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1210)
                     at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
                     at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                     at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1598)
                     at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                     at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1062)
                     at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                     at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                     at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                     at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
                     at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:698)
                     at org.jboss.system.server.profileservice.ProfileServiceBootstrap.loadProfile(ProfileServiceBootstrap.java:304)
                     at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:205)
                     at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:405)
                     at org.jboss.Main.boot(Main.java:209)
                     at org.jboss.Main$1.run(Main.java:547)
                     at java.lang.Thread.run(Thread.java:619)
                    


                    We looked at the src generated by JAD for the aspectized object and this particular cast was not apparent. Re-aop-compiling with optimized="false" allowed us to progress past this issue without the ClassCastExceptions.

                    We'll report back on our luck with the ClassicInstrumentor workaround for the original GeneratedAdvisor locking issue.

                    thanks, Mike


                    • 7. Re: Method Advised class on getInterceptorChainReadWriteLock
                      kabirkhan

                      Thanks, the patch has been applied

                      • 8. Re: Method Advised class on getInterceptorChainReadWriteLock
                        mike03

                        Cool, glad that patch worked out.

                        Do you have any thoughts on the CCE we're getting with the ClassicInstrumentor and optimize=true? It seems like this is caused by generated code since were talking about an Item$ItemAdvisor but it wasn't apparent in the src decompilation of Item.class with JAD.

                        How do you guys approach debugging generated bytecode?

                        thanks for your help,
                        Mike

                        • 9. Re: Method Advised class on getInterceptorChainReadWriteLock
                          kabirkhan

                           

                          "Mike03" wrote:
                          Cool, glad that patch worked out.

                          Do you have any thoughts on the CCE we're getting with the ClassicInstrumentor and optimize=true? It seems like this is caused by generated code since were talking about an Item$ItemAdvisor but it wasn't apparent in the src decompilation of Item.class with JAD.

                          How do you guys approach debugging generated bytecode?

                          thanks for your help,
                          Mike


                          Am about to fly back to Europe, so I can't look now. There isn't really a way to debug generated bytecode apart from adding System.out.printlns when adding it in javassist

                          • 10. Re: Method Advised class on getInterceptorChainReadWriteLock
                            flavia.rainone

                            Pete and Mike,

                            There must be at least a third element involved in the deadlock you described. If there is one read/write lock and one of the threads acquire the read lock, while a second one waits on a write lock, the first thread will finish execution and the second will be resumed right away. A deadlock wouldn't occur also if one thread holds the write lock while the second one is waiting to retrieve the read lock.

                            I need to see the piece that is missing. Where are the threads that hold these locks, and why won't they let this lock go.

                            Is there a way I can reproduce this? If you could attach your app to a Jira issue, it would be much appreciated.

                            Cheers