8 Replies Latest reply on Dec 22, 2009 7:54 AM by jaikiran pai

    Deployers Sorting

    Andrew Rubinger Master

      https://jira.jboss.org/jira/browse/JBDEPLOY-230

       

      I've added a stress test:

       

      http://anonsvn.jboss.org/repos/jbossas/projects/reloaded/trunk/vdf-bootstrap-minimal/src/test/java/DeployersOrderingStressUnitTest.java

      /**
          * Stress test to add the specified number of deployers to MC, and 
          * note the amount of time taken
          */
         void stressAddingDeployers(final int numDeployers) throws Throwable
         {
            // Get the MainDeployer
            final Kernel kernel = server.getKernel();
            MainDeployer mainDeployer = (MainDeployer) kernel.getController().getInstalledContext("MainDeployer").getTarget();
            TestCase.assertNotNull(MainDeployer.class.getName() + " instance was not installed into MC", mainDeployer);
      
            // Add Deployers
            final long start = System.currentTimeMillis();
            for (int i = 0; i < numDeployers; i++)
            {
               // Install a deployer
               final Deployer deployer = new AbstractDeployer()
               {
      
                  @Override
                  public void deploy(final DeploymentUnit unit) throws DeploymentException
                  {
                     //NOOP            
                  }
               };
               final String deployerName = deployer.getClass().getSimpleName() + numDeployers + "-" + i;
               final BeanMetaDataBuilder bmdb = BeanMetaDataBuilder
                     .createBuilder(deployerName, deployer.getClass().getName());
               kernel.getController().install(bmdb.getBeanMetaData(), deployer);
      
            }
            final long total = System.currentTimeMillis() - start;
            log.info("Adding " + numDeployers + " deployer(s) took " + total + "ms");
      
         }
      

       

      Output is:

      11:02:34,737 INFO  [MainDeployerViaBootstrapUnitTest] Adding 1 deployer(s) took 6ms
      11:02:35,010 INFO  [MainDeployerViaBootstrapUnitTest] Adding 10 deployer(s) took 83ms
      11:02:35,305 INFO  [MainDeployerViaBootstrapUnitTest] Adding 20 deployer(s) took 136ms
      11:02:35,918 INFO  [MainDeployerViaBootstrapUnitTest] Adding 30 deployer(s) took 459ms
      11:02:37,268 INFO  [MainDeployerViaBootstrapUnitTest] Adding 40 deployer(s) took 1088ms
      11:02:41,033 INFO  [MainDeployerViaBootstrapUnitTest] Adding 50 deployer(s) took 3516ms
      11:02:50,794 INFO  [MainDeployerViaBootstrapUnitTest] Adding 60 deployer(s) took 9616ms
      11:03:13,712 INFO  [MainDeployerViaBootstrapUnitTest] Adding 70 deployer(s) took 22740ms
      11:04:04,886 INFO  [MainDeployerViaBootstrapUnitTest] Adding 80 deployer(s) took 50835ms
      11:05:44,774 INFO  [MainDeployerViaBootstrapUnitTest] Adding 90 deployer(s) took 99716ms
      11:08:45,736 INFO  [MainDeployerViaBootstrapUnitTest] Adding 100 deployer(s) took 180761ms
      

       

      To run, check out the project and run under JDK6.  I've also provided a standalone (Main) test harness to ease the process of attaching a debugger.  The build will place all runtime libs under target/dependencies (useful for adding to the profiler ClassPath).

       

      S,

      ALR

        • 1. Re: Deployers Sorting
          Andrew Rubinger Master

          BTW, this test should be in the Deployers project, and be even more isolated (targeting DominoOrdering), but I've run out of time and could get this up quickest.  Will check back here after some personal travel.

           

          S,

          ALR

          • 2. Re: Deployers Sorting
            Bill Burke Master
            This project is totally broken from a clean checkout standpoint (try removing your repository to see what i mean).  Also, I can't load your stuff through Intellij as I get invalid poms from vfs.
            • 3. Re: Deployers Sorting
              Andrew Rubinger Master

              I know (the SNAPs required have since been purged from the repo); I'll ping back here when I get this stuff moved into a more reliable location (in jboss-deployers).

               

              S,

              ALR

              • 4. Re: Deployers Sorting
                Richard Opalka Master

                Here are real numbers for AS trunk (default configuration):

                 

                AS boot time: 43s:836ms (100%)

                Deployers sorting: 6s:524ms (14.77% of bootime - 136 deployers)

                 

                I commited DeployerFlowUnitTestCase.testRealWorldAS6DeployersScenario() to VDF test suite.

                • 5. Re: Deployers Sorting
                  Richard Opalka Master
                  11:02:34,737 INFO  [MainDeployerViaBootstrapUnitTest] Adding 1 deployer(s) took 6ms
                  11:02:35,010 INFO  [MainDeployerViaBootstrapUnitTest] Adding 10 deployer(s) took 83ms
                  11:02:35,305 INFO  [MainDeployerViaBootstrapUnitTest] Adding 20 deployer(s) took 136ms
                  11:02:35,918 INFO  [MainDeployerViaBootstrapUnitTest] Adding 30 deployer(s) took 459ms
                  11:02:37,268 INFO  [MainDeployerViaBootstrapUnitTest] Adding 40 deployer(s) took 1088ms
                  11:02:41,033 INFO  [MainDeployerViaBootstrapUnitTest] Adding 50 deployer(s) took 3516ms
                  11:02:50,794 INFO  [MainDeployerViaBootstrapUnitTest] Adding 60 deployer(s) took 9616ms
                  11:03:13,712 INFO  [MainDeployerViaBootstrapUnitTest] Adding 70 deployer(s) took 22740ms
                  11:04:04,886 INFO  [MainDeployerViaBootstrapUnitTest] Adding 80 deployer(s) took 50835ms
                  11:05:44,774 INFO  [MainDeployerViaBootstrapUnitTest] Adding 90 deployer(s) took 99716ms
                  11:08:45,736 INFO  [MainDeployerViaBootstrapUnitTest] Adding 100 deployer(s) took 180761ms

                   

                  This isn't that bad in AS trunk because deployers

                  are in maps associated with stages

                  and domino sorting is executed on particular stage

                  Below you can see how many deployers are in each stage.

                   

                   

                  PARSE (32 deployers)

                  org.jboss.deployers.vfs.deployer.kernel.BeanDeployer
                  org.jboss.deployers.vfs.plugins.annotations.ScanningMetaDataDeployer
                  org.jboss.deployers.vfs.plugins.dependency.AliasesParserDeployer
                  org.jboss.deployers.vfs.plugins.dependency.DependenciesParserDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployers.vfs.spi.deployer.SchemaResolverDeployer
                  org.jboss.deployment.AppParsingDeployer
                  org.jboss.deployment.JBossAppParsingDeployer
                  org.jboss.ejb3.deployers.AppClientParsingDeployer
                  org.jboss.ejb3.deployers.JBossClientParsingDeployer
                  org.jboss.ejb3.deployers.PersistenceUnitParsingDeployer
                  org.jboss.jpa.deployers.PersistenceParsingDeployer
                  org.jboss.resource.deployers.ManagedConnectionFactoryParserDeployer
                  org.jboss.resource.deployers.RARParserDeployer
                  org.jboss.security.deployers.AclConfigParsingDeployer
                  org.jboss.security.deployers.XacmlConfigParsingDeployer
                  org.jboss.system.deployers.SARDeployer
                  org.jboss.varia.deployment.LegacyBeanShellDeployer
                  org.jboss.webservices.integration.deployers.WSDescriptorDeployer
                  org.jboss.aop.asintegration.jboss5.AOPAnnotationMetaDataParserDeployer
                  org.jboss.deployment.EARContentsDeployer
                  org.jboss.deployment.WebAppParsingDeployer
                  org.jboss.deployment.WebAppFragmentParsingDeployer
                  org.jboss.deployment.TldParsingDeployer
                  org.jboss.deployment.JBossWebAppParsingDeployer
                  org.jboss.deployment.EjbParsingDeployer
                  org.jboss.deployment.JBossEjbParsingDeployer

                  POST_PARSE (13 deployers)

                  org.jboss.deployers.vfs.plugins.dependency.DependenciesMetaDataDeployer
                  org.jboss.deployers.vfs.plugins.dependency.DeploymentAliasesDeployer
                  org.jboss.deployers.vfs.plugins.dependency.DeploymentDependencyDeployer
                  org.jboss.deployment.EarClassLoaderDeployer
                  org.jboss.deployment.EjbClassLoaderDeployer
                  org.jboss.deployment.LegacyWebXmlLessDeployer
                  org.jboss.resource.deployers.ManagedConnectionFactoryClassLoaderDeployer
                  org.jboss.system.deployers.ServiceClassLoaderDeployer
                  org.jboss.web.tomcat.service.deployers.ClusteringDefaultsDeployer
                  org.jboss.web.tomcat.service.deployers.WarClassLoaderDeployer
                  org.jboss.weld.integration.deployer.metadata.PostJBossMetadataDeployer
                  org.jboss.weld.integration.deployer.metadata.PostJBossWebMetadataDeployer
                  org.jboss.weld.integration.deployer.metadata.WeldFilesDeployer

                  PRE_DESCRIBE (1 deployer)

                  org.jboss.deployers.plugins.classloading.ClassLoadingDefaultDeployer

                  DESCRIBE (5 deployers)

                  org.jboss.deployers.vfs.plugins.classloader.InMemoryClassesDeployer
                  org.jboss.seam.integration.microcontainer.deployers.SeamWebUrlIntegrationDeployer
                  org.jboss.weld.integration.deployer.cl.WeldFacesIntegrationDeployer
                  org.jboss.deployers.vfs.plugins.classloader.VFSClassLoaderClassPathDeployer
                  org.jboss.deployers.vfs.plugins.classloader.VFSClassLoaderDescribeDeployer

                  CLASSLOADER (2 deployers)

                  org.jboss.deployers.plugins.classloading.AbstractLevelClassLoaderSystemDeployer
                  org.jboss.aop.asintegration.jboss5.AOPClassLoaderDeployer

                  POST_CLASSLOADER (21 deployers)

                  org.jboss.aop.asintegration.jboss5.AOPDeploymentAopMetaDataDeployer
                  org.jboss.aop.asintegration.jboss5.BeansDeploymentAopMetaDataDeployer
                  org.jboss.deployment.EarLibExcludeDeployer
                  org.jboss.deployers.vfs.plugins.annotations.FilteredAnnotationEnvironmentDeployer
                  org.jboss.deployment.EarSecurityDeployer
                  org.jboss.deployment.OptAnnotationMetaDataDeployer
                  org.jboss.ejb.deployers.MergedJBossMetaDataDeployer
                  org.jboss.ejb3.deployers.MergedJBossClientMetaDataDeployer
                  org.jboss.ha.framework.server.deployers.Ejb2HAPartitionDependencyDeployer
                  org.jboss.ha.framework.server.deployers.Ejb3HAPartitionDependencyDeployer
                  org.jboss.ejb.deployers.StandardJBossMetaDataDeployer
                  org.jboss.ejb3.deployers.Ejb3MetadataProcessingDeployer
                  org.jboss.ejb3.deployers.EjbMetadataJndiPolicyDecoratorDeployer
                  org.jboss.weld.integration.deployer.metadata.PostWebMetadataDeployer
                  org.jboss.weld.integration.deployer.metadata.WeldEjbInterceptorMetadataDeployer
                  org.jboss.ejb.deployers.EjbSecurityDeployer
                  org.jboss.web.deployers.WarAnnotationMetaDataDeployer
                  org.jboss.web.deployers.MergedJBossWebMetaDataDeployer
                  org.jboss.deployment.MappedReferenceMetaDataResolverDeployer
                  org.jboss.web.deployers.ServletContainerInitializerDeployer
                  org.jboss.web.deployers.WarSecurityDeployer

                  PRE_REAL (7 deployers)

                  org.jboss.beanvalidation.deployers.ValidatorFactoryDeployer
                  org.jboss.ejb3.deployers.Ejb3DependenciesDeployer
                  org.jboss.system.server.profileservice.persistence.deployer.ProfileServicePersistenceDeployer
                  org.jboss.weld.integration.deployer.env.EjbServicesDeployer
                  org.jboss.weld.integration.deployer.env.WeldDiscoveryDeployer
                  org.jboss.weld.integration.deployer.env.FlatDeploymentDeployer
                  org.jboss.weld.integration.deployer.env.WeldBootstrapDeployer

                  REAL (55 deployers)

                  org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataFactoryDeployer
                  org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataFactoryDeployer
                  org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataFactoryDeployer
                  org.jboss.ejb.deployers.CreateDestinationDeployer
                  org.jboss.ejb.deployers.EjbDeployer
                  org.jboss.ejb3.deployers.Ejb3ClientDeployer
                  org.jboss.ejb3.deployers.Ejb3Deployer
                  org.jboss.deployers.vfs.deployer.kernel.AliasDeploymentDeployer
                  org.jboss.ejb3.endpoint.deployers.EJB3EndpointDeployer
                  org.jboss.ejb3.metrics.deployer.Ejb3MetricsDeployer
                  org.jboss.hibernate.deployers.HibernateDeployer
                  org.jboss.jpa.deployers.PersistenceDeployer
                  org.jboss.jpa.deployers.PersistenceUnitDeployer
                  org.jboss.deployers.vfs.deployer.kernel.KernelDeploymentDeployer
                  org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer
                  org.jboss.deployers.vfs.deployer.kernel.DeploymentAliasMetaDataDeployer
                  org.jboss.management.j2ee.deployers.EarModuleJSR77Deployer
                  org.jboss.management.j2ee.deployers.EjbModuleJSR77Deployer
                  org.jboss.management.j2ee.deployers.JCAResourceJSR77Deployer
                  org.jboss.management.j2ee.deployers.RarModuleJSR77Deployer
                  org.jboss.resource.deployers.ManagedConnectionFactoryDeployer
                  org.jboss.management.j2ee.deployers.ServiceModuleJSR77Deployer
                  org.jboss.resource.deployers.RARDeployer
                  org.jboss.varia.deployment.LegacyBeanShellScriptDeployer
                  org.jboss.webservices.integration.deployers.WSEJBAdapterDeployer
                  org.jboss.webservices.integration.deployers.WSTypeDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.webservices.integration.deployers.WSDeploymentAspectDeployer
                  org.jboss.management.j2ee.deployers.WebModuleJSR77Deployer
                  org.jboss.web.tomcat.service.deployers.TomcatDeployer
                  org.jboss.system.deployers.ServiceDeploymentDeployer
                  org.jboss.management.j2ee.deployers.JMSResourceJSR77Deployer
                  org.jboss.system.deployers.ServiceDeployer
                  • 6. Re: Deployers Sorting
                    Andrew Rubinger Master

                    bill.burke@jboss.com wrote:

                     

                    This project is totally broken from a clean checkout standpoint (try removing your repository to see what i mean). 


                    Committed a fix so it doesn't depend on the reloaded/build.pom.  Checkout *just* vdf-minimal for it to work.

                     

                    bill.burke@jboss.com wrote:

                     

                    Also, I can't load your stuff through Intellij as I get invalid poms from vfs.

                     

                    You'd have to ask Ales; I'm on Eclipse and haven't touched VFS.


                    S,

                    ALR

                    • 7. Re: Deployers Sorting
                      Andrew Rubinger Master

                      richard.opalka@jboss.com wrote:


                      Below you can see how many deployers are in each stage.

                       

                      Nice analysis; curious as to how you got the full list of each deployer and associated stage (some simple util I'm not aware of)?

                       

                      richard.opalka@jboss.com wrote:


                      This isn't that bad in AS trunk because deployers

                      are in maps associated with stages

                      and domino sorting is executed on particular stage

                       

                      The test targets one simple thing and that only.  A problem in this area doesn't preclude it in others (ie. absence of in/out chain).

                       

                      S,

                      ALR

                      • 8. Re: Deployers Sorting
                        jaikiran pai Master

                        ALRubinger wrote:

                         

                        richard.opalka@jboss.com wrote:


                        Below you can see how many deployers are in each stage.

                         

                        Nice analysis; curious as to how you got the full list of each deployer and associated stage (some simple util I'm not aware of)?

                         


                        Edit the JBOSS_HOME/server/< servername>/conf/bootstrap/deployers.xml to uncomment the following:

                         

                           <bean name="Deployers" class="org.jboss.deployers.plugins.deployers.DeployersImpl">
                           <!-- Uncomment the following @org.jboss.aop.microcontainer.aspects.jmx.JMX to enable registering
                            MC beans as MBeans. These MBeans provide debug level information about which deployers were selected for
                            processing the deployment and what attachments were added to the deployment. More details http://www.jboss.org/index.html?module=bb&op=viewtopic&t=155074&start=10#4233874
                            -->
                            <!--
                              <annotation>@org.jboss.aop.microcontainer.aspects.jmx.JMX(name="jboss.deployment:service=DeployersImpl", exposedInterface=org.jboss.deployers.plugins.deployers.DeployersImplMBean.class, registerDirectly=true)</annotation>
                              -->
                        

                         

                        Once the server boots, from the jmx-console access the jboss.deployment:service=DeployersImpl which has the "listDeployers" MBean operation.