4 Replies Latest reply on Jul 21, 2011 12:51 AM by jaikiran

    Deploying ejbs in a war with lots of jars slow: JBoss5DeploymentUnit.getResources()

    shahinaskari

      Can anyonone point me to some configuration I have screwed up.  I have a WAR with some annotated EJBs complied in its WEB-INF/classes directory.  This WAR also has a very large list of jars in its WEB-INF/lib directory.  Deploying the WAR takes a very long time on a clean JBoss AS 6.0.0.Final using the all server template.  I am aware of the reported issue about jars were being expanded, but I dont see that happening in my case, the tmp and work directories are not getting filled up from what I can see although again I may be missing something. 

       

      From what I can see, the main amount of time in the deployment is going into the stack trace below, which if I understood correctly from source, is attempting to determine if any of the classpath elements are children of the other.  From the code, I imagine there was an attempt to skip any jars that are in the list, but I dont know if I understood correctly. 

       

      I am trying to make use of the jboss-scanning.xml to make sure the jars can be ignored, but I don't know if that affects this stack, but here is the scanning file I am using:

       

      <scanning xmlns="urn:jboss:scanning:1.0">
        <path name="test.war/WEB-INF/classes" >
          <include name="com.testing.service" recurse="true"/>
        </path>
        <path name="release.war/WEB-INF/lib/mytest.jar">
          <include name="com.testing.service" recurse="true"/>
        </path>
      </scanning>
      
      

       

      This is the code where most of the time is being spent.  Each file is doing a linear search in the classpath. 

       

         public List<org.jboss.ejb3.vfs.spi.VirtualFile> getResources(org.jboss.ejb3.vfs.spi.VirtualFileFilter filter)
         {
            List<VirtualFile> classPath = unit.getClassPath();
            if(classPath == null || classPath.isEmpty())
               return Collections.emptyList();
      
      
            VisitorAttributes va = new VisitorAttributes();
            va.setLeavesOnly(true);
            SuffixesExcludeFilter noJars = new SuffixesExcludeFilter(Arrays.asList(".zip", ".ear", ".jar", ".rar", ".war", ".sar",".har", ".aop")); // TODO:  Where should these come from?
            va.setRecurseFilter(noJars);
            FilterVirtualFileVisitor visitor = new FilterVirtualFileVisitor(new VirtualFileFilterAdapter(filter), va);
      
      
            for(VirtualFile root : classPath)
            {
               try
               {
                  // The class path can contain subdirectories of other entries
                  // these must be ignored.
                  if(isChildOf(classPath, root)) 
                     continue;
      
      
                  if( root.isLeaf() == false ) 
                     root.visit(visitor);
               }
               catch (IOException e)
               {
                  throw new RuntimeException(e);
               }
            }
            final List<VirtualFile> matches = visitor.getMatched();
            final List<org.jboss.ejb3.vfs.spi.VirtualFile> wrappedMatches = new ArrayList<org.jboss.ejb3.vfs.spi.VirtualFile>(matches.size());
            for(VirtualFile match : matches)
            {
               wrappedMatches.add(new VirtualFileWrapper(match));
            }
      
      
            return wrappedMatches;
         }
      
      

       

      This is the thread dump (relevant thread) for most of the time I spent deploying:

       

      "Thread-2" prio=10 tid=0x000000004b28f800 nid=0x2b92 runnable [0x0000000041ce8000]
         java.lang.Thread.State: RUNNABLE
              at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
              at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
              at java.io.File.isDirectory(File.java:754)
              at org.jboss.vfs.spi.RootFileSystem.isDirectory(RootFileSystem.java:109)
              at org.jboss.vfs.VirtualFile.isDirectory(VirtualFile.java:222)
              at org.jboss.vfs.VirtualFile.getChildren(VirtualFile.java:312)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:249)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:266)
              at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.getResources(JBoss5DeploymentUnit.java:226)
              at org.jboss.ejb3.Ejb3Deployment.deployUrl(Ejb3Deployment.java:680)
              at org.jboss.ejb3.Ejb3Deployment.deploy(Ejb3Deployment.java:655)
              at org.jboss.ejb3.Ejb3Deployment.create(Ejb3Deployment.java:544)
              at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:177)
              at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:60)
              at org.jboss.deployers.vfs.spi.deployer.AbstractSimpleVFSRealDeployer.deploy(AbstractSimpleVFSRealDeployer.java:56)
              at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
              at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:55)
              at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:179)
              at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1832)
              at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1550)
              at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1491)
              at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:379)
              at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:2044)
              at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:1083)
              at org.jboss.dependency.plugins.AbstractController.executeOrIncrementStateDirectly(AbstractController.java:1322)
              at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1246)
              at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1139)
              at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:939)
              at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:654)
              at org.jboss.deployers.plugins.deployers.DeployersImpl.change(DeployersImpl.java:1983)
              at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:1076)
              at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:679)
              at org.jboss.system.server.profileservice.deployers.MainDeployerPlugin.process(MainDeployerPlugin.java:106)
              at org.jboss.profileservice.dependency.ProfileControllerContext$DelegateDeployer.process(ProfileControllerContext.java:143)
              at org.jboss.profileservice.dependency.ProfileDeployAction.deploy(ProfileDeployAction.java:151)
              at org.jboss.profileservice.dependency.ProfileDeployAction.installActionInternal(ProfileDeployAction.java:94)
              at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
              at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
              at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
              at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
              at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
              at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:379)
              at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:2044)
              at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:1083)
              at org.jboss.dependency.plugins.AbstractController.executeOrIncrementStateDirectly(AbstractController.java:1322)
              at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1246)
              at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1139)
              at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:939)
              at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:654)
              at org.jboss.profileservice.dependency.ProfileActivationWrapper$BasicProfileActivation.start(ProfileActivationWrapper.java:190)
              at org.jboss.profileservice.dependency.ProfileActivationWrapper.start(ProfileActivationWrapper.java:87)
              at org.jboss.profileservice.dependency.ProfileActivationService.activateProfile(ProfileActivationService.java:215)
              at org.jboss.profileservice.dependency.ProfileActivationService.activate(ProfileActivationService.java:159)
              at org.jboss.profileservice.bootstrap.AbstractProfileServiceBootstrap.activate(AbstractProfileServiceBootstrap.java:112)
              at org.jboss.profileservice.resolver.BasicResolverFactory$ProfileResolverFacade.deploy(BasicResolverFactory.java:87)
              at org.jboss.profileservice.bootstrap.AbstractProfileServiceBootstrap.start(AbstractProfileServiceBootstrap.java:91)
              at org.jboss.system.server.profileservice.bootstrap.BasicProfileServiceBootstrap.start(BasicProfileServiceBootstrap.java:132)
              at org.jboss.system.server.profileservice.bootstrap.BasicProfileServiceBootstrap.start(BasicProfileServiceBootstrap.java:56)
              at org.jboss.bootstrap.impl.base.server.AbstractServer.startBootstraps(AbstractServer.java:827)
              at org.jboss.bootstrap.impl.base.server.AbstractServer$StartServerTask.run(AbstractServer.java:417)
              - locked <0x00000000e1598cc8> (a org.jboss.bootstrap.impl.as.server.JBossASServerImpl)
              at java.lang.Thread.run(Thread.java:662)
      
      

       

      Thank you for your time.

        • 1. Re: Deploying ejbs in a war with lots of jars slow: JBoss5DeploymentUnit.getResources()
          alesj
          "Thread-2" prio=10 tid=0x000000004b28f800 nid=0x2b92 runnable [0x0000000041ce8000]
             java.lang.Thread.State: RUNNABLE
                  at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
                  at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:228)
                  at java.io.File.isDirectory(File.java:754)
                  at org.jboss.vfs.spi.RootFileSystem.isDirectory(RootFileSystem.java:109)
                  at org.jboss.vfs.VirtualFile.isDirectory(VirtualFile.java:222)
                  at org.jboss.vfs.VirtualFile.getChildren(VirtualFile.java:312)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:249)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:254)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.isChildOf(JBoss5DeploymentUnit.java:266)
                  at org.jboss.ejb3.deployers.JBoss5DeploymentUnit.getResources(JBoss5DeploymentUnit.java:226)
                  at org.jboss.ejb3.Ejb3Deployment.deployUrl(Ejb3Deployment.java:680)
                  at org.jboss.ejb3.Ejb3Deployment.deploy(Ejb3Deployment.java:655)
                  at org.jboss.ejb3.Ejb3Deployment.create(Ejb3Deployment.java:544)
                  at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:177)
                  at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:60)
          

          This is not part of the scanning, hence jboss-scanning.xml has no effect here.

          This looks like some EJB3 resource visiting logic is slow.

          • 2. Re: Deploying ejbs in a war with lots of jars slow: JBoss5DeploymentUnit.getResources()
            shahinaskari

            Thank you Ales.  Are there any deployment options to avoid this code or patches which improve the performance of the isChildOf or visit methods? 

             

            I have tried a dumb fix which improves the perofrmance dramatically by just ignoring classpath elements that are .jars, but without understanding well the responsibility and data structures, I'm sure my changes will not be safe.

            • 3. Re: Deploying ejbs in a war with lots of jars slow: JBoss5DeploymentUnit.getResources()
              alesj

              This is EJB3 code, and I don't know what exactly it does.

              Hence you'll probably need to wait for EJB3 guys to suggest some tweak.

              • 4. Re: Deploying ejbs in a war with lots of jars slow: JBoss5DeploymentUnit.getResources()
                jaikiran

                I'll take a look. I kind of remember seeing this long back. Although I don't know if we had a fix for that.