11 Replies Latest reply on Apr 24, 2009 7:24 AM by Brian Stansberry

    HDScanner scanPeriod

    Pete Muir Master

      Ales has recently updated the hot deployment scanner so that it is also responsible for updating the temp copy of existing apps - people use this for being able to copy a new JSF etc. page and have the app show it without restarting the app.

      The Hot Deployment scanner scan interval is currently 5000ms, which is really too long for zero-turnaround development - the period should be no longer than the time to switch from an IDE to a browser...

      Any reason not reduce this by a factor of 10 to 500ms?

        • 1. Re: HDScanner scanPeriod
          Ales Justin Master

           

          "pete.muir@jboss.org" wrote:
          Ales has recently updated the hot deployment scanner so that it is also responsible for updating the temp copy of existing apps - people use this for being able to copy a new JSF etc. page and have the app show it without restarting the app.

          It's actually not hd scanner that has been updated,
          it's this piece of configuration that does it (in profile.xml):
           <!-- The structure modification cache and checker -->
           <bean name="StructureModCache" class="org.jboss.deployers.vfs.spi.structure.modified.DefaultStructureCache">
           <destroy method="flush"/>
           </bean>
          
           <!-- This just checks metadata locations -->
           <bean name="MetaDataStructureModificationChecker" class="org.jboss.deployers.vfs.spi.structure.modified.MetaDataStructureModificationChecker">
           <constructor>
           <parameter><inject bean="MainDeployer" /></parameter>
           </constructor>
           <property name="cache"><inject bean="StructureModCache" /></property>
           <property name="filter"><bean class="org.jboss.system.server.profile.basic.XmlIncludeVirtualFileFilter" /></property>
           </bean>
          
           <!-- Synchs modifed files -->
           <bean name="SynchAdapter" class="org.jboss.deployers.vfs.spi.structure.modified.OverrideSynchAdapter"/>
          
           <!-- We just visit wars, but exclude classes, lib, dev dirs -->
           <bean name="WebVisitorAttributes" class="org.jboss.system.server.profile.basic.IncludeExcludeVisitorAttributes">
           <constructor>
           <parameter>
           <set>
           <value>.war</value>
           </set>
           </parameter>
           <parameter>
           <set>
           <value>WEB-INF/classes</value>
           <value>WEB-INF/lib</value>
           <value>WEB-INF/dev</value>
           </set>
           </parameter>
           </constructor>
           </bean>
          
           <bean name="StructureModificationChecker" class="org.jboss.deployers.vfs.spi.structure.modified.SynchWrapperModificationChecker">
           <constructor>
           <parameter><inject bean="MetaDataStructureModificationChecker"/></parameter>
           <parameter><inject bean="SynchAdapter"/></parameter>
           </constructor>
           <property name="originalAttributes"><inject bean="WebVisitorAttributes"/></property>
           <property name="tempAttributes"><inject bean="WebVisitorAttributes"/></property>
           </bean>
          


          HDScanner then asks Profiles for modified deployments,
          which eventually ask MDSMChecker for real modifications.
          If a deployment is not modified and is temped,
          we go and check for any updates with SynchWSMChecker.
          If it's modified then there is no need to check since it's gonna be updated anyway.

          • 2. Re: HDScanner scanPeriod
            jaikiran pai Master

             

            HDScanner then asks Profiles for modified deployments,
            which eventually ask MDSMChecker for real modifications.
            If a deployment is not modified and is temped ....


            I am not sure how this will work for new pages (.jsp or any JSF pages) that are added to any existing temped deployment [1]. As Pete mentions:

            people use this for being able to copy a new JSF etc. page and have the app show it without restarting the app.

            From my understanding of HDScanner (and i might be wrong), it monitors the last modified time of the deployment descriptor and nothing more than that. So how is it going to trigger the OverrideSynchAdapter to copy a new page (jsp/jsf) to a temped deployment, when a new page has been added to the original deployment?

            [1] Just to be sure - Based on the discussion in the other thread in this forum, starting JBossAS-5.1.0CR1, we will be having temped deployments for all deployments, irrespective of whether the original deployment is archived or exploded. Am i correct in understanding this?






            • 3. Re: HDScanner scanPeriod
              Ales Justin Master

               

              "jaikiran" wrote:

              From my understanding of HDScanner (and i might be wrong), it monitors the last modified time of the deployment descriptor and nothing more than that.

              It depends on the type of StructureModificationChecker.
              Currently we have an impl that watches only metadata locations.

              "jaikiran" wrote:

              So how is it going to trigger the OverrideSynchAdapter to copy a new page (jsp/jsf) to a temped deployment, when a new page has been added to the original deployment?

              That's the tricky new part. ;-)

              From SynchWrapperModificationChecker
               protected boolean hasStructureBeenModifed(VirtualFile root, VFSDeploymentContext deploymentContext) throws IOException
               {
               boolean modified = delegate.hasStructureBeenModifed(root, deploymentContext);
               // it was not modifed & we're actually temped
               if (modified == false && root != deploymentContext.getRoot())
               {
               // check for update or delete
               UpdateDeleteVisitor udVisitor = new UpdateDeleteVisitor(tempAttributes, getCache(), synchAdapter, root);
               VirtualFile tempRoot = deploymentContext.getRoot();
               tempRoot.visit(udVisitor);
               // check for addition
               AddVisitor addVisitor = new AddVisitor(originalAttributes, getCache(), synchAdapter, tempRoot, root.getPathName().length());
               root.visit(addVisitor);
               }
               return modified;
               }
              


              UpdateDeleteVisitor works on tempRoot, as that should be transparent to the user
              (the user should never (have to) modify temp),
              traversing over temps and asking matching original if it changed or if it still exists.

              AddVisitor traverses over originals, checking if matching temp exists,
              if not, new file has been added --> solving your problem. ;-)

              • 4. Re: HDScanner scanPeriod
                jaikiran pai Master

                 

                "alesj" wrote:

                 protected boolean hasStructureBeenModifed(VirtualFile root, VFSDeploymentContext deploymentContext) throws IOException
                 {
                ...
                 // check for addition
                 AddVisitor addVisitor = new AddVisitor(originalAttributes, getCache(), synchAdapter, tempRoot, root.getPathName().length());
                 root.visit(addVisitor);
                 }
                 return modified;
                 }
                


                AddVisitor traverses over originals, checking if matching temp exists,
                if not, new file has been added --> solving your problem. ;-)


                Assuming the HDScanner is set to run every 5 seconds, wouldn't this scanning for new files within the original deployment lead to time delays? From what i see, the AddVisitor looks for new files in the original deployments and these new files can be nested too. Which effectively means scanning the entire deployment every 5 seconds. I haven't given this change a try against the AS, so don't know whether i am talking sense :)



                • 5. Re: HDScanner scanPeriod
                  jaikiran pai Master

                   

                  "pete.muir@jboss.org" wrote:


                  Any reason not reduce this by a factor of 10 to 500ms?


                  I only side-effect that i can think of is the HDScanner picking up incompletely "copied" deployments. Assuming that a existing deployment is being overwritten or a new deployment is being copied to the deploy folder and the copy is still in progress, then the HDSCanner will complain about invalid deployments. Till scenario exists even now (when the scanning time is set to 5 seconds), but reducing the scanning time to a very low value is going increase the likelihood of such cases.



                  • 6. Re: HDScanner scanPeriod
                    Ales Justin Master

                     

                    "jaikiran" wrote:

                    Assuming the HDScanner is set to run every 5 seconds, wouldn't this scanning for new files within the original deployment lead to time delays?

                    How else are you gonna know if something changed?

                    You could put something similar into every sub-system.
                    Which automatically leads to duplication.

                    This mechanism is generic, if adding a few sec to already asynch
                    mechanism is all we do, then I don't see a problem.

                    There is always gonna be a cost, if you wanna have the best of both worlds; temping & updating.


                    "jaikiran" wrote:

                    From what i see, the AddVisitor looks for new files in the original deployments and these new files can be nested too. Which effectively means scanning the entire deployment every 5 seconds.

                    We don't scan entire deployment(s).
                    See original/tempAttributes property, which limits the scope of scanning.
                    Currently we only scan for possible changes in web resources.
                    (as far as I could gather the info where users put their .jsp, .xhtml, ...
                    as it's been a while when I did my last .jsp :-))

                    • 7. Re: HDScanner scanPeriod
                      jaikiran pai Master

                       

                      "alesj" wrote:


                      We don't scan entire deployment(s).
                      See original/tempAttributes property, which limits the scope of scanning.
                      Currently we only scan for possible changes in web resources.
                      (as far as I could gather the info where users put their .jsp, .xhtml, ...
                      as it's been a while when I did my last .jsp :-))


                      Hmmm, that would mean the user would have add more to the configuration files to enable other folders/files within their deployment to be scanned for (hot) additions. (Not complaining, just thinking out loud).




                      • 8. Re: HDScanner scanPeriod
                        Ales Justin Master

                        Exactly.

                        • 9. Re: HDScanner scanPeriod
                          Bob McWhirter Novice

                          Is there some happy way for my rails.deployer jar to inject additional paths/files to scan, or add a visitor that'd search search rails deployments?

                          Right now, changes to config/**.yml doesn't seem to cause my rails apps to redeploy, but config/ is a metadata path according to the StructureMetaData in these.

                          A lot of rails reloading is baked right into the framework, so my redeployments are seldom. But there's a handful of files (config/** is the biggie) that do require a full redeploy to be picked up.

                          -Bob

                          • 10. Re: HDScanner scanPeriod
                            Ales Justin Master

                             

                            "bob.mcwhirter" wrote:

                            Right now, changes to config/**.yml doesn't seem to cause my rails apps to redeploy, but config/ is a metadata path according to the StructureMetaData in these.

                            Currently we only monitor *.xml files.
                            See the filter property with StructureModificationChecker in profile.xml.

                            • 11. Re: HDScanner scanPeriod
                              Brian Stansberry Master

                              Note that the farm/ folder is completely scanned every HDScanner run. This is because the contents of a ClusteredDeploymentRepository (e.g. farm/) need to be kept in sync regardless of whether the affected file triggers a redeploy.