HDScanner scanPeriod

[pmuir]

Ales has recently updated the hot deployment scanner so that it is also responsible for updating the temp copy of existing apps - people use this for being able to copy a new JSF etc. page and have the app show it without restarting the app.

The Hot Deployment scanner scan interval is currently 5000ms, which is really too long for zero-turnaround development - the period should be no longer than the time to switch from an IDE to a browser...

Any reason not reduce this by a factor of 10 to 500ms?

[alesj]

Ales has recently updated the hot deployment scanner so that it is also responsible for updating the temp copy of existing apps - people use this for being able to copy a new JSF etc. page and have the app show it without restarting the app.

It's actually not hd scanner that has been updated,
it's this piece of configuration that does it (in profile.xml):

 <!-- The structure modification cache and checker -->
 <bean name="StructureModCache" class="org.jboss.deployers.vfs.spi.structure.modified.DefaultStructureCache">
 <destroy method="flush"/>
 </bean>

 <!-- This just checks metadata locations -->
 <bean name="MetaDataStructureModificationChecker" class="org.jboss.deployers.vfs.spi.structure.modified.MetaDataStructureModificationChecker">
 <constructor>
 <parameter><inject bean="MainDeployer" /></parameter>
 </constructor>
 <property name="cache"><inject bean="StructureModCache" /></property>
 <property name="filter"><bean class="org.jboss.system.server.profile.basic.XmlIncludeVirtualFileFilter" /></property>
 </bean>

 <!-- Synchs modifed files -->
 <bean name="SynchAdapter" class="org.jboss.deployers.vfs.spi.structure.modified.OverrideSynchAdapter"/>

 <!-- We just visit wars, but exclude classes, lib, dev dirs -->
 <bean name="WebVisitorAttributes" class="org.jboss.system.server.profile.basic.IncludeExcludeVisitorAttributes">
 <constructor>
 <parameter>
 <set>
 <value>.war</value>
 </set>
 </parameter>
 <parameter>
 <set>
 <value>WEB-INF/classes</value>
 <value>WEB-INF/lib</value>
 <value>WEB-INF/dev</value>
 </set>
 </parameter>
 </constructor>
 </bean>

 <bean name="StructureModificationChecker" class="org.jboss.deployers.vfs.spi.structure.modified.SynchWrapperModificationChecker">
 <constructor>
 <parameter><inject bean="MetaDataStructureModificationChecker"/></parameter>
 <parameter><inject bean="SynchAdapter"/></parameter>
 </constructor>
 <property name="originalAttributes"><inject bean="WebVisitorAttributes"/></property>
 <property name="tempAttributes"><inject bean="WebVisitorAttributes"/></property>
 </bean>

HDScanner then asks Profiles for modified deployments,
which eventually ask MDSMChecker for real modifications.
If a deployment is not modified and is temped,
we go and check for any updates with SynchWSMChecker.
If it's modified then there is no need to check since it's gonna be updated anyway.

[jaikiran]

HDScanner then asks Profiles for modified deployments,
which eventually ask MDSMChecker for real modifications.
If a deployment is not modified and is temped ....

I am not sure how this will work for new pages (.jsp or any JSF pages) that are added to any existing temped deployment [1]. As Pete mentions:

people use this for being able to copy a new JSF etc. page and have the app show it without restarting the app.

From my understanding of HDScanner (and i might be wrong), it monitors the last modified time of the deployment descriptor and nothing more than that. So how is it going to trigger the OverrideSynchAdapter to copy a new page (jsp/jsf) to a temped deployment, when a new page has been added to the original deployment?

[1] Just to be sure - Based on the discussion in the other thread in this forum, starting JBossAS-5.1.0CR1, we will be having temped deployments for all deployments, irrespective of whether the original deployment is archived or exploded. Am i correct in understanding this?

[alesj]

"jaikiran" wrote:

From my understanding of HDScanner (and i might be wrong), it monitors the last modified time of the deployment descriptor and nothing more than that.

It depends on the type of StructureModificationChecker.
Currently we have an impl that watches only metadata locations.

"jaikiran" wrote:

So how is it going to trigger the OverrideSynchAdapter to copy a new page (jsp/jsf) to a temped deployment, when a new page has been added to the original deployment?

That's the tricky new part. ;-)

From SynchWrapperModificationChecker

 protected boolean hasStructureBeenModifed(VirtualFile root, VFSDeploymentContext deploymentContext) throws IOException
 {
 boolean modified = delegate.hasStructureBeenModifed(root, deploymentContext);
 // it was not modifed & we're actually temped
 if (modified == false && root != deploymentContext.getRoot())
 {
 // check for update or delete
 UpdateDeleteVisitor udVisitor = new UpdateDeleteVisitor(tempAttributes, getCache(), synchAdapter, root);
 VirtualFile tempRoot = deploymentContext.getRoot();
 tempRoot.visit(udVisitor);
 // check for addition
 AddVisitor addVisitor = new AddVisitor(originalAttributes, getCache(), synchAdapter, tempRoot, root.getPathName().length());
 root.visit(addVisitor);
 }
 return modified;
 }

UpdateDeleteVisitor works on tempRoot, as that should be transparent to the user
(the user should never (have to) modify temp),
traversing over temps and asking matching original if it changed or if it still exists.

AddVisitor traverses over originals, checking if matching temp exists,
if not, new file has been added --> solving your problem. ;-)

[jaikiran]

"alesj" wrote:

 protected boolean hasStructureBeenModifed(VirtualFile root, VFSDeploymentContext deploymentContext) throws IOException
 {
...
 // check for addition
 AddVisitor addVisitor = new AddVisitor(originalAttributes, getCache(), synchAdapter, tempRoot, root.getPathName().length());
 root.visit(addVisitor);
 }
 return modified;
 }

AddVisitor traverses over originals, checking if matching temp exists,
if not, new file has been added --> solving your problem. ;-)

Assuming the HDScanner is set to run every 5 seconds, wouldn't this scanning for new files within the original deployment lead to time delays? From what i see, the AddVisitor looks for new files in the original deployments and these new files can be nested too. Which effectively means scanning the entire deployment every 5 seconds. I haven't given this change a try against the AS, so don't know whether i am talking sense :)

[jaikiran]

Any reason not reduce this by a factor of 10 to 500ms?

I only side-effect that i can think of is the HDScanner picking up incompletely "copied" deployments. Assuming that a existing deployment is being overwritten or a new deployment is being copied to the deploy folder and the copy is still in progress, then the HDSCanner will complain about invalid deployments. Till scenario exists even now (when the scanning time is set to 5 seconds), but reducing the scanning time to a very low value is going increase the likelihood of such cases.

[alesj]

"jaikiran" wrote:

Assuming the HDScanner is set to run every 5 seconds, wouldn't this scanning for new files within the original deployment lead to time delays?

How else are you gonna know if something changed?

You could put something similar into every sub-system.
Which automatically leads to duplication.

This mechanism is generic, if adding a few sec to already asynch
mechanism is all we do, then I don't see a problem.

There is always gonna be a cost, if you wanna have the best of both worlds; temping & updating.

"jaikiran" wrote:

From what i see, the AddVisitor looks for new files in the original deployments and these new files can be nested too. Which effectively means scanning the entire deployment every 5 seconds.

We don't scan entire deployment(s).
See original/tempAttributes property, which limits the scope of scanning.
Currently we only scan for possible changes in web resources.
(as far as I could gather the info where users put their .jsp, .xhtml, ...
as it's been a while when I did my last .jsp :-))

[jaikiran]

"alesj" wrote:


We don't scan entire deployment(s).
See original/tempAttributes property, which limits the scope of scanning.
Currently we only scan for possible changes in web resources.
(as far as I could gather the info where users put their .jsp, .xhtml, ...
as it's been a while when I did my last .jsp :-))

Hmmm, that would mean the user would have add more to the configuration files to enable other folders/files within their deployment to be scanned for (hot) additions. (Not complaining, just thinking out loud).

[alesj]

Exactly.

[bob.mcwhirter]

Is there some happy way for my rails.deployer jar to inject additional paths/files to scan, or add a visitor that'd search search rails deployments?

Right now, changes to config/**.yml doesn't seem to cause my rails apps to redeploy, but config/ is a metadata path according to the StructureMetaData in these.

A lot of rails reloading is baked right into the framework, so my redeployments are seldom. But there's a handful of files (config/** is the biggie) that do require a full redeploy to be picked up.

-Bob

[alesj]

"bob.mcwhirter" wrote:

Right now, changes to config/**.yml doesn't seem to cause my rails apps to redeploy, but config/ is a metadata path according to the StructureMetaData in these.

Currently we only monitor *.xml files.
See the filter property with StructureModificationChecker in profile.xml.

[brian.stansberry]

Note that the farm/ folder is completely scanned every HDScanner run. This is because the contents of a ClusteredDeploymentRepository (e.g. farm/) need to be kept in sync regardless of whether the affected file triggers a redeploy.