AS-5 Files to watch for application redeployment

[jaikiran]

Background: http://lists.jboss.org/pipermail/jboss-development/2009-June/014478.html and other threads where AS-5 re-deployments have been discussed

As discussed in the mailing list, unlike AS-4.x, in AS-5, *by default* we watch for all xml files in a metadata location, to decide whether a application redeployment(hotdpeloyment) is necessary. In AS-4.x, we used to watch only for known files application.xml, web.xml (and other known top level descriptor). The nice thing about that feature was we let the end users decide when they want to trigger a redeployment instead of guessing it ourselves. So in AS-4.x, an end user could do 'n' number of changes to various xml files (or other files) in the metadata locations of the application (or any other location) and then at some point "decide" to trigger the redeployment by touching the application.xml or the appropriate top level descriptor. There were no surprises here.

However in AS-5, this has changed. It's now the application server which does redeployments more aggressive by guessing that the application needs redeployment. So if the user has to do 'n' number of changes to various xml files in the metadata locations, then the AS could end up triggering those many redeployments, effectively taking away the control that end users had over redeployments in AS-4.x. Users are finding this default behaviour difficult to manage.

In AS-5, the configuration/filter which decides which files to consider for redeployments is managed by this (in profile.xml):

<bean name="MetaDataStructureModificationChecker" class="org.jboss.deployers.vfs.spi.structure.modified.MetaDataStructureModificationChecker">
 <constructor>
 <parameter><inject bean="MainDeployer" /></parameter>
 </constructor>
 <property name="cache"><inject bean="StructureModCache" /></property>
 <property name="filter"><bean class="org.jboss.system.server.profile.basic.XmlIncludeVirtualFileFilter" /></property>
 </bean>

So *by default* in AS-5, the filter is meant to check for all xml files. It can be argued that the end user can configure/tweak this filter as per his wish to take care of his re-deployment needs. But do we really want the normal end users (most of them) to be changing this? Or should be *by default* manage it such that we have a similar behaviour as AS-4.x may be by using the PatternIncludeVirtualFileFilter

<bean name="MetaDataStructureModificationChecker" class="org.jboss.deployers.vfs.spi.structure.modified.MetaDataStructureModificationChecker">
 <constructor>
 <parameter><inject bean="MainDeployer" /></parameter>
 </constructor>
 <property name="cache"><inject bean="StructureModCache" /></property>

 <property name="filter">
 <!-- A pattern matching filter. We can configure this to our custom pattern.
 Here we configure it to include only application.xml and web.xml
 for modification check (and any subsequent redeployments) -->
 <bean class="org.jboss.system.server.profile.basic.PatternIncludeVirtualFileFilter">
 <constructor>
 <!-- Remember, the parameter is a regex so use the correct syntax
 as below -->
 <parameter class="java.lang.String">application.xml|web.xml</parameter>
 </constructor>
 </bean>
 </property>

 </bean>

The PatternIncludeVirtualFileFilter approach is just an example. The real question is, *by default* should we be really doing the guess work for redeployments or like AS-4.x only watch for known top level files and give the end users more freedom to trigger an redeployment? Furthermore, if some advanced user wants to customize the way redeployments are done then he can always change the default settings, but for most of the users should we switch to AS-4.x behaviour?

Thoughts?



[OT] - The PatternIncludeVirtualFileFilter (and other similar filters) in Branch_5_x needs to be changed to extend the new class that was introduced in JBDEPLOYERS for 2.0.8 version. Or else these filters won't be usable as a cacheFilter.

[alesj]

"jaikiran" wrote:
*by default* should we be really doing the guess work for redeployments or like AS-4.x only watch for known top level files

I already have this implemented, but I don't like the catch behind it. :-(

We already have a notion of known file names, it's "hidden" under FileMatcher interface.
Currently the only one's implementing it are exactly the parsing deployers that know certain configuration files == true candidates for re-deploy on change.

But here comes the catch. :-)

FileMatcher only works on VirtualFile, whereas in StructureCacheFilter we match against path.
Hence I added PathMatcher interface, which is again currently only implemented by parsing deployers.
And the catch is how to guarantee consistency between file and path matchers?
A VFFilter and SCFilter for MetaDataStructureModificationChecker must filter the same resources, meaning sets of file and path matchers must be in-synch/consistent.
Currently with only parsing deployers implementing those two interfaces, where I made the impls consistent,
I don't see an issue, but if the user deploys any bean implementing either of those two interfaces,
that bean will be automatically (via in-callback) part of filtering, hence potentially breaking consistency.

Perhaps the solution is to change SCFilter to work against VF instead of just path.
I'll check how much work it is to change this.

[alesj]

"alesj" wrote:

Perhaps the solution is to change SCFilter to work against VF instead of just path.
I'll check how much work it is to change this.

This is now done in Deployers trunk - JBDEPLOY-210.

There is now no need for any StructureCacheFilter (deleted in trunk),
VirtualFileFilter instance handles both - current files and the cache.
Hence no consistency issue any more - at least not from this address.

[jaikiran]

This is now done in Deployers trunk - JBDEPLOY-210.

Is the deployers trunk (2.2.x) compatible with AS 5_x branch and do we have plans to upgrade 5_x branch with 2.2.x or are we planning to continue with 2.0.x series in AS 5_x branch? I can give this a try on 5_x branch if 2.2.x is compatible there. I could have given this a quick try there but don't know what other things might break if 2.2.x is not supposed to be in 5_x :)

[alesj]

"jaikiran" wrote:

Is the deployers trunk (2.2.x) compatible with AS 5_x branch

I don't think so.
At least no if you only update Deployers.
There is Kabir's asynch work that requires Kernel 2.2.x as well.

But you can always give it a try with updating component-matrix pom,
to use the latest trunk (== 2.2.x) snapshots - of both: Kernel and Deployers.
(you can deploy them yourself or I can do it for you)

"jaikiran" wrote:
do we have plans to upgrade 5_x branch with 2.2.x or are we planning to continue with 2.0.x series in AS 5_x branch?

I'm more in favor of releasing middle option - 2.1.x.
It would say we did some api changes, but not any major refactoring.

[pmuir]

Does this solution address this use case - a user wants to update a number of deployment descriptors - say persistence.xml, beans.xml and faces-config.xml - and knows that they don't want a restart until all changes are made?

[alesj]

Does this solution address this use case - a user wants to update a number of deployment descriptors - say persistence.xml, beans.xml and faces-config.xml - and knows that they don't want a restart until all changes are made?

Not exactly.

If the changes are atomic - saved all at once - the first one will be picked up,
triggering re-deployment of the whole app, hence the rest two changes are then irrelevant.

But if the changes span over HD check period, there will be multiple re-deployments.
In this case user should suspend HD scanning; there is some MBean that knows how to do this - Tools use it.

[jaikiran]

But if the changes span over HD check period, there will be multiple re-deployments.
In this case user should suspend HD scanning; there is some MBean that knows how to do this - Tools use it.

For non-IDE users this won't be practical.

"alesj" wrote:

Not exactly.

I think, we should allow that usecase, similar to AS 4.x. Shouldn't we? I think its possible to achieve this by using the PatternIncludeVirtualFileFilter filter as the default filter, which only looks for application.xml, web.xml files. So the users can keep changing 'n' files and then finally "touch" the application.xml or the web.xml to trigger the redeployment. Similar to AS-4.x.

[maxandersen]

Fully agree that the automatic default scanning for hotrestart of a deployment should be very limited, i.e. the application descriptors (application.xml, web.xml etc.) and the toplevel directory (for those without deployment descriptors - but this might only work for some OS's)

Scanning for all .xml is very arbitrary and if the API is built for each deployer to say which files should trigger deployment why don't we get the right deployers to implement the specific rules instead of having a global one ? (or did I misunderstand how it is structured today?)

[alesj]

Scanning for all .xml is very arbitrary and if the API is built for each deployer to say which files should trigger deployment why don't we get the right deployers to implement the specific rules instead of having a global one ? (or did I misunderstand how it is structured today?)

With the JBDEPLOY-210 fix this is now trivial to do - JarStructure already in-callbacks all FileMatchers.
And currently only FileMatcher impls are parsing deployers - exactly what we need.

So, this, your wanted feature, will be part of next Deployers release.