Quicker AS build?

[jaikiran]

When building the AS trunk, i usually see numerous snapshots being downloded. This seems to happen (atleast) once a day. For example, this morning when i triggered a build on the "testsuite" module of AS trunk:

..testsuite$] ./build.sh clean jars

I see a whole lot of snapshots being downloaded:

init-dependencies:
[maven:dependencies] [INFO] snapshot org.jboss.jbossas:jboss-as-aspects:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[maven:dependencies] Downloading: org/jboss/jbossas/jboss-as-aspects/6.0.0-SNAPSHOT/jboss-as-aspects-6.0.0-20100128.002825-636.pom from repository snapshots.jboss.org at http://snapshots.jboss.org/maven2
[maven:dependencies] Transferring 5K from snapshots.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop/2.1.8.GA/jboss-aop-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 20K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-parent/2.1.8.GA/jboss-aop-parent-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 18K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-aspects/2.1.8.GA/jboss-aop-aspects-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 3K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-core/2.1.8.GA/jboss-aop-asintegration-core-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 5K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/pluggable-instrumentor/2.1.8.GA/pluggable-instrumentor-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 1K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-jmx/2.1.8.GA/jboss-aop-asintegration-jmx-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
[maven:dependencies] Transferring 6K from repository.jboss.org
[maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-mc/2.1.8.GA/jboss-aop-asintegration-mc-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
....

This snapshot download lasts anywhere between 15-20 minutes (and sometimes even half an hour) on my system. Is there any way this can be avoided?

By the way, my understanding was that the AS doesn't depend on any snapshot dependencies. So why are these being downloaded in first place?

[dimitris]

I've asked Paul the same thing:

"It's an annoying bug [1] in Maven where it will check for snapshot updates  according to the update policy (once per day by default) even if an up to date  copy is installed locally.  The good news is that it still uses the correct jars  in the build, even though it looks at the remote repo, and I think it is already  fixed in Maven 3.  But we're stuck with it until Maven 3 goes to GA.

[1]http://jira.codehaus.org/browse/MNG-4326"

[jaikiran]

BUILD SUCCESSFUL
Total time: 26 minutes 11 seconds

That's the total time on the build of "testsuite" module. Out of which 24 minutes were spent in downloading the snapshots.

[jaikiran]

[e-mail] wrote:

I've asked Paul the same thing:

"It's an annoying bug [1] in Maven where it will check for snapshot updates  according to the update policy (once per day by default) even if an up to date  copy is installed locally. 

[1]http://jira.codehaus.org/browse/MNG-4326"

Hmm, maybe as a workaround, we need some kind of custom "never update" policy as the default for the AS build.

[wolfc]

You both are missing the real issue, it's downloading a snapshot of a dependency that really is a module within AS.

Instead of downloading such a snapshot, it should build the other module.

[jaikiran]

wolfc wrote:

You both are missing the real issue, it's downloading a snapshot of a dependency that really is a module within AS.

Instead of downloading such a snapshot, it should build the other module.

Yep, and that bug has been acknowledged in http://jira.codehaus.org/browse/MNG-4326

And it doesn't have a fix in Maven-2.x

[pgier]

jaikiran wrote:

[e-mail] wrote:

I've asked Paul the same thing:

"It's an annoying bug [1] in Maven where it will check for snapshot updates  according to the update policy (once per day by default) even if an up to date  copy is installed locally. 

[1]http://jira.codehaus.org/browse/MNG-4326"

Hmm, maybe as a workaround, we need some kind of custom "never update" policy as the default for the AS build.

You can set the update policy for the snapshots repo to "never" using your settings.xml.

http://maven.apache.org/settings.html#Repositories

Then when you want to update the snapshot dependencies, use "mvn -U install"

[brian.stansberry]

Thanks, Paul, that seems like excellent advice.

I know 99.9% of the time my interest in snapshots is because I'm working on two projects simultaneously and want the second to see the results of the first. I only want snapshots from my local repo.

The other .1% of the time someone has published a snapshot I need and told me about it, in which case I should use -U.

0% of the time am I interested in some random snapshot on snapshots.jboss.org.

[jaikiran]

Gave that a try this morning:

<repository>
    <id>snapshots.jboss.org</id>
    <url>http://snapshots.jboss.org/maven2</url>
    <releases>
    <enabled>false</enabled>
    </releases>
    <snapshots>
    <enabled>true</enabled>
    <!-- We don't want to download snapshots by default
        http://community.jboss.org/thread/147383 -->
    <updatePolicy>never</updatePolicy>
    </snapshots>
</repository>

No more downloads from the snapshot repository during the build. The build is definitely faster now.

It however, still "checks" the snapshot repo though:

init-dependencies:
[INFO] snapshot org.jboss.jbossas:jboss-as-aspects:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-system-jmx:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-system:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-cluster:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-server:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-deployment:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-security:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-connector:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-console:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
[INFO] snapshot org.jboss.jbossas:jboss-as-management:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
...

Doesn't download anything (which is good news).

[dimitris]

All right, that worked great for me too:

     .....

<updatePolicy>never</updatePolicy>

I'm wondering, can we have those repository settings applied by default as part of the AS checkout ? I.e. for someone that just checks out the code and wants to start coding, if he doesn't have overrides in his local settings.xml, sensible defaults should apply.

[ropalka]

Thanks for the advice Paul, really useful.

[whitingjr]

An alternative technique is to pass to maven the offline flag which avoids artifact downloading

> mvn --offline compile

or

> mvn -o compile

[pgier]

Just to provide some more information on this, the problem is that with the current AS build, the Maven Ant tasks have to re-resolve the dependencies during the dist build.  This is because the antrun plugin by itself does not provide enough information to the Ant part of the build.  I'm currently working on a release of the maven-antrun-plugin that will allow me to remove the Maven Ant tasks from the dist part of the build.  This means that the dependencies will only have to be resolved once, and you won't see snapshots being downloaded if they were just installed.

Also related to this is an upstream issue where Maven will check (but not download) the snapshot repositories even when there is an up to date version available in the local repo.  This has been fixed in Maven 3.

http://jira.codehaus.org/browse/MNG-4326