12 Replies Latest reply on May 25, 2010 4:56 PM by Paul Gier

    Quicker AS build?

    jaikiran pai Master

      When building the AS trunk, i usually see numerous snapshots being downloded. This seems to happen (atleast) once a day. For example, this morning when i triggered a build on the "testsuite" module of AS trunk:

       

      ..testsuite$] ./build.sh clean jars
      

       

      I see a whole lot of snapshots being downloaded:

       

      init-dependencies:
      [maven:dependencies] [INFO] snapshot org.jboss.jbossas:jboss-as-aspects:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
      [maven:dependencies] Downloading: org/jboss/jbossas/jboss-as-aspects/6.0.0-SNAPSHOT/jboss-as-aspects-6.0.0-20100128.002825-636.pom from repository snapshots.jboss.org at http://snapshots.jboss.org/maven2
      [maven:dependencies] Transferring 5K from snapshots.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/jboss-aop/2.1.8.GA/jboss-aop-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      [maven:dependencies] Transferring 20K from repository.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/jboss-aop-parent/2.1.8.GA/jboss-aop-parent-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      [maven:dependencies] Transferring 18K from repository.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/jboss-aop-aspects/2.1.8.GA/jboss-aop-aspects-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      [maven:dependencies] Transferring 3K from repository.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-core/2.1.8.GA/jboss-aop-asintegration-core-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      [maven:dependencies] Transferring 5K from repository.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/pluggable-instrumentor/2.1.8.GA/pluggable-instrumentor-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      [maven:dependencies] Transferring 1K from repository.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-jmx/2.1.8.GA/jboss-aop-asintegration-jmx-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      [maven:dependencies] Transferring 6K from repository.jboss.org
      [maven:dependencies] Downloading: org/jboss/aop/jboss-aop-asintegration-mc/2.1.8.GA/jboss-aop-asintegration-mc-2.1.8.GA.pom from repository repository.jboss.org at http://repository.jboss.org/maven2
      ....
      

       

      This snapshot download lasts anywhere between 15-20 minutes (and sometimes even half an hour) on my system. Is there any way this can be avoided?

       

      By the way, my understanding was that the AS doesn't depend on any snapshot dependencies. So why are these being downloaded in first place?

        • 1. Re: Quicker AS build?
          Dimitris Andreadis Master

          I've asked Paul the same thing:

           

          "It's an annoying bug [1] in Maven where it will check for snapshot updates  according to the update policy (once per day by default) even if an up to date  copy is installed locally.  The good news is that it still uses the correct jars  in the build, even though it looks at the remote repo, and I think it is already  fixed in Maven 3.  But we're stuck with it until Maven 3 goes to GA.

          [1]http://jira.codehaus.org/browse/MNG-4326"

          • 2. Re: Quicker AS build?
            jaikiran pai Master
            BUILD SUCCESSFUL
            Total time: 26 minutes 11 seconds
            
            

             

            That's the total time on the build of "testsuite" module. Out of which 24 minutes were spent in downloading the snapshots.

            • 3. Re: Quicker AS build?
              jaikiran pai Master

              dimitris@jboss.org wrote:

               

              I've asked Paul the same thing:

               

              "It's an annoying bug [1] in Maven where it will check for snapshot updates  according to the update policy (once per day by default) even if an up to date  copy is installed locally. 

              [1]http://jira.codehaus.org/browse/MNG-4326"

              Hmm, maybe as a workaround, we need some kind of custom "never update" policy as the default for the AS build.

              • 4. Re: Quicker AS build?
                Carlo de Wolf Master

                You both are missing the real issue, it's downloading a snapshot of a dependency that really is a module within AS.

                 

                Instead of downloading such a snapshot, it should build the other module.

                • 5. Re: Quicker AS build?
                  jaikiran pai Master

                  wolfc wrote:

                   

                  You both are missing the real issue, it's downloading a snapshot of a dependency that really is a module within AS.

                   

                  Instead of downloading such a snapshot, it should build the other module.

                  Yep, and that bug has been acknowledged in http://jira.codehaus.org/browse/MNG-4326

                   

                  And it doesn't have a fix in Maven-2.x

                  • 6. Re: Quicker AS build?
                    Paul Gier Master

                    jaikiran wrote:

                     

                    dimitris@jboss.org wrote:

                     

                    I've asked Paul the same thing:

                     

                    "It's an annoying bug [1] in Maven where it will check for snapshot updates  according to the update policy (once per day by default) even if an up to date  copy is installed locally. 

                    [1]http://jira.codehaus.org/browse/MNG-4326"

                    Hmm, maybe as a workaround, we need some kind of custom "never update" policy as the default for the AS build.

                    You can set the update policy for the snapshots repo to "never" using your settings.xml.

                    http://maven.apache.org/settings.html#Repositories

                     

                    Then when you want to update the snapshot dependencies, use "mvn -U install"

                    • 7. Re: Quicker AS build?
                      Brian Stansberry Master
                      Thanks, Paul, that seems like excellent advice.

                      I know 99.9% of the time my interest in snapshots is because I'm working on two projects simultaneously and want the second to see the results of the first. I only want snapshots from my local repo.

                      The other .1% of the time someone has published a snapshot I need and told me about it, in which case I should use -U.

                      0% of the time am I interested in some random snapshot on snapshots.jboss.org.
                      • 8. Re: Quicker AS build?
                        jaikiran pai Master

                        Gave that a try this morning:

                         

                        <repository>
                            <id>snapshots.jboss.org</id>
                            <url>http://snapshots.jboss.org/maven2</url>
                            <releases>
                            <enabled>false</enabled>
                            </releases>
                            <snapshots>
                            <enabled>true</enabled>
                            <!-- We don't want to download snapshots by default
                                http://community.jboss.org/thread/147383 -->
                            <updatePolicy>never</updatePolicy>
                            </snapshots>
                        </repository>
                        

                         

                         

                         

                        No more downloads from the snapshot repository during the build. The build is definitely faster now.

                         

                        It however, still "checks" the snapshot repo though:

                         

                        init-dependencies:
                        [INFO] snapshot org.jboss.jbossas:jboss-as-aspects:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-system-jmx:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-system:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-cluster:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-server:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-deployment:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-security:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-connector:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-console:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        [INFO] snapshot org.jboss.jbossas:jboss-as-management:6.0.0-SNAPSHOT: checking for updates from snapshots.jboss.org
                        ...
                        

                         

                        Doesn't download anything (which is good news).

                        • 9. Re: Quicker AS build?
                          Dimitris Andreadis Master
                          All right, that worked great for me too:
                               .....
                          <updatePolicy>never</updatePolicy>

                          I'm wondering, can we have those repository settings applied by default as part of the AS checkout ? I.e. for someone that just checks out the code and wants to start coding, if he doesn't have overrides in his local settings.xml, sensible defaults should apply.
                          • 10. Re: Quicker AS build?
                            Richard Opalka Master
                            Thanks for the advice Paul, really useful.
                            • 11. Re: Quicker AS build?
                              Jeremy Whiting Expert

                              An alternative technique is to pass to maven the offline flag which avoids artifact downloading

                               

                              > mvn --offline compile

                              or

                              > mvn -o compile

                              • 12. Re: Quicker AS build?
                                Paul Gier Master

                                Just to provide some more information on this, the problem is that with the current AS build, the Maven Ant tasks have to re-resolve the dependencies during the dist build.  This is because the antrun plugin by itself does not provide enough information to the Ant part of the build.  I'm currently working on a release of the maven-antrun-plugin that will allow me to remove the Maven Ant tasks from the dist part of the build.  This means that the dependencies will only have to be resolved once, and you won't see snapshots being downloaded if they were just installed.

                                 

                                Also related to this is an upstream issue where Maven will check (but not download) the snapshot repositories even when there is an up to date version available in the local repo.  This has been fixed in Maven 3.

                                http://jira.codehaus.org/browse/MNG-4326