10 Replies Latest reply on May 27, 2009 9:14 AM by kabirkhan

    Uploading MC artifacts to the labs downloads area

    kabirkhan

      I'm working on a script to automate uploading the Microcontainer releases to the labs downloads area. From the comments of the script:

       <!--
       Script to put the releases belonging to the microcontainer project in the downloads on labs.
      
       Properties for this script are set in local.properties.xml:
      
       'maven.home' contains the root of your maven installation.
      
       'labs.checkout.folder' should point to a checkout of the jbossmc area of labs,
       svn url: https://cms.labs.jboss.com/prod/forge/portal-content/default/members/jbossmc
      
       Which individual sub-projects to be released are controlled by the version set in
       the 'version.xxx' properties. If you don't want to release a particular sub-project,
       then comment out that version in local.properties.
      
       'release.checkout.folder' should point to an empty directory, which is where the
       script will check out each individual sub-project releases from subversion.
      
       Some projects are 'simple', i.e. they contain only one module. For these projects
       the artifacts will be checked out from the subversion repository backing the maven
       repository. The binary and source artifacts are combined into one zip.
      
       Some sub-projects are 'complex', i.e. they contain more than one module. These projects
       will be checked out from the tag in subversion and built using their distribution build.
       This is a safeguard, since whoever case whoever is publishing the release to maven
       will probably forget to publish the whole release artifact. For all complex projects
       the resulting source zip is added to the resulting binary zip.
      
       Once the script has all the required artifacts required, it invokes a custom ant task
       to copy the zips to a sub-folder of ${labs.checkout.folder}/individual/downloads per sub-project.
       The ant task also looks for the sub-project's marker in ${labs.checkout.folder}/project.xml
       and updates the downloads section with information about the released files. These releases
       go in the "Indivdual Projects" category of project.xml.
      
       In addition the custom ant task is run for the dist with dependencies for jboss-kernel (formerly
       known as microcontainer) and jboss-deployers, so that they are copied across to the
       ${labs.checkout.folder}/framework/downloads, and the ${labs.checkout.folder}/project.xml
       updated. These releases go in the "Indivdual Projects" category of project.xml.
      
       NOTE: The lastModified time of the file is used as the release date in ${labs.checkout.folder}/project.xml,
       so that might need tweaking if this script is run after the actual maven artifacts were initially published.
      
       The ant task will not add any files to ${labs.checkout.folder}/project.xml if it can find a file with that name
       there already, for example if a release that has already been done has accidentally been included using
       'version.xxx'.
      
       Once done, you should verify that what is in ${labs.checkout.folder} looks correct, and then
       manually invoke 'svn add' for any new files and then 'svn commit' to make the download become available.
       -->
      

      local.properties:
      #Point to your maven install root
      maven.home=/usr/share/maven/
      
      #Enable if testing this. It avoids cleaning/checking out the complex projects every time
      disable.complex.checkout=true
      
      # Point this to the checkout folder of the jbossmc labs area
      labs.checkout.folder=/Users/kabir/sourcecontrol/labs/jbossmc
      
      # The folder where you want to check out and build the releases from
      release.checkout.folder=/Users/kabir/sourcecontrol/mcrelease
      
      # The folder where you want to check out the subversion repository backing the maven
      # repository for publishing the release zips of 'complex' projects. This is only done
      # if the release artifacts were not added to maven
      maven.checkout.folder=sourcecontrol/thirdparty/maven2
      
      # The versions of the sub-projects you want to release. They must be in the form
      # of the release number, e.g. 2.0.1.GA (luckily this matches the convention for
      # how the releases are tagged for when we need to build the projects).
      # If you do not want to release a particular sub-project, comment out the property
      # for its version
      version.reflect=2.0.2.GA
      version.mdr=2.0.1.GA
      #version.mdrc=
      version.man=2.1.0.SP1
      version.kernel=2.0.6.GA
      version.cl=2.0.6.GA
      version.deployers=2.0.7.GA
      #version.reliance=XXXXNotThereYet
      version.vfs=2.1.2.GA
      version.mcint=2.2.0.Alpha1
      


      It currently only publishes the source and binary artifacts since the javadocs don't seem to be published yet? Once we have the improved javadocs that we talked about in Neuchatel those can be added easily.

        • 1. Re: Uploading MC artifacts to the labs downloads area
          kabirkhan

          After talking to Ales we decided to combine the 'simple' projects (jboss-reflect, jboss-vfs, jboss-mdr, jboss-mdr-cache) into zips containing binaries + sources + javadocs (once available) so there is one download per project. These downloads will contain just the project jars and no dependencies.

          Similarly, for the 'complex' projects (man, deployers, cl and kernel/mc) we will publish the zipped artifacts resulting from a 'mvn -Passembly install'. Again, these will just contain the project output and no dependencies.

          Finally, we will ship a 'full' MC zip with MC/Kernel + all dependencies, and a 'full' VDF zip (MC/Kernel + cl + deployers) + all dependencies.

          I'll publish the new build.xml comments once I'm done :-)

          • 2. Re: Uploading MC artifacts to the labs downloads area
            kabirkhan

            Problem:
            microcontainer/tags/2.0.6.GA/docs/pom.xml contained the wrong version (2.0.3-SNAPSHOT) which in turn makes it impossible to do a release build

            [kabir@~/sourcecontrol/mcrelease/microcontainer]
            $mvn -Passembly -Dmaven.test.skip=true install
            [INFO] Scanning for projects...
            Downloading: http://snapshots.jboss.org/maven2/org/jboss/microcontainer/jboss-microcontainer/2.0.3-SNAPSHOT/jboss-microcontainer-2.0.3-SNAPSHOT.pom
            [INFO] ------------------------------------------------------------------------
            [ERROR] FATAL ERROR
            [INFO] ------------------------------------------------------------------------
            [INFO] Failed to resolve artifact.
            
            GroupId: org.jboss.microcontainer
            ArtifactId: jboss-microcontainer
            Version: 2.0.3-SNAPSHOT
            
            Reason: Unable to download the artifact from any repository
            
             org.jboss.microcontainer:jboss-microcontainer:pom:2.0.3-SNAPSHOT
            
            from the specified remote repositories:
             central (http://repo1.maven.org/maven2),
             repository.jboss.org (http://repository.jboss.org/maven2),
             snapshots.jboss.org (http://snapshots.jboss.org/maven2)
            
            
            [INFO] ------------------------------------------------------------------------
            [INFO] Trace
            org.apache.maven.reactor.MavenExecutionException: Cannot find parent: org.jboss.microcontainer:jboss-microcontainer for project: org.jboss.microcontainer.docs:jboss-microcontainer-docs:pom:null for project org.jboss.microcontainer.docs:jboss-microcontainer-docs:pom:null
             at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:378)
             at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:292)
             at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:129)
             at org.apache.maven.cli.MavenCli.main(MavenCli.java:287)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:585)
             at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
             at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
             at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
             at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
            Caused by: org.apache.maven.project.ProjectBuildingException: Cannot find parent: org.jboss.microcontainer:jboss-microcontainer for project: org.jboss.microcontainer.docs:jboss-microcontainer-docs:pom:null for project org.jboss.microcontainer.docs:jboss-microcontainer-docs:pom:null
             at org.apache.maven.project.DefaultMavenProjectBuilder.assembleLineage(DefaultMavenProjectBuilder.java:1370)
             at org.apache.maven.project.DefaultMavenProjectBuilder.buildInternal(DefaultMavenProjectBuilder.java:821)
             at org.apache.maven.project.DefaultMavenProjectBuilder.buildFromSourceFileInternal(DefaultMavenProjectBuilder.java:506)
             at org.apache.maven.project.DefaultMavenProjectBuilder.build(DefaultMavenProjectBuilder.java:198)
             at org.apache.maven.DefaultMaven.getProject(DefaultMaven.java:583)
             at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:461)
             at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:534)
             at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:365)
             ... 11 more
            Caused by: org.apache.maven.project.ProjectBuildingException: POM 'org.jboss.microcontainer:jboss-microcontainer' not found in repository: Unable to download the artifact from any repository
            
             org.jboss.microcontainer:jboss-microcontainer:pom:2.0.3-SNAPSHOT
            
            from the specified remote repositories:
             central (http://repo1.maven.org/maven2),
             repository.jboss.org (http://repository.jboss.org/maven2),
             snapshots.jboss.org (http://snapshots.jboss.org/maven2)
             for project org.jboss.microcontainer:jboss-microcontainer
             at org.apache.maven.project.DefaultMavenProjectBuilder.findModelFromRepository(DefaultMavenProjectBuilder.java:603)
             at org.apache.maven.project.DefaultMavenProjectBuilder.assembleLineage(DefaultMavenProjectBuilder.java:1366)
             ... 18 more
            Caused by: org.apache.maven.artifact.resolver.ArtifactNotFoundException: Unable to download the artifact from any repository
            
             org.jboss.microcontainer:jboss-microcontainer:pom:2.0.3-SNAPSHOT
            
            from the specified remote repositories:
             central (http://repo1.maven.org/maven2),
             repository.jboss.org (http://repository.jboss.org/maven2),
             snapshots.jboss.org (http://snapshots.jboss.org/maven2)
            
             at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:212)
             at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:74)
             at org.apache.maven.project.DefaultMavenProjectBuilder.findModelFromRepository(DefaultMavenProjectBuilder.java:556)
             ... 19 more
            Caused by: org.apache.maven.wagon.ResourceDoesNotExistException: Unable to download the artifact from any repository
             at org.apache.maven.artifact.manager.DefaultWagonManager.getArtifact(DefaultWagonManager.java:331)
             at org.apache.maven.artifact.resolver.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:200)
             ... 21 more
            [INFO] ------------------------------------------------------------------------
            [INFO] Total time: 1 second
            [INFO] Finished at: Fri May 22 16:36:58 BST 2009
            [INFO] Final Memory: 2M/4M
            [INFO] ------------------------------------------------------------------------
            

            I have updated the docs pom in the tag to use the correct version.

            I know we will have this problem in deployers and possibly man as well (cl is fine).
            Paul says it is because the maven build does not include the docs in the normal build since they take considerable time to build, but from what I gather this also means they do not get included when the release plugin updates the version numbers, so this currently needs doing by hand.

            For the current releases I will update the docs versions in the tag's poms. But I don't think this is something we should keep doing, both for the obvious 'don't touch the tag' reason, but also to avoid having to manually fiddle with something that we are trying to automate :-) Also, the script uses anonsvn, so there is the normal time-lag issue of changes done in proper svn getting replicated to anonsvn.

            I know we have started the process of moving docs to a separate svn location, am I correct in assuming that once this is done the problem I am seeing will disappear?

            • 3. Re: Uploading MC artifacts to the labs downloads area
              kabirkhan

              This problem also extends to several poms in docs/ subfolders, I am committing these changes as well.

              [kabir@~/sourcecontrol/microcontainer/tags/2.0.6.GA/subversion]
              $svn diff
              Index: docs/gettingstarted/en-US/pom.xml
              ===================================================================
              --- docs/gettingstarted/en-US/pom.xml (revision 89292)
              +++ docs/gettingstarted/en-US/pom.xml (working copy)
              @@ -2,7 +2,7 @@
               <parent>
               <groupId>org.jboss.microcontainer.docs</groupId>
               <artifactId>jboss-microcontainer-docs</artifactId>
              - <version>2.0.0-SNAPSHOT</version>
              + <version>2.0.6.GA</version>
               </parent>
               <modelVersion>4.0.0</modelVersion>
               <groupId>org.jboss.microcontainer</groupId>
              @@ -43,4 +43,4 @@
               <translation>en_US</translation>
               </properties>
              
              -</project>
              \ No newline at end of file
              +</project>
              Index: docs/gettingstarted/pom.xml
              ===================================================================
              --- docs/gettingstarted/pom.xml (revision 89292)
              +++ docs/gettingstarted/pom.xml (working copy)
              @@ -3,7 +3,7 @@
               <parent>
               <groupId>org.jboss.microcontainer.docs</groupId>
               <artifactId>jboss-microcontainer-docs</artifactId>
              - <version>2.0.0-SNAPSHOT</version>
              + <version>2.0.6.GA</version>
               </parent>
               <modelVersion>4.0.0</modelVersion>
               <groupId>org.jboss.microcontainer.docs</groupId>
              Index: docs/reference/en-US/pom.xml
              ===================================================================
              --- docs/reference/en-US/pom.xml (revision 89292)
              +++ docs/reference/en-US/pom.xml (working copy)
              @@ -2,7 +2,7 @@
               <parent>
               <groupId>org.jboss.microcontainer.docs</groupId>
               <artifactId>jboss-microcontainer-docs</artifactId>
              - <version>2.0.0-SNAPSHOT</version>
              + <version>2.0.6.GA</version>
               </parent>
               <modelVersion>4.0.0</modelVersion>
               <groupId>org.jboss.microcontainer</groupId>
              @@ -43,4 +43,4 @@
               <translation>en_US</translation>
               </properties>
              
              -</project>
              \ No newline at end of file
              +</project>
              Index: docs/reference/pom.xml
              ===================================================================
              --- docs/reference/pom.xml (revision 89292)
              +++ docs/reference/pom.xml (working copy)
              @@ -3,7 +3,7 @@
               <parent>
               <groupId>org.jboss.microcontainer.docs</groupId>
               <artifactId>jboss-microcontainer-docs</artifactId>
              - <version>2.0.0-SNAPSHOT</version>
              + <version>2.0.6.GA</version>
               </parent>
               <modelVersion>4.0.0</modelVersion>
               <groupId>org.jboss.microcontainer</groupId>
              




              • 4. Re: Uploading MC artifacts to the labs downloads area
                kabirkhan

                Paul,

                Now that I have fixed the docs I have a new problem:

                [kabir@~/sourcecontrol/microcontainer/tags/2.0.6.GA/subversion]
                $mvn -Passembly install -Dmaven.test.skip
                [INFO] Scanning for projects...
                [INFO] Reactor build order:
                [INFO] JBoss Microcontainer Parent POM
                [INFO] JBoss Microcontainer Dependency
                [INFO] JBoss Microcontainer Kernel
                [INFO] JBoss Microcontainer AOP MC INT
                [INFO] JBoss Microcontainer Spring Int
                [INFO] JBoss Microcontainer Guice Int
                [INFO] JBoss Microcontainer Distribution Build
                [INFO] JBoss Microcontainer Docs
                [INFO] JBossMC Getting Started Guide (en_US)
                [INFO] JBossMC Getting Started Guide (aggregator)
                [INFO] JBossMC Reference Guide (en_US)
                [INFO] JBossMC Reference Guide (aggregator)
                WAGON_VERSION: 1.0-beta-2
                ....
                [INFO] Building JBoss Microcontainer Kernel
                [INFO] task-segment: [install]
                [INFO] ------------------------------------------------------------------------
                [INFO] [enforcer:enforce {execution: enforce-versions}]
                [INFO] [resources:resources]
                [INFO] Using default encoding to copy filtered resources.
                [INFO] [compiler:compile]
                [INFO] Compiling 409 source files to /Users/kabir/sourcecontrol/microcontainer/tags/2.0.6.GA/subversion/kernel/null
                [INFO] ------------------------------------------------------------------------
                [ERROR] BUILD FAILURE
                [INFO] ------------------------------------------------------------------------
                [INFO] Compilation failure
                
                /Users/kabir/sourcecontrol/microcontainer/tags/2.0.6.GA/subversion/kernel/src/main/java/org/jboss/beans/metadata/spi/builder/BeanMetaDataBuilder.java:[42,32] package org.jboss.dependency.spi does not exist
                
                /Users/kabir/sourcecontrol/microcontainer/tags/2.0.6.GA/subversion/kernel/src/main/java/org/jboss/beans/metadata/spi/builder/BeanMetaDataBuilder.java:[43,32] package org.jboss.dependency.spi does not exist
                ...
                


                I have looked in the kernel/pom.xml, but can't see anything obvious that would be affected by the -Passembly switch? Also, I notice that a normal 'mvn install' has this output for the kernel module
                [INFO] Compiling 409 source files to /Users/kabir/sourcecontrol/microcontainer/tags/2.0.6.GA/subversion/kernel/target/classes
                

                I don't know if that is significant or not?

                • 5. Re: Uploading MC artifacts to the labs downloads area
                  pgier

                  The problem is that any profile with activation "activeByDefault" will be deactivated if another profile is activated. This is a bad "feature" of Maven IMO because it's counter-intuitive to most people. As a work around you can explicitly activate the default profile and do something like.

                  mvn -Passembly,default install -Dmaven.test.skip
                  


                  In general I would recommend not using activeByDefault and instead using a not well documented feature of the property activation to make a profile activeByDefault. So it looks something like this.
                   <profile>
                   <id>default</id>
                   <activation>
                   <property>!skip.default.profile</property>
                   </activation>
                   <properties>
                   </profile>
                  


                  This tells maven that the profile "default" is active unless the property "skip.default.profile" is set.


                  • 6. Re: Uploading MC artifacts to the labs downloads area
                    kabirkhan

                     

                    "kabir.khan@jboss.com" wrote:
                    I know we will have this problem in deployers and possibly man as well (cl is fine).

                    I was wrong. deployers is fine, and I can do a release build for man too. man does not construct the dist jars in the same way as the other 'complex' builds, but I can work around that.

                    • 7. Re: Uploading MC artifacts to the labs downloads area
                      kabirkhan

                      Thanks Paul, that seems to do the job

                      • 8. Re: Uploading MC artifacts to the labs downloads area
                        kabirkhan

                        Actually, the build works, but I get no dist artifacts in build/target

                        • 9. Re: Uploading MC artifacts to the labs downloads area
                          kabirkhan

                           

                          "kabir.khan@jboss.com" wrote:
                          Actually, the build works, but I get no dist artifacts in build/target

                          The dist assembly seems to be tied to the 'dist' profile, so
                          mvn -Pdefault,assembly,dist install -Dmaven.test.skip=true
                          

                          works

                          • 10. Re: Uploading MC artifacts to the labs downloads area
                            kabirkhan

                            I updated the comments in the initial post with the new contents of build.xml and local.properties