7 Replies Latest reply on Feb 21, 2005 7:18 PM by ryan.campbell

    Proof of Concept Plan

    ryan.campbell Expert

      The plan so far has been to roll out jbossbuild to each component under jboss-head. I'd like to propose a change to this plan so that once we do complete the rollout to all components, we can be confident that the other pieces besides jbossbuild will work like we want.

      We need to take the implementation of the new build system, project structure, and build repository full circle for one component. This will implement/prove the following functionality:

      - automated builds published to a build repository (snapshots)
      - thirdparty dependencies downloaded from the build repository
      - performing a standalone build of a component which depends on a snapshot component

      Let's take aop as the initial use case because of it is only dependent on jboss-common. When this work is complete, you sholud be able to:

      cd $WORK
      cvs co jboss-aop
      cd jboss-aop
      ant -f jbossbuild.xml synchronize
      ant -f jbossbuild.xml build
      


      The synchronize target will download the thirdparty jars & a snapshot of jboss-common. The build target will build the standalone release of jboss-aop.

      This should give us a concrete basis for moving forward with jbossbuild and the over all restructuring of the build.

        • 1. Re: Proof of Concept Plan
          Adrian Brock Master

          Ok with me. But remember we want to get to the stage ASAP where
          JBoss Cache and JBoss AOP can be true standalone projects.

          This obviously means we need to implement the repository to do what you are
          proposing, but we also need to test the new build against jboss-head so
          we can then backport it to jboss3/4

          Only when this is all done will these standalone projects be able to easily
          develop their code/branches and target different releases of JBoss
          and QA/Labs be able to manage the process.

          The only option for these standalone projects at the moment is to
          deliver binaries to the different JBoss releases
          hoping for the best that everything hangs together with the thirdparty jar versions
          and other dependencies.

          • 2. Re: Proof of Concept Plan
            Ivelin Ivanov Expert

            The plan makes sense. Hopefully the projects that want to jump early on the new build system will help with the rest of the module migration in HEAD once the full proof of concept is done.

            • 3. Re: Proof of Concept Plan
              ryan.campbell Expert

              I have a standalone build of AOP that works like this.

              build.properties <- desginates the main (toplevel) build for this workspace
              tools/*
              aop/
              release.xml <- contains toplevel build for aop
              jbossbuild.xml <- contains componentdef for aop
              thirdparty/ <-- working on getting this from repository

              So, in this setup, aop/jbossbuild.xml uses a property from build.properties to determine where the main build is. If it can't find this file, it defaults to jbossas.

              So to build the release, you would do ant -f release.xml build

              Two issues I can see thus far:

              1. Using build.properties in the root of the work dir is a hassle and won't work long term, unless perhaps we create a cvs module which contains this and aop & tools. Not sure we want more fun with modules. What is a better way for a component to know which main build to use?

              2. Having the aop main build in the same directory as the aop componentdef (jbossbuild.xml) doesn't seem to jive well with the synchronize target which wants to update the main build & then the components which results in aop being update twice. I think I can work around this, but perhaps there are other good reasons to have main builds be in seperate directories?

              I am ignoring these issues for the moment while I implement the cruisecontrol build & publish task, but I wanted to get them out here.

              • 4. Re: Proof of Concept Plan
                Adrian Brock Master

                Let's talk about this at JBossWorld.

                My thinking is moving towards having the artifacts in each project
                rather than using the following in each project.

                 <!-- The main build -->
                 <import file="../jbossas/build.xml"/>
                


                The way I'm seeing this is:

                Top Level Builds:
                Define the projects that make up the build
                Define where to copy artificats from the projects during the "release" process

                Project Builds
                Define there own artifacts
                Define the projects they import

                Binary Builds
                When the binary versions are downloaded into thirdparty
                it also includes the build.xml where the artificats (and project exports) are defined

                Changes Required:
                1) Projects don't use a specific top level build
                2) Instead they import a project
                 <source id="main">
                 <include project="commons"/>
                etc.
                

                3) Each project defines there artifacts and exports
                The include project above maps to the following export in
                either common/build.xml or thirdparty/common/build.xml depending upon
                whether the project is source or binary
                 <export id="common">
                 <include input="jboss-common.jar"/>
                 <include input="namespace.jar"/>
                 </export>
                


                PROS
                + The project is no longer tied to a top level build
                + A project can be used in multiple top level builds
                + A project is defined in one place
                + New top level builds with custom project configurations become "trivial"

                CONS
                - A change to a project might make a top level build's release tags invalid
                (e.g. somebody changes a jar name and doesn't update all top level builds where
                it is referenced)
                - To support the artificats definition being in either common/ or thirdparty/common
                it would require that we handle the
                <include project="common"/>

                ourselves, i.e. to drive the xml parsing.
                The importStack processing in ant's ProjectHelper2 is described as
                experimental in the javadoc.
                We need to confirm that this can be used in this use case.
                - The synchronization process (cvs update/checkout or download from
                repository) becomes more complicated. After we
                synchronize the source projects we need to discover whether we have
                all the projects referenced on the includes and synchronize those as well.

                • 5. Re: Proof of Concept Plan
                  ryan.campbell Expert

                  I know that I originally wanted all builds to be standalone, but working with your impl has made me feel differently :-)

                  The main reason I see to keep project (component) builds dependent on a toplevel build is because the toplevel build is repsonsible for defining the versions of all components.

                  <include project="common">
                  


                  The above defines a dependency on common, but doesn't specify which version. Even if we were to structure the repository as you suggest-- essentially have the repository structure implicitly represent required component versions, you would still need to specify the repsitory root, ie, repository.jboss.com/jbossAS/4.0/ which again ties you to an integration project.

                  I suppose each component could define the version for each of its dependencies, but I don't think this is scalable.

                  What if each component simply referenced the toplevel build as:
                  <import file="../build/release.xml"/>
                  

                  And then each workspace would be structured (and potentially reconfigured by the developer) to support this convention.

                  This would preserve the simplicity of the existing structure and would acheive the decoupling of the component builds from a specific release build. The aop-standalone prototype (coming next) will follow this convention.

                  • 6. Re: Proof of Concept Plan
                    ryan.campbell Expert

                    I have completed the prototype of the aop standalone build. The functionality is equivalent to that originally described, but some things have changed.

                    Here is how to create the build:

                    cvs co -d aop jboss-aop
                    cvs co tools
                    cvs co -d build tools/etc/jbossbuild/release.xml


                    This will give you the following directory structure:

                    aop
                    build
                    tools


                    release.xml should be elsewhere, perhaps in its own module. At any rate, once you have the above checked out, you can call

                    cd build
                    ant -f release.xml synchronize.components


                    This will download the jars from the repository and put them in thirdparty. This is currently located at http://cruisecontrol.jboss.com/repository. Cruisecontrol is setup to run the jbossas/build.xml daily, publish the binaries, and email me: http://cruisecontrol.jboss.com/cc/buildresults/jboss-head-repository

                    You can then

                    ant -f release.xml build


                    And the release structure for aop will be built:

                    ./output/jboss-aop-1.0.0-FINAL/lib/concurrent.jar
                    ./output/jboss-aop-1.0.0-FINAL/lib/javassist.jar
                    ./output/jboss-aop-1.0.0-FINAL/lib/jboss-aop.jar
                    ./output/jboss-aop-1.0.0-FINAL/lib/jboss-common.jar
                    ./output/jboss-aop-1.0.0-FINAL/lib/junit.jar
                    ./output/jboss-aop-1.0.0-FINAL/lib/qdox.jar
                    ./output/jboss-aop-1.0.0-FINAL/lib/trove.jar


                    I'll followup next with some discussion on limitiations, improvements, etc.

                    • 7. Re: Proof of Concept Plan
                      ryan.campbell Expert

                      A few points:

                      !! I just realized I forgot to checkin a change to aop/jbossbuild.xml which uses build/release.xml as toplevel build if available. I will commit this when cvs is back up.

                      + Obviously, release.xml is not in the Right Place. After discussing this a bit with Ivelin, I think the ideal structure is this:

                      aop-standalone (virtual module in CVSROOT/modules)
                       build (actual module: aop-build)
                       release.xml
                       jbossbuild.jar
                       tasks.xml
                       aop (actual module: jboss-aop)
                      


                      So you just have to do a cvs co aop-standalone, and you are ready to roll.

                      + In my previous email, I neglected how you would change this from a partial build (ie, based on snapshots of common & container) to a full source build. To do this at this point, you would need to add the version tag back on to the component definition. So that:

                      <component id="container"
                       version="5.0-SNAPSHOT">....
                      becomes this:
                      <component id="container"
                       version="5.0-SNAPSHOT"
                       module="container">....

                      And then you would do an "ant -f release.xml synchronize.components" to check it out.

                      I know this needs to be in a property file instead. Perhaps just something like:

                      container.isLocal=true
                      common.isLocal=false
                      
                      http://jira.jboss.com/jira/browse/JBBUILD-15

                      + The cvs repo information needs to come from the component's component-info.xml in the repository.
                      http://jira.jboss.com/jira/browse/JBBUILD-16

                      + I disabled sychronize.after (ide generation) because execant is broken on windows. http://jira.jboss.com/jira/browse/JBBUILD-17