1 2 Previous Next 28 Replies Latest reply on Jun 22, 2005 11:17 AM by Scott Stark

    New Build - Restate requirements

    Adrian Brock Master

      After discussing this with Scott yesterday, we appear to have different views
      on what the build system should be providing.

      The requirements have certainly moved a long way since I initiated this work
      and
      1) the jbossbuild project no longer appears to model what people are trying
      to achieve.
      2) what people are trying to achieve is not clearly defined to me anymore

        • 1. Re: New Build - Restate requirements
          Adrian Brock Master

          First some history.

          The initial jbossbuild worked by having three different models.

          1) The top level build model that defined all artifacts, these are the
          build/component/artifact elements
          2) The component build model that defined how those artifacts were constructed
          componentdef/artifactdef/sourcedef
          3) The tasks.xml that defined the artifact types and the macros that provide
          foreach type behaviour on the model above

          Assumptions:
          1) The top level build information was shared by all the components and defined
          common/consistent properties. It also defined the release structure
          2) The top level/component level model could be combined where you just had one
          project
          3) Components were treated equivalently whether they were binary or source
          the only difference being whether an artifact's location was resolved to
          thirdparty/component or the ROOT/component/output
          4) The build should be as declarative as possible, potentially allowing custom tasks
          to other build scripts to mine the build structure to provide other behaviour,
          e.g. cruisecontrol

          • 2. Re: New Build - Restate requirements
            Adrian Brock Master

            Overtime, the requirements have changed and I believe we have a current
            structure something like the following:

            1) A top level build that just defines the list of primary components
            2) The component level build is made up of two files
            component-info which describes the external view of the component
            component build which desribes how the component is built
            3) The tasks/macros as before

            The component-info is intended to describe what this component provides,
            but also what it uses. i.e. it could import other components (secondary components)
            e.g. the test module will import junit, there is no need in the top level build to define
            that we use junit because the test module provides that point of integration.

            • 3. Re: New Build - Restate requirements
              Adrian Brock Master

              Further requirements that have been picked up along the way:

              1) Being able to federate builds.
              e.g. jboss-portal includes jbossas
              i.e. one project is not based on a specfic list of components, it is based on a list
              provided by another project
              2) Being able to version components to check their consistency
              3) Provide a more rich definition of the component, e.g. licenses what "spec" it implements
              4) Allow for custom release structures, e.g. being able to build testsuite configs using
              similar information that is used to build the main release
              5) Making a lot of the component information available at runtime

              • 4. Re: New Build - Restate requirements
                Adrian Brock Master

                What I propose we do is clearly define the current requirements with
                use cases explaining how they will be used.
                Then people can show counter examples or argue why it should be done a different
                way and we will more aware of why we are doing something in a particular way.

                • 5. Re: New Build - Restate requirements
                  Scott Stark Master

                  Agreed. We need a real project to prototype the requirements and the interaction of components in both source and binary form, as well as getting to the release aspect of the build system. Matching artifacts on the legacy build was an ok start, but its not really getting us where we need to be.

                  I would like to see the jbossas component used for this purpose. We can prototype this to point of building a release of the microkernel, ejb3, tomcat + dependencies to flesh out usecases and implementation.

                  • 6. Re: New Build - Restate requirements
                    Adrian Brock Master

                    I was thinking more of writing a regression testsuite for the new build.

                    This could be based on a local cvs repository and binary repository inside the
                    testsuite's output folder.

                    The tests could be based on one of the oldest forms of regression tests
                    where you start with an initial config and apply diffs/commands in steps/stages
                    comparing the result with the expected output after each stage.

                    test1/initial
                    test1/step1/patch (the diffs)
                    test1/step1/commands(ant build invocations)
                    test1/step1/expected
                    etc.

                    • 7. Re: New Build - Restate requirements
                      Scott Stark Master

                      That certainly should be part of the jbossbuild project tests to validate the implementation, but I don't see that it allows project developers to interact easily with the usecase driven design we are asking for.

                      • 8. Re: New Build - Restate requirements
                        Adrian Brock Master

                         

                        "scott.stark@jboss.org" wrote:
                        That certainly should be part of the jbossbuild project tests to validate the implementation, but I don't see that it allows project developers to interact easily with the usecase driven design we are asking for.


                        Yes, but that process is stuck in a deadlock where people request features
                        without actually trying it.
                        The continuous addition of new/changed features means it is not in a state
                        where it is finished or being used.

                        I'd be prepared to cutover jboss-head to the new build so that people can try it
                        (leaving a deprecated-build.xml in place for people who need to get work done)
                        But only, because this is likely to move things forward in terms of feedback and stabilization.

                        • 9. Re: New Build - Restate requirements
                          Scott Stark Master

                          I would like to see a full release build working before cutting over to the sink or swim strategy ;)

                          Bang on a standalone remoting, aop and small jbossas release to better flush out some of the current thirdparty/versioning/release usecases before taking the plunge.

                          • 10. Re: New Build - Restate requirements
                            ryan.campbell Expert

                            Ok, I would like to propose the following plan:

                            1. Get agreement on the following use cases (by mid next week):
                            - versioning/materialize
                            - top-level build structure
                            - release process/markup

                            2. Create a partial build of jbossas which implements the above use cases
                            - in jbossas/jbossbuild.xml on HEAD.
                            - have a task breakdown/roadmap complete by end of next week
                            - each task tied to an approved use case

                            Objections/approval/acquiescence?

                            • 11. Re: New Build - Restate requirements
                              Adrian Brock Master

                              My preferred solution is to try to define upfront what we want it do
                              (I'm not sure what this is anymore ;-)

                              I think Scott wants to prototype so he is confident it is correctly managing thirdparty/versioning.

                              But I'm in favour of anything that speeds up the ditching of buildmagic and enables us
                              to get useful information out of the build so we can more easily manage the process!

                              We don't have any information at the moment other than reading the build scripts
                              or refering to the manually maintained xml files in thirdparty.

                              • 12. Re: New Build - Restate requirements
                                Scott Stark Master

                                I'm not sure the release process can be defined in much detail as there are issues to look into regarding how the installer fits into this. I'll create a seperate thread on issues with creating a 4.0.x installer. The long term release features should not be holding up getting the new build working so that projects can get migrated over to it.

                                The plan seems fine, but I want to see it validated with a minimal prototype build as we have to get the features validated as soon as possible. The approach of migrating a large existing project like jboss-head for each feature change will simply take too long.

                                • 13. Re: New Build - Restate requirements
                                  Scott Stark Master

                                  So the first thirdparty test I need to get working is something like this:

                                  <?xml version="1.0"?>
                                  
                                  <project name="main.build"
                                   default="synchronize"
                                   basedir=".">
                                  
                                   <property file="local.properties"/>
                                   <!-- Import the types -->
                                   <import file="../tools/etc/jbossbuild/tasks.xml"/>
                                  
                                   <property file="synchronize.properties"/>
                                  
                                   <build id="jbossas-thirdparty"
                                   impltitle="JBossAS"
                                   implversion="4.0.2"
                                   implvendor="JBoss Inc."
                                   implurl="http://www.jboss.org"
                                   description="JBoss Application Server"
                                   cvsroot="${cvs.prefix}@cvs.forge.jboss.com:/cvsroot/jboss"
                                   thirdpartypath="../thirdparty/"
                                   location="http://cruisecontrol.jboss.com/repository"
                                   targetdefs="targets"
                                   componentVisitor="com.acme.SomeGraphVisitor"
                                   >
                                  
                                   <component id="hibernate"
                                   version="3.0.3">
                                   </component>
                                   </build>
                                  
                                   <!-- Generate the targets -->
                                   <generate generate="jbossas-thirdparty"/>
                                  
                                   <target name="test" depends="synchronize">
                                   <fail>
                                   <condition>
                                   <not>
                                   <available file="../thirdparty/hibernate/3.0.3/hibernate3.jar" />
                                   <available file="../thirdparty/hibernate/3.0.3/hibernate-metadata.jar" />
                                   <available file="../thirdparty/antlr/2.7.5H3/antlr.jar" />
                                   <available file="../thirdparty/asm/1.5.3/asm.jar" />
                                   </not>
                                   </condition>
                                   </fail>
                                   </target>
                                  </project>
                                  


                                  This tests pulling in the hibernate dependencies. A new feature is the ability to specify a vistor that will have access to the graph of components and their dependencies. This will allow for the generation of the license-info.xml for example.

                                  I have added a org.jboss.ant.util.graph to jbossbuild as a start for graph visitation and algorithms.


                                  • 14. Re: New Build - Restate requirements
                                    ryan.campbell Expert

                                    So in the case of the synchronization cycle, we will want the visitor to traverse the graph and collect a set of unresolved dependencies (ie, no local component-info), download those component-info's and repeat until all component-info's are downloaded.

                                    Once the component-info's are complete, another visitor can traverse the graph and generate synchornize targets for each component.


                                    1 2 Previous Next