1 2 Previous Next 28 Replies Latest reply on Jan 7, 2005 6:53 PM by Scott Stark

    NEW BUILD

    Adrian Brock Master

      BEGIN RANT
      We desparately need a replacement for buildmagic.
      Development is becoming totally unscalable for me.

      It can take me over three hours to resync and rebuild the three trees with CVS.
      But then usually there is some change in the build config which means I've got to
      go reget thirdparty from cvs or do a full rebuild to remove old xdoclet generated classes,
      etc.
      It never runs smoothly.

      It makes using something like eclipse for development nearly impossible because
      you've then got to reproduce the build config changes in the eclipse configuration,
      especially since there appear to be a competition amongst some developers
      to include as much of thirdparty or as many other projects in their projects as
      possible.
      Some of the classpaths seem to change at rate of more than once per week.
      END RANT

        • 1. Re: NEW BUILD
          Adrian Brock Master

          I propose to spend some time fixing the build to use plain ant1.6 features
          and make the thing incremental. I tried before, but people have gradually broken it.

          That includes removing the totally unscalable xdoclet generation of mbean interfaces.

          Having, just started to use eclipse for development, it is so much easier when you
          build is incremental to get a turnaround on testing.
          i.e. Change the code, build (what has changed), run the test, takes a few seconds
          rather than a few minutes.

          We need this in the main JBoss build.

          • 2. Re: NEW BUILD
            Adrian Brock Master

            This task looks like it can do most (if not all) of buildmagic is trying to achieve:
            http://ant.apache.org/manual/CoreTasks/subant.html

            • 3. Re: NEW BUILD
              Scott Stark Master

              Hook up with Ryan to get this done. This is his top priority post the jira migration.

              • 4. Re: NEW BUILD
                ryan.campbell Expert

                I agree that ant 1.6 features can be used to replace large parts of buildmagic. tools/buildmagic/buildmagic.ent should become a template from which other build files can import and customize via properties and overrides.

                However, I can't find a way to control the order in which subant builds each module. Perhaps this can be done by explicitly ordering the modules in the fileset passed to subant, but I don't think this ordering is guaranteed.

                What is the alternative to xdoclet?

                • 5. Re: NEW BUILD
                  Scott Stark Master

                  xdoclet is really a seperate issue from the existing structure problems and build system.

                  • 6. Re: NEW BUILD
                    ryan.campbell Expert

                    It seems like many of these issues are symptomatic of the application server project hosting the code of so many super-subprojects like aop and the microcontainer. Are these going to be split into their own projects? IE, can aop become a thirdparty jar for the application server?

                    I have put together some ideas on how the build system might deal with such an organization.

                    http://www.jboss.org/wiki/Wiki.jsp?page=BuildDependencyManagement

                    It also addresses management of thirdparty libraries in a more scalable manner.

                    As an aside, by providing declarative dependencies, it would be possible to do things like automatically generate eclipse .classpath files automatically.

                    • 7. Re: NEW BUILD
                      Scott Stark Master

                      Yes, that is the idea. Seperation of the projects needs to be combined with increased integration tests not owned by the project developers to validate that the expected contracts are maintained across versions.

                      • 8. Re: NEW BUILD
                        Adrian Brock Master

                         

                        "ryan.campbell@jboss.com" wrote:

                        However, I can't find a way to control the order in which subant builds each module. Perhaps this can be done by explicitly ordering the modules in the fileset passed to subant, but I don't think this ordering is guaranteed.


                        See the FileList subelement

                        • 9. Re: NEW BUILD
                          mazz Master

                          Maven can produce Eclipse .project and .classpath files automatically for you. It also has the ability to pull down dependency jars (with versioning). It also has features for subcomponent and "super project" definitions (though I've never used that part of it).

                          Have you considered maven? It does take getting used to and I admit I've had mixed results. However, it does have some very nice features.

                          BTW: Mevenide is an Eclipse plugin that allows you to synchronize your Eclipse .classpath with the Maven config - so no more will you have to examine the ant build scripts to determine your environment and manually port that to your Eclipse environment.

                          • 10. Re: NEW BUILD
                            mazz Master

                            ok, so I read the other thread about "To Maven Or Not" and it seems most folks are dead set against it. So... never mind (I'm not married to maven; I was more or less curious as to what people think - now I know :-)

                            • 11. Re: NEW BUILD
                              ryan.campbell Expert

                              Some tools which provide maven-like dependency resolution:

                              Savant - http://inversoft.com/online/savant/savant.html

                              Antlion - http://antlion.sourceforge.net/inline-manual/ht-libraries.html

                              Both of these are just Ant task libraries. They both can download depedencies from a repository and integrate with your build by producing ant paths which you can use in your compile target.

                              • 12. Re: NEW BUILD
                                Adrian Brock Master

                                So I was playing with ant over the Christmas holiday and after a couple of iterations,
                                I came up with something that works and provides a more declaritive
                                way of building projects.

                                This is just a quick overview. I didn't work too hard on the thirdparty
                                libraries since I didn't have a copy of cvs server on my machine.

                                The basic idea is that you declare the top level integration build then declare
                                each component. All the common stuff is defined in a separate tasks file.

                                Although the declarations require some code, I tried to keep it user definable as
                                possible.

                                Examples following in subsequent posts,

                                Imagine a project structured as follows:

                                ${somedir}/JBossAS - the main integration build
                                ${somedir}/test1 - test1 component
                                ${somedir}/test2 - test2 component
                                ${somedir}/tools - the tools directory containing jboss/tasks.xml

                                • 13. Re: NEW BUILD
                                  Adrian Brock Master

                                  The main build contains a declaration of the components that make up the
                                  overall integration project:

                                  <?xml version="1.0"?>
                                  
                                  <!--
                                   JBoss, the OpenSource J2EE webOS
                                  
                                   Distributable under LGPL license.
                                   See terms of license at gnu.org.
                                  -->
                                  <project name="main.build"
                                   default="build"
                                   basedir="."
                                  >
                                   <!-- Import the types -->
                                   <import file="../tools/jboss/tasks.xml"/>
                                  
                                   <!-- The test classpath -->
                                   <includes id="test.path">
                                   <include input="junit.jar"/>
                                   </includes>
                                  
                                   <!-- The build -->
                                   <build id="JBossAS"
                                   version="5.0.0alpha"
                                   description="JBoss Application Server"
                                   targetdefs="targets">
                                  
                                   <component id="jboss">
                                   <artifact id="jboss-common.jar"
                                   location="file:///home/ejort/jboss-head/workspace/common/output/lib/jboss-common.jar"
                                   />
                                   </component>
                                  
                                   <component id="junit">
                                   <artifact id="junit.jar"
                                   location="file:///home/ejort/jboss-head/workspace/thirdparty/junit-junit/lib/junit.jar"
                                   />
                                   </component>
                                  
                                   <component id="test1">
                                   <artifact id="test1.jar" release="lib"/>
                                   <artifact id="test1.api"/>
                                   </component>
                                  
                                   <component id="test2">
                                   <artifact id="test2.jar" release="lib"/>
                                   <artifact id="test2.api"/>
                                   </component>
                                   </build>
                                  
                                  </project>
                                  


                                  build - defines what you are building
                                  component - defines the subprojects which might be thirdparty rather than local projects
                                  artifact - defines what you expect to come out of the component and where it should be placed in the integration release structure.

                                  Like I said, I didn't do too much work on the thirdparty/download stuff.
                                  But you can imagine that it can contain the location of the components
                                  that maybe urls or cvs checkout parameters.

                                  I would imagine that the location would contain the version of the jar in the url?
                                  I would also image that most of the CVS parameters would be common
                                  and actually defined in the build, e.g. the CVSROOT and the tag.

                                  You can ignored the includes this just works to define a common classpath
                                  in this case the jars required for running tests.

                                  • 14. Re: NEW BUILD
                                    Adrian Brock Master

                                    The component build defines how to build the artifacts referenced in the main
                                    build:

                                    There is a componentdef and an artifactdef referencing the ids defined in the main
                                    build file.
                                    There are also definitions so you can differentiate the different source
                                    types.
                                    e.g. I can imagine source types of main (like we have now), interfaces,
                                    implementation, client (classes used on the client), test (the test classes
                                    not for distribution), etc.

                                    <?xml version="1.0"?>
                                    
                                    <!--
                                     JBoss, the OpenSource J2EE webOS
                                    
                                     Distributable under LGPL license.
                                     See terms of license at gnu.org.
                                    -->
                                    <project name="project"
                                     default="build"
                                     basedir="."
                                    >
                                     <!-- The main build -->
                                     <import file="../JBossAS/build.xml"/>
                                    
                                     <includes id="classes">
                                     <include input="main"/>
                                     </includes>
                                    
                                     <!-- The component definition -->
                                     <componentdef component="test2" description="Project test2">
                                    
                                     <source id="main">
                                     <include input="test1.jar"/>
                                     <include input="jboss-common.jar"/>
                                     </source>
                                    
                                     <artifactdef artifact="test2.jar">
                                     <include input="classes"/>
                                     </artifactdef>
                                    
                                     <source id="test" test="**/*TestCase.java">
                                     <include input="classes"/>
                                     <include input="test.path"/>
                                     </source>
                                    
                                     <artifactdef artifact="test2.api">
                                     <include input="main"/>
                                     <include input="test"/>
                                     </artifactdef>
                                     </componentdef>
                                    
                                    </project>
                                    


                                    1 2 Previous Next