Proof of Concept Plan

[ryan.campbell]

The plan so far has been to roll out jbossbuild to each component under jboss-head. I'd like to propose a change to this plan so that once we do complete the rollout to all components, we can be confident that the other pieces besides jbossbuild will work like we want.

We need to take the implementation of the new build system, project structure, and build repository full circle for one component. This will implement/prove the following functionality:

- automated builds published to a build repository (snapshots)
- thirdparty dependencies downloaded from the build repository
- performing a standalone build of a component which depends on a snapshot component

Let's take aop as the initial use case because of it is only dependent on jboss-common. When this work is complete, you sholud be able to:

cd $WORK
cvs co jboss-aop
cd jboss-aop
ant -f jbossbuild.xml synchronize
ant -f jbossbuild.xml build

The synchronize target will download the thirdparty jars & a snapshot of jboss-common. The build target will build the standalone release of jboss-aop.

This should give us a concrete basis for moving forward with jbossbuild and the over all restructuring of the build.

[adrian.brock]

Ok with me. But remember we want to get to the stage ASAP where
JBoss Cache and JBoss AOP can be true standalone projects.

This obviously means we need to implement the repository to do what you are
proposing, but we also need to test the new build against jboss-head so
we can then backport it to jboss3/4

Only when this is all done will these standalone projects be able to easily
develop their code/branches and target different releases of JBoss
and QA/Labs be able to manage the process.

The only option for these standalone projects at the moment is to
deliver binaries to the different JBoss releases
hoping for the best that everything hangs together with the thirdparty jar versions
and other dependencies.

[ivelin.ivanov]

The plan makes sense. Hopefully the projects that want to jump early on the new build system will help with the rest of the module migration in HEAD once the full proof of concept is done.

[ryan.campbell]

I have a standalone build of AOP that works like this.

build.properties <- desginates the main (toplevel) build for this workspace
tools/*
aop/
release.xml <- contains toplevel build for aop
jbossbuild.xml <- contains componentdef for aop
thirdparty/ <-- working on getting this from repository

So, in this setup, aop/jbossbuild.xml uses a property from build.properties to determine where the main build is. If it can't find this file, it defaults to jbossas.

So to build the release, you would do ant -f release.xml build

Two issues I can see thus far:

1. Using build.properties in the root of the work dir is a hassle and won't work long term, unless perhaps we create a cvs module which contains this and aop & tools. Not sure we want more fun with modules. What is a better way for a component to know which main build to use?

2. Having the aop main build in the same directory as the aop componentdef (jbossbuild.xml) doesn't seem to jive well with the synchronize target which wants to update the main build & then the components which results in aop being update twice. I think I can work around this, but perhaps there are other good reasons to have main builds be in seperate directories?

I am ignoring these issues for the moment while I implement the cruisecontrol build & publish task, but I wanted to get them out here.

[adrian.brock]

Let's talk about this at JBossWorld.

My thinking is moving towards having the artifacts in each project
rather than using the following in each project.

 <!-- The main build -->
 <import file="../jbossas/build.xml"/>

The way I'm seeing this is:

Top Level Builds:
Define the projects that make up the build
Define where to copy artificats from the projects during the "release" process

Project Builds
Define there own artifacts
Define the projects they import

Binary Builds
When the binary versions are downloaded into thirdparty
it also includes the build.xml where the artificats (and project exports) are defined

Changes Required:
1) Projects don't use a specific top level build
2) Instead they import a project

 <source id="main">
 <include project="commons"/>
etc.

3) Each project defines there artifacts and exports
The include project above maps to the following export in
either common/build.xml or thirdparty/common/build.xml depending upon
whether the project is source or binary

 <export id="common">
 <include input="jboss-common.jar"/>
 <include input="namespace.jar"/>
 </export>

PROS
+ The project is no longer tied to a top level build
+ A project can be used in multiple top level builds
+ A project is defined in one place
+ New top level builds with custom project configurations become "trivial"

CONS
- A change to a project might make a top level build's release tags invalid
(e.g. somebody changes a jar name and doesn't update all top level builds where
it is referenced)
- To support the artificats definition being in either common/ or thirdparty/common
it would require that we handle the

<include project="common"/>

ourselves, i.e. to drive the xml parsing.
The importStack processing in ant's ProjectHelper2 is described as
experimental in the javadoc.
We need to confirm that this can be used in this use case.
- The synchronization process (cvs update/checkout or download from
repository) becomes more complicated. After we
synchronize the source projects we need to discover whether we have
all the projects referenced on the includes and synchronize those as well.

[ryan.campbell]

I know that I originally wanted all builds to be standalone, but working with your impl has made me feel differently :-)

The main reason I see to keep project (component) builds dependent on a toplevel build is because the toplevel build is repsonsible for defining the versions of all components.

<include project="common">

The above defines a dependency on common, but doesn't specify which version. Even if we were to structure the repository as you suggest-- essentially have the repository structure implicitly represent required component versions, you would still need to specify the repsitory root, ie, repository.jboss.com/jbossAS/4.0/ which again ties you to an integration project.

I suppose each component could define the version for each of its dependencies, but I don't think this is scalable.

What if each component simply referenced the toplevel build as:

<import file="../build/release.xml"/>

And then each workspace would be structured (and potentially reconfigured by the developer) to support this convention.

This would preserve the simplicity of the existing structure and would acheive the decoupling of the component builds from a specific release build. The aop-standalone prototype (coming next) will follow this convention.

[ryan.campbell]

I have completed the prototype of the aop standalone build. The functionality is equivalent to that originally described, but some things have changed.

Here is how to create the build:

cvs co -d aop jboss-aop
cvs co tools
cvs co -d build tools/etc/jbossbuild/release.xml

This will give you the following directory structure:

aop
build
tools

release.xml should be elsewhere, perhaps in its own module. At any rate, once you have the above checked out, you can call

cd build
ant -f release.xml synchronize.components

This will download the jars from the repository and put them in thirdparty. This is currently located at http://cruisecontrol.jboss.com/repository. Cruisecontrol is setup to run the jbossas/build.xml daily, publish the binaries, and email me: http://cruisecontrol.jboss.com/cc/buildresults/jboss-head-repository

You can then

ant -f release.xml build

And the release structure for aop will be built:

./output/jboss-aop-1.0.0-FINAL/lib/concurrent.jar
./output/jboss-aop-1.0.0-FINAL/lib/javassist.jar
./output/jboss-aop-1.0.0-FINAL/lib/jboss-aop.jar
./output/jboss-aop-1.0.0-FINAL/lib/jboss-common.jar
./output/jboss-aop-1.0.0-FINAL/lib/junit.jar
./output/jboss-aop-1.0.0-FINAL/lib/qdox.jar
./output/jboss-aop-1.0.0-FINAL/lib/trove.jar

I'll followup next with some discussion on limitiations, improvements, etc.

[ryan.campbell]

A few points:

!! I just realized I forgot to checkin a change to aop/jbossbuild.xml which uses build/release.xml as toplevel build if available. I will commit this when cvs is back up.

+ Obviously, release.xml is not in the Right Place. After discussing this a bit with Ivelin, I think the ideal structure is this:

aop-standalone (virtual module in CVSROOT/modules)
 build (actual module: aop-build)
 release.xml
 jbossbuild.jar
 tasks.xml
 aop (actual module: jboss-aop)

So you just have to do a cvs co aop-standalone, and you are ready to roll.

+ In my previous email, I neglected how you would change this from a partial build (ie, based on snapshots of common & container) to a full source build. To do this at this point, you would need to add the version tag back on to the component definition. So that:

<component id="container"
 version="5.0-SNAPSHOT">....

becomes this:

<component id="container"
 version="5.0-SNAPSHOT"
 module="container">....

And then you would do an "ant -f release.xml synchronize.components" to check it out.

I know this needs to be in a property file instead. Perhaps just something like:

container.isLocal=true
common.isLocal=false

http://jira.jboss.com/jira/browse/JBBUILD-15

+ The cvs repo information needs to come from the component's component-info.xml in the repository.
http://jira.jboss.com/jira/browse/JBBUILD-16

+ I disabled sychronize.after (ide generation) because execant is broken on windows. http://jira.jboss.com/jira/browse/JBBUILD-17