Maven 2.0 - single module and minimal server build

[ruel.loehr]

I've been working further in depth with maven exploring the following two items:

1) Build of a single module (common)
2) Build of and deployment of the minimal server

The main problem I have encountered so far is source tree splitting into a maven-esque fashion.

Again, for background info in the maven world 1 source tree = 1 jar. Take into consideration the common module. To reproduce its outputs it would need to be broken into 5 subprojects. In attempting to do this I found it necessary to split packages across subprojects. Basically, if you try to pull the logging package out, you can't compile it as it has a dependency on items from the main source. Other items in the main source have a dependency on the logging. This leads to a need for an additional sub project, which houses shared source. I'm eager see the theasability of this with more complicated modules.

The second issue that splitting sources brings up is how would an IDE handle it (e.g eclipse)? Eclipse currently does not support nested projects. In source control the project would look like:

common/
 /subproj1
 /subproj2
 /subproj3

The eclipse user would be need to checkout subproj1, subproj2, and subproj3 as top level projects.

[starksm64]

But the inability to decompose a source tree that is producing multiple artifacts is an indication that its not properly setup. You should be able to pull the common module org.jboss.logging package into a seperate src tree. The only import other than log4j I see is in the org.jboss.logging.filter.TCLFilter where org.jboss.util.collection.WeakSet is imported, so there will have to be a dependency on whatever source tree contains this. What is the problem you are seeing? Does maven support source tree level dependencies?

The common module is one we do want to split up anyway so its decoposition is something to pursue.

In terms of 1 src tree = 1 jar, take the tomcat module which only has one real output, but its a composite of merged jars and the tomcat55-service.jar:

jbossweb-tomcat55.sar/
[starksm@lamia tomcat]$ ls output/deploy/jbossweb-tomcat55.sar/
catalina.jar jasper-compiler-jdt.jar servlets-invoker.jar
catalina-manager.jar jasper-runtime.jar servlets-webdav.jar
catalina-optional.jar jsf-libs/ tomcat55-service.jar
commons-el.jar META-INF/ tomcat-ajp.jar
commons-modeler.jar naming-resources.jar tomcat-apr.jar
conf/ ROOT.war/ tomcat-coyote.jar
context.xml server.xml tomcat-http.jar
jasper-compiler.jar servlets-default.jar tomcat-util.jar

In this case the one artifact constraint seems pedantic as the jbossweb-tomcat55.sar is a derivative of the previous tomcat55-service.jar and its dependencies. Can't maven be mixed with ant to create this type of thing?

[ruel.loehr]

But the inability to decompose a source tree that is producing multiple artifacts is an indication that its not properly setup.

Agreed. Maven is sure to expose a certain number of problem areas in this regard. I was able to split common up, but in order to do, some of the packages were split across the projects. Exactly as you mentioned, logging is dependant on the WeakSet (and all of the things it imports) so these items had to be in a project which is compiled prior to logging (or along with it).

What do you mean by source tree level dependencies? If you mean that each source tree can declare it's own dependencies, then yes.

Can't maven be mixed with ant to create this type of thing?

Yes! This is an important feature. As stringent as Maven is regarding filesystems, it does allow ant scripting to be put inline. This allows for leeway when shenanigans take place in the build. An example of how this could be used is in situations which it doesn't make sense to split a source tree up, but you want 3 outputs, each of which are pretty similiar.

Maven would by default build a jar which contains all of the compiled source (plus your includes/excludes) and you could then use ant to create your other artifacts.

[ryan.campbell]

There are circular dependencies between packages.

For instance:

org.jboss.logging.filter.TCLFilter -imports-> org.jboss.util.collection.WeakSet -imports-> org.jboss.util.WeakObject

In the util package, org.jboss.util.Base64 imports org.jboss.logging.Logger.

So there is no way to just pull out the logging package without resolving these circular package dependencies, correct?

IE, org.jboss.util will need to be split between a jar which depends on org.jboss.logging and one that org.jboss.logging can depend on.

[ryan.campbell]

So to solve the above example we would need:

jboss-commons-util-core.jar
org.jboss.util.WeakObject
org.jboss.util.collection.WeakSet

jboss-logging.jar
org.jboss.logging.*

jboss-common-util-more.jar
org.jboss.util.Base64

[adrian.brock]

There are circular dependencies between packages.

For instance:

org.jboss.logging.filter.TCLFilter -imports-> org.jboss.util.collection.WeakSet -imports-> org.jboss.util.WeakObject

This is just broken and needs fixing anyway.

There are a number of issues in jboss-common with packages referencing each other.

e.g. I remember one problem (now fixed) where you couldn't run JBossMQ in an applet
because it was referencing Strings for isJavaIdentifier()
which in turn referenced via the unrelated property replacement method the property classes
which in turn tried to automagically install/bootstrap the property manager
and also does System.setProperties()
and also then tried to bootstrap the url handling.

There is a discussion thread and some JIRA tasks about it somewhere.

[adrian.brock]

So to solve the above example we would need:

jboss-commons-util-core.jar
org.jboss.util.WeakObject
org.jboss.util.collection.WeakSet

jboss-logging.jar
org.jboss.logging.*

jboss-common-util-more.jar
org.jboss.util.Base64

No there should be:
jboss-logging-core.jar (just the contract no dependencies - usable everywhere)
jboss-logging-log4j.jar
jboss-logging-jdk.jar
etc.

[ryan.campbell]

Ok. Back to the subject of the present thread, we are goinig to pursue not making any structural changes to the source tree for maven2. We know the above modularizations need to be made, but they should be discussed and implemented along a timeline which is incompatible with rolling out a maven2 build.

Furthermore, Ruel has created a seperate source tree in svn. This is just to prevent any maven2 work from cluttering up cvs histories more than it already has. This is a shortlived tree until the maven2 work is ready for inclusion into the main source line.

Ruel, please post the svn info so we can see the progress you've documented in the roadmap.

[adrian.brock]

I'm going to look at this properly when you've finished.
I have an open mind and I will look at what you produce and how
usable it is for development, but the following is my historical view:

I have continually rejected using Maven for builds besides the fact it is
steaming pile of ***, it is not what I want from the build.

As usuall I am ignored, so I will ignore this :-)
although I did miss one call about it :-(

I want something that is declarative, not a scripting langauge.

I don't know about Maven2, Maven1 was more of a tool for building
"project websites" based on standard project source layouts.

Anything other than trivial builds, requried lots of horrible scripting
and "jumping through hoops".

I am also scepticle that we should rely on this tool for something as key
as our build procedure.

[ryan.campbell]

I want something that is declarative, not a scripting langauge.
...
Anything other than trivial builds, requried lots of horrible scripting
and "jumping through hoops".

There is no jelly scripting in Maven2, just pure Java plugins which traverse the model. The only horrible scripting will be some inline Ant, if needed.

[starksm64]

For me its just a tradeoff between the maven view of the build process, vs writing our own framework. Whatever provides the best usability/maintainability ratio is what we need to go with.

[ruel.loehr]

I have the minimal server building locally with maven2. In order to make this usable by others, I will need to add some dependencies and plugins into our repository. As this is *experimental* i have added a maven2 folder which will house these items. The reason for this is that a slightly different repository structure is required.

http://repository.jboss.com/maven2/

I will post further usage instructions once this is complete and tested.

[ruel.loehr]

I have completed a first revision of the minimal server build using Maven2 and tested it locally and in the QA lab. We have housed this in a JBoss Labs project.

To run this example you will need two things:
1) Maven2
2) Subversion

Maven 2 can be downloaded from http://maven.apache.org/download.html. Install the appropriate distro and add it to your path (see instructions on maven download page).

The Subversion client can be downloaded from
http://subversion.tigris.org/project_packages.html
You can select the binary particular to your distro. Again, you will need to add the svn executable to your path.

I have opened tickets to get both Maven and Subversion setup for all users in the QA lab.

Once these programs are installed you can issue the following commands to build:

svn co https://svn.labs.jboss.com/trunk/labs/jbossbuild/projects/jboss/ buildtest
cd buildtest
cd build
mvn install

This will execute the install phase of the build lifecycle as well as all prior phases for each of the defined modules. The first time the build executes it will download any necessary plugins needed as well as dependencies. Later builds are much speedier.

Once the build is completed:

cd ../dist/jboss-5.0-SNAPSHOT/bin
./run.sh -c minimal

To kick off the discussion, notice that each module contains a pom.xml file. This is the required buildfile. The modules build, common, j2se, are heavy on comments to give you a quicker idea of how things work.

The build/pom.xml file defines which modules are to be built as well as any project wide settings. The child pom's contain directions for performing specific build tasks (e.g. RMIC, javacc).

Basically, anywhere within these files you see ant scripting indicates that the project is a good candidate for further division or that a plugin should be developed to address the needs of the module (e.g building the entire jboss deployment after the build has finished). These plugins will come in later revisions.

With this iteration the source tree was not disturbed. The only exception to this was the module deploy which was added to hold the output of the build (as a child project). Maven's default assembly plugin does not handle tasks of assembling project artifacts in a sufficient manner for our needs. Once a proper assembly plugin has been created, this portion of the project can be removed and the build output can returned to the correct location.

[starksm64]

I have built this. Its not building all of the modules that are pulled down. I wanted to look at this before changing the current security/server dependencies. I don't understand the server/pom.xml, for example:

<project>
 <modelVersion>4.0.0</modelVersion>
 <groupId>org.jboss.server.transaction</groupId>

what is the groupId?

How are these jboss module dependencies being handled?

 <dependency>
 <groupId>org.jboss.server.common</groupId>
 <artifactId>jboss-common</artifactId>
 <version>5.0-SNAPSHOT</version>
 </dependency>

they are not in the maven repository so there is some correlation between build components?

Definitely need an overview of the maven pom.xml structure and tasks.

[starksm64]

As part of this we really need to cleanup the varia module as well. This is one that should be several source trees for unique artifacts, and some just dropped as obsolete unsupported stuff. One thing I want to move here from the security module is the srp stuff to break the dependency between the security and server modules.