Moving jbossas to a maven repo for thirdparty

[starksm64]

One of the things touched on for the CR1 release build is getting to a single maven repository for thirdparty artifacts, dropping the usage of the jboss repo. Who has looked at ivy vs the maven ant tasks for this?

[pgier]

Just wanted to give an update on progress of trying to reproduce the thirdparty directory. I decided to work on a new plugin instead of trying to use the standard assembly plugin to do this for two reasons.

1. assembly plugin has no idea how to create libraries.ent
2. the assembly plugin's existing features don't give much advantage over the standard maven stuff available to all maven plugins (downloading dependencies, reading the pom, etc).

So far I don't see any technical issues. The only issue is that it will be a bit tedious and time consuming to configure a mapping between the maven names of artifacts and the names that the build scripts expect in the thirdparty folder. But so far, I don't see any reason why I can't have this working by next Friday.

I'll give another update probably Tuesday with progress on this.

[starksm64]

So the mapping would be configured something like:

 <plugin>
 ...
 <configuration>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-compiler-plugin</artifactId>
 <dependencies>
 <dependency>
 <groupId>org.jboss</groupId>
 <artifactId>jboss-common-core</artifactId>
 <mapping>
 <groupId>org.jboss</groupId>
 <artifactId>jboss-common-core</artifactId>
 </mapping>
 </dependency>
 <dependency>
 <groupId>org.jboss.microcontainer</groupId
 <artifactId>jboss-aop-mc-int</artifactId>
 <mapping>
 <groupId>jboss.microcontainer</groupId>
 <artifactId>jboss-aop-mc-int</artifactId>
 </mapping>
 </dependency>
 <dependency>
 <groupId>org.jboss.microcontainer</groupId>
 <artifactId>jboss-dependency</artifactId>
 <mapping>
 <groupId>jboss.microcontainer</groupId>
 <artifactId>jboss-dependency</artifactId>
 </mapping>
 </dependency>
...
 <dependency>
 <groupId>jboss</groupId
 <artifactId>jboss-cache</artifactId>
 <mapping>
 <groupId>jboss.cache</groupId>
 <artifactId>jbosscache</artifactId>
 </mapping>
 </dependency>
...
 </dependencies>
 </configuration>
 </plugin>

the problem I still see are the artifacts not in maven. For example, the jboss repo for jboss/cache includes a jbosscache.jar & pojocache.jar. I don't see any pojo cache artifact in maven. I guess these will have to be added to maven. I would put out a list of these first so that we can start adding them.

[pgier]

Unfortunately the mapping part will be separate from the dependencies, because the mapping will have to be a parameter to the plugin. I can't change the XML for the dependencies in the pom without hacking up and forking the maven code. I'll post an example tomorrow of what it looks like once I get a couple more of the details figured out.

I'm going to make this a new mojo (maven plugin class) in the current jboss-deploy plugin because they can share some code for writing component-info files and other stuff. I'm thinking about renaming the plugin to something like jboss-thirdparty plugin, but that's low priority.

I'll post artifacts that aren't in the repository as I find them. It shouldn't be too bad to add the ones that are missing. They don't have to be built with maven, they can just have a basic auto-generated pom. I had to add some stuff like that already when I was working on the app server poms that are there now.

[starksm64]

"pgier" wrote:
Unfortunately the mapping part will be separate from the dependencies, because the mapping will have to be a parameter to the plugin. I can't change the XML for the dependencies in the pom without hacking up and forking the maven code. I'll post an example tomorrow of what it looks like once I get a couple more of the details figured out.

Right, I was just reproducing the dependency info in the plugin config to allow the mapping to be clear.

"pgier" wrote:

I'm going to make this a new mojo (maven plugin class) in the current jboss-deploy plugin because they can share some code for writing component-info files and other stuff. I'm thinking about renaming the plugin to something like jboss-thirdparty plugin, but that's low priority.

Remember we really don't need the component-info.xml in the local thirdparty contents as we would not be using the jbossbuild repository mechanism any longer.

[pgier]

"pgier" wrote:
Unfortunately the mapping part will be separate from the dependencies, because the mapping will have to be a parameter to the plugin. I can't change the XML for the dependencies in the pom without hacking up and forking the maven code. I'll post an example tomorrow of what it looks like once I get a couple more of the details figured out.

Right, I was just reproducing the dependency info in the plugin config to allow the mapping to be clear.

Oh yeah, sorry, I didn't read your earlier post very carefully. It will look very similar to what you put above.

"pgier" wrote:

I'm going to make this a new mojo (maven plugin class) in the current jboss-deploy plugin because they can share some code for writing component-info files and other stuff. I'm thinking about renaming the plugin to something like jboss-thirdparty plugin, but that's low priority.

Remember we really don't need the component-info.xml in the local thirdparty contents as we would not be using the jbossbuild repository mechanism any longer.

It's not a big deal since I already had some small code for it from the jboss-deploy plugin.

[pgier]

Ok, I deployed the plugin to the snapshots repo. And I added a pom.xml to the thirdparty directory. This new pom has the basic plugin configuration, but I haven't finished adding all the dependencies yet.

You can run it with "mvn package". Currently it will put all the jars into the thirdparty/target directory, but once the configuration is complete, we can change that to output directly to the thirdparty directory. I just didn't wanted to be able to compare the two, so I didn't make it overwrite the existing stuff yet.

Please take a look and let me know if anything isn't working.

[starksm64]

I tried adding two new dependencies/mappings for mc jars into the jboss/microcontainer location to see how many to one type of mappings would work, and I'm seeing the pom.xml fail to validate because of missing dependency versions. I don't see them for the existing dependency elements, so is this because of the org.jboss.jbossas:jboss-as-parent missing them?

[starksm@succubus thirdparty]$ mvn package
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error building POM (may not be this project's POM).


Project ID: org.jboss.jbossas:jboss-as-thirdparty
POM Location: /home/svn/JBossHead/jboss-head/thirdparty/pom.xml
Validation Messages:

 [0] 'dependencies.dependency.version' is missing for org.jboss.microcontainer:jboss-classloader


Reason: Failed to validate POM for project org.jboss.jbossas:jboss-as-thirdparty at /home/svn/JBossHead/jboss-head/thirdparty/pom.xml


[INFO] ------------------------------------------------------------------------[INFO] Trace
org.apache.maven.reactor.MavenExecutionException: Failed to validate POM for project org.jboss.jbossas:jboss-as-thirdparty at /home/svn/JBossHead/jboss-head/thirdparty/pom.xml
 at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:378)
 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:290)
 at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:125)
 at org.apache.maven.cli.MavenCli.main(MavenCli.java:280)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
 at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
 at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
 at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.project.InvalidProjectModelException: Failed to validate POM for project org.jboss.jbossas:jboss-as-thirdparty at /home/svn/JBossHead/jboss-head/thirdparty/pom.xml
 at org.apache.maven.project.DefaultMavenProjectBuilder.processProjectLogic(DefaultMavenProjectBuilder.java:998)
 at org.apache.maven.project.DefaultMavenProjectBuilder.buildInternal(DefaultMavenProjectBuilder.java:801)
 at org.apache.maven.project.DefaultMavenProjectBuilder.buildFromSourceFileInternal(DefaultMavenProjectBuilder.java:479)
 at org.apache.maven.project.DefaultMavenProjectBuilder.build(DefaultMavenProjectBuilder.java:200)
 at org.apache.maven.DefaultMaven.getProject(DefaultMaven.java:553)
 at org.apache.maven.DefaultMaven.collectProjects(DefaultMaven.java:467) at org.apache.maven.DefaultMaven.getProjects(DefaultMaven.java:364)
 ... 11 more

The example fragments I added were:

 <mappedDependencies>
 <dependency>
 <groupId>corg.jboss.microcontainer</groupId>
 <artifactId>jboss-aop-mc-int</artifactId>
 <mapping>
 <componentId>jboss/microcontainer</componentId>
 <artifactId>jboss-aop-mc-int</artifactId>
 </mapping>
 </dependency>
 <dependency>
 <groupId>corg.jboss.microcontainer</groupId>
 <artifactId>jboss-classloader</artifactId>
 <mapping>
 <componentId>jboss/microcontainer</componentId>
 <artifactId>jboss-classloader</artifactId>
 </mapping>
 </dependency>
 </mappedDependencies>
...
 <dependency>
 <groupId>org.jboss.microcontainer</groupId>
 <artifactId>jboss-aop-mc-int</artifactId>
 </dependency>
 <dependency>
 <groupId>org.jboss.microcontainer</groupId>
 <artifactId>jboss-classloader</artifactId>
 </dependency>

[pgier]

Yes, I put the versions for the other ones in the parent to make sure the same dependency versions are being used everywhere. If you add the versions to the parent it should fix the problem.

[pgier]

I haven't finished duplicating all the thirdparty directory yet, but I should be able to finish it today. So the next things to do are:

1. Make sure the testsuite works against the thirdparty with maven artifacts.
2. Change the testsuite to use the maven generated module artifacts. And make sure that everything still works.

What still needs to be done after this?

[starksm64]

So what I was talking about on the call was just updating the libraries.ent that was being included in the testsuite/build.xml and build/build-distr.xml to include the ../thirdparty/target/ versions:

testsuite/build.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project [
 <!ENTITY libraries SYSTEM "../thirdparty/target/libraries.ent">
 <!ENTITY testsuite-libraries SYSTEM "../thirdparty/target/testsuite-libraries.ent">
 <!ENTITY modules SYSTEM "../tools/etc/buildmagic/modules.ent">
]>

<project default="main" name="JBoss/Testsuite"

build/build-distr.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE project [
 <!ENTITY libraries SYSTEM "../thirdparty/target/libraries.ent">
]>

<project name="JBoss/Build">

If you want to not even affect the existing build files, then have the thirdparty maven package target replace the thirdparty/libraries.ent and testsuite-libraries.ent with updated versions that refer to the thirdparty/target/* jars.

[pgier]

Just wanted to give an update on the progress of this. I have most of the thirdparty repo generated, now I'm trying to build against the maven generated thirdparty directory. And I'm still finding some things that don't match.

I think that by mid next week, I should have it all working. My plan is to change the main build script to call my maven thirdparty build instead of the current build-thirdparty.xml. Then the ant script won't be needed any more. At this point build-thirdparty.xml should no longer be maintained, and in order to update dependency versions everyone will have to update the parent pom and/or the thirdparty/pom.xml.

Instead of modifying the build scripts to point to "target", I discussed with Rajesh and it seems like it will be better just to generate the thirdparty output directly to the thirdparty directory and overwrite what is there now. This way the only modification needed to the build.xml files is to turn on "inhibit.downloads". And then make the call to thirdparty "mvn package" automatic just for convenience.

Does this sound reasonable?

[pgier]

I have a question about one of the dependencies. There is a jar file called snmptrapappender.jar. It is located in the repository here http://repository.jboss.org/snmptrapappender/. It appears to be a dependency of apache-log4j in our thirdparty repo.

This jar is not in the central maven repo. Should I add it to our repo? Can this be replaced by anything else?

[starksm64]

"pgier" wrote:

Does this sound reasonable?

Yes.

[starksm64]

"pgier" wrote:
I have a question about one of the dependencies. There is a jar file called snmptrapappender.jar. It is located in the repository here http://repository.jboss.org/snmptrapappender/. It appears to be a dependency of apache-log4j in our thirdparty repo.

This jar is not in the central maven repo. Should I add it to our repo? Can this be replaced by anything else?

It looks like Fernando created a jpackage bundle for it:
http://www.jpackage.org/browser/rpm.php?jppversion=1.7&id=6100

The link to the http://www.m2technologies.net/asp/snmpTrapAppender.asp is not responding, so I guess its defunct. I don't know of a replacement.