1 2 Previous Next 23 Replies Latest reply on Mar 3, 2008 11:18 AM by starksm64

    JBossAS CR1 build plan

    starksm64

      On today's call we still have confusion about what will be done for the CR1 build of jbossas, including exporting of artifacts to maven repos for use by other projects. The main point of confusion is related to how ejb3 is going to be built. It needs jbossas artifacts in maven repo, jbossas needs the ejb3 artifacts for the assembly.

      The issues to resolve are:
      1. What are the jbossas artifacts and how are they being uploaded to a maven repo as part of the build. One simple issue I already see is that the names of the artifacts are changed in the existing pom.xml files. For example in the system project, while the current build produces jboss-system-client.jar and jboss-system.jar artifacts, the system/pom.xml defines jboss-as-system.jar.

      We need to get to consistent artifact names in both builds as it affects classpaths and configuration files, docs.

      2. Getting the jbossas build to use maven artifacts. Nothing currently validates the jbossas maven artifacts. The first step in doing this is to replace the legacy jbossbuild repo with the maven repo as the source for the thirdparty contents. How we do this using ivy, maven ant tasks, etc. needs to be defined and documented, and the existing build-thirdparty target replaced.

      The next step would be to replace the existing project dependencies (classpaths defined in tools/etc/buildmagic/modules.ent) with the same type of maven repo references as was done for thirdparty.

      At that point the build and testsuite would be based on the exportable artifacts.

      3. How is ejb3/jbossas going to be built with ejb3 as a separate project, that depends on jbossas artifacts. Logically jbossas/ejb3 are a collection of dependent projects. How this translates into a build procedure from tagged codebases is what needs to be defined for CR1. We won't have jbossas completely mavenized by that point, so how a reproducible build results is the question.

      4. Related to 3, do we need to restructure the jbossas project, including breaking it up into separate svn projects to support the creation of jbossas artifacts and the jbossas release assembly.

        • 1. Re: JBossAS CR1 build plan
          pgier

           

          "scott.stark@jboss.org" wrote:

          1. What are the jbossas artifacts and how are they being uploaded to a maven repo as part of the build. One simple issue I already see is that the names of the artifacts are changed in the existing pom.xml files. For example in the system project, while the current build produces jboss-system-client.jar and jboss-system.jar artifacts, the system/pom.xml defines jboss-as-system.jar.

          We need to get to consistent artifact names in both builds as it affects classpaths and configuration files, docs.



          I spent some time the last couple weeks trying to reproduce the various module artifacts from the ant build using the maven assembly plugin. I can reproduce the jars but it might be better to use different names than we currently use. The assembly plugin can create jars with the same names locally (what I tried doing at first), but when the deploy plugin deploys stuff to the maven repository it changes the names to fit the maven standard. I believe the reasoning for this has to do with the dependency resolution.

          So for example, instead of jboss-system-client.jar we'll end up with jboss-as-system-${version}-client.jar. Where "client" is a maven classifier that can be used to itdentify this jar as a dependency.
          (http://snapshots.jboss.org/maven2/org/jboss/jbossas/jboss-as-system/5.0.0-SNAPSHOT/)
          See build 8 for an example of the client jar.

          So while the names are not the same, I tried to follow a pattern in the naming conventions in order to make it easier to find the matching jars. Will this work?

          I can change jboss-as-system to jboss-system (and follow the pattern for the rest of the artifacts) if it helps. I just chose that convention because I thought it would make it easier to identify app server component vs. mc components, aop components, etc.


          • 2. Re: JBossAS CR1 build plan
            pgier

             

            "scott.stark@jboss.org" wrote:

            2. Getting the jbossas build to use maven artifacts. Nothing currently validates the jbossas maven artifacts. The first step in doing this is to replace the legacy jbossbuild repo with the maven repo as the source for the thirdparty contents. How we do this using ivy, maven ant tasks, etc. needs to be defined and documented, and the existing build-thirdparty target replaced.

            The next step would be to replace the existing project dependencies (classpaths defined in tools/etc/buildmagic/modules.ent) with the same type of maven repo references as was done for thirdparty.

            At that point the build and testsuite would be based on the exportable artifacts.


            Yes, this is basically my current goal with the mavenization, but I probably should have done a better job of communicating this.

            I just wanted to point out that ivy or maven ant tasks would be needed only for the testsuite. My plan is to create a new testsuite which is basically a copy of the current one except that the classpath points to the maven generated jars instead of the ant ones.

            We wouldn't need to modify the existing ant build because once the testsuite works for the maven build, we won't need the existing build anymore.




            • 3. Re: JBossAS CR1 build plan
              pgier

               

              "scott.stark@jboss.org" wrote:

              3. How is ejb3/jbossas going to be built with ejb3 as a separate project, that depends on jbossas artifacts. Logically jbossas/ejb3 are a collection of dependent projects. How this translates into a build procedure from tagged codebases is what needs to be defined for CR1. We won't have jbossas completely mavenized by that point, so how a reproducible build results is the question.

              4. Related to 3, do we need to restructure the jbossas project, including breaking it up into separate svn projects to support the creation of jbossas artifacts and the jbossas release assembly.


              Once ejb3 is mavenized it can still be deployed to the legacy repository using the jboss-deploy plugin or manually. Then the existing build can use the artifacts.

              Maybe Andrew can provide some input on this since he is working on the EJB3 mavenization now.



              • 4. Re: JBossAS CR1 build plan
                wolfc

                 

                "pgier" wrote:
                I spent some time the last couple weeks trying to reproduce the various module artifacts from the ant build using the maven assembly plugin. I can reproduce the jars but it might be better to use different names than we currently use. The assembly plugin can create jars with the same names locally (what I tried doing at first), but when the deploy plugin deploys stuff to the maven repository it changes the names to fit the maven standard. I believe the reasoning for this has to do with the dependency resolution.

                So for example, instead of jboss-system-client.jar we'll end up with jboss-as-system-${version}-client.jar. Where "client" is a maven classifier that can be used to itdentify this jar as a dependency.
                (http://snapshots.jboss.org/maven2/org/jboss/jbossas/jboss-as-system/5.0.0-SNAPSHOT/)
                See build 8 for an example of the client jar.

                So while the names are not the same, I tried to follow a pattern in the naming conventions in order to make it easier to find the matching jars. Will this work?

                That's fine for the repo, but in the release there should be no version in the jar name. jbossall-client.jar will contain a Class-Path (soon) which has all jar names in it. I don't want to re-jar because we do a component update. (That's the thing we want to get rid of.)
                "pgier" wrote:
                I can change jboss-as-system to jboss-system (and follow the pattern for the rest of the artifacts) if it helps. I just chose that convention because I thought it would make it easier to identify app server component vs. mc components, aop components, etc.

                Personally I don't care if it's called jboss-as-system or jboss-system. We need to redo the class path entries into our build anyway.

                • 5. Re: JBossAS CR1 build plan
                  wolfc

                   

                  "pgier" wrote:
                  I just wanted to point out that ivy or maven ant tasks would be needed only for the testsuite. My plan is to create a new testsuite which is basically a copy of the current one except that the classpath points to the maven generated jars instead of the ant ones.

                  We wouldn't need to modify the existing ant build because once the testsuite works for the maven build, we won't need the existing build anymore.

                  Let's guinea pig EJB 3 for this. We can use the ejb3-core test suite as a blue print for how AS test suite is going to work. It has all aspects in there (starting/stopping instances of JBoss, doing deployment and unit tests etc).

                  • 6. Re: JBossAS CR1 build plan
                    starksm64

                     

                    "pgier" wrote:

                    (http://snapshots.jboss.org/maven2/org/jboss/jbossas/jboss-as-system/5.0.0-SNAPSHOT/)
                    See build 8 for an example of the client jar.

                    So while the names are not the same, I tried to follow a pattern in the naming conventions in order to make it easier to find the matching jars. Will this work?

                    I can change jboss-as-system to jboss-system (and follow the pattern for the rest of the artifacts) if it helps. I just chose that convention because I thought it would make it easier to identify app server component vs. mc components, aop components, etc.

                    Whatever names we choose, we need to have both builds using them. I'd rather see jbossas as the common prefix. There is still the problem of jars in configurations. These should not have versions in the artifact names so that we have to change config files, jar manifests, docs, etc. We can drop the version in the jar in the local target, but not from the repository without writing a new repository plugin. What can we do about this disconnection in naming conventions?


                    • 7. Re: JBossAS CR1 build plan
                      starksm64

                       

                      "pgier" wrote:

                      Once ejb3 is mavenized it can still be deployed to the legacy repository using the jboss-deploy plugin or manually. Then the existing build can use the artifacts.

                      Maybe Andrew can provide some input on this since he is working on the EJB3 mavenization now.

                      But the problem is that there is a chicken & egg problem. jbossas-5.0.0.GA needs ejb3 2.1.x, which needs jbossas-5.0.0.GA SPI jars. If both were mavenized, the dependencies would handle this by having access to all projects during the build. Since jbossas is not mavenized, we have to build the spi artifacts, uploaded to a maven repo, build the ejb3 artifacts, upload to a maven repo, build the ejb3 dependent artifacts of jbossas, build the jbossas assembly product. Its getting this procedure defined/automated that I'm wanting to get defined for jbossas-5.0.0.CR1.


                      • 8. Re: JBossAS CR1 build plan
                        starksm64

                         

                        "pgier" wrote:

                        Yes, this is basically my current goal with the mavenization, but I probably should have done a better job of communicating this.

                        I just wanted to point out that ivy or maven ant tasks would be needed only for the testsuite. My plan is to create a new testsuite which is basically a copy of the current one except that the classpath points to the maven generated jars instead of the ant ones.

                        We wouldn't need to modify the existing ant build because once the testsuite works for the maven build, we won't need the existing build anymore.

                        The problem is I don't know that we can wait for mavenization of jbossas for CR1. We can't have an unusable trunk for any amount of time (other than next week) as there is too much work to do. We also can't have the tck runs impacted. So what I'm suggesting is that start using the maven repo with the current ant build to validate artifacts, classpaths, eclipse project files, etc. Once that is working, how everything is mavenized can be decided.


                        • 9. Re: JBossAS CR1 build plan
                          wolfc

                           

                          "pgier" wrote:
                          "scott.stark@jboss.org" wrote:

                          3. How is ejb3/jbossas going to be built with ejb3 as a separate project, that depends on jbossas artifacts. Logically jbossas/ejb3 are a collection of dependent projects. How this translates into a build procedure from tagged codebases is what needs to be defined for CR1. We won't have jbossas completely mavenized by that point, so how a reproducible build results is the question.

                          4. Related to 3, do we need to restructure the jbossas project, including breaking it up into separate svn projects to support the creation of jbossas artifacts and the jbossas release assembly.


                          Once ejb3 is mavenized it can still be deployed to the legacy repository using the jboss-deploy plugin or manually. Then the existing build can use the artifacts.

                          This is not the problem, the problem is the circular dependency.
                          "pgier" wrote:
                          Maybe Andrew can provide some input on this since he is working on the EJB3 mavenization now.

                          Alternative strategy: let's demote jbossas/trunk to being a component. We remove jbossas/trunk/ejb3 and build & tag it as CR1. Now the repository contains all jboss-as-components at CR1. EJB 3 builds & tags as CR1, repository has jboss-ejb3-components. Then we take a new project with whatever name we come up with. Which is an assembly of jboss-as-components, jboss-ejb3-plugin, jboss-ws-plugin etc with a complete integration test suite. That new project will be the JBoss AS community edition. (Then EAP is either another project like this or another version of the same project. SOA-P will be another one (either sucking in EAP or AS-Community).)
                          I think we tackle 3 & 4 with 1 stone then (shouldn't there be a bird in there somewhere).

                          • 10. Re: JBossAS CR1 build plan
                            wolfc

                             

                            "scott.stark@jboss.org" wrote:
                            But the problem is that there is a chicken & egg problem. jbossas-5.0.0.GA needs ejb3 2.1.x, which needs jbossas-5.0.0.GA SPI jars. If both were mavenized, the dependencies would handle this by having access to all projects during the build. Since jbossas is not mavenized, we have to build the spi artifacts, uploaded to a maven repo, build the ejb3 artifacts, upload to a maven repo, build the ejb3 dependent artifacts of jbossas, build the jbossas assembly product. Its getting this procedure defined/automated that I'm wanting to get defined for jbossas-5.0.0.CR1.

                            Even if both were mavenized it still wouldn't work, because jbossas and ejb3 run with different aggregators. It needs to become and will from point on always be a multi-stage rocket.

                            • 11. Re: JBossAS CR1 build plan
                              dimitris

                               

                              "scott.stark@jboss.org" wrote:
                              The problem is I don't know that we can wait for mavenization of jbossas for CR1. We can't have an unusable trunk for any amount of time (other than next week) as there is too much work to do. We also can't have the tck runs impacted. So what I'm suggesting is that start using the maven repo with the current ant build to validate artifacts, classpaths, eclipse project files, etc. Once that is working, how everything is mavenized can be decided.

                              Just to stress out how important this is:

                              There are *only* 6 development weeks, after JBW for CR1, which means we have only the luxury of the "dead" JBW week to come up with a viable (even interim) solution.

                              Anything beyond has great chances of jeopartizing the whole plan.

                              So let's keep the forum discussion going, and if necessary get everyone on a confcall the comming Tuesday.

                              • 12. Re: JBossAS CR1 build plan
                                pgier

                                 

                                "scott.stark@jboss.org" wrote:
                                "pgier" wrote:

                                (http://snapshots.jboss.org/maven2/org/jboss/jbossas/jboss-as-system/5.0.0-SNAPSHOT/)
                                See build 8 for an example of the client jar.

                                So while the names are not the same, I tried to follow a pattern in the naming conventions in order to make it easier to find the matching jars. Will this work?

                                I can change jboss-as-system to jboss-system (and follow the pattern for the rest of the artifacts) if it helps. I just chose that convention because I thought it would make it easier to identify app server component vs. mc components, aop components, etc.

                                Whatever names we choose, we need to have both builds using them. I'd rather see jbossas as the common prefix. There is still the problem of jars in configurations. These should not have versions in the artifact names so that we have to change config files, jar manifests, docs, etc. We can drop the version in the jar in the local target, but not from the repository without writing a new repository plugin. What can we do about this disconnection in naming conventions?


                                It's not too difficult to write the jars without the version in the name. Then when they are deployed, they have the version. I think this will be clear as far as where jars are coming from.

                                However one problem is that I'm currently using a snapshot version of the maven assembly plugin. If I can't get an official release of the plugin on apache, then I can just for a version for us internally.

                                I'll plan to go through the jars and make this change early next week.




                                • 13. Re: JBossAS CR1 build plan
                                  alrubinger

                                   

                                  "pgier" wrote:
                                  Once ejb3 is mavenized it can still be deployed to the legacy repository using the jboss-deploy plugin or manually. Then the existing build can use the artifacts.


                                  To drive home this point, it's unfortunately more involved than that. :) I'll try to be concise (and repeat some key points).

                                  Problem 1:
                                  EJB3 depends on AS components, many of which do not have their own lifecycle;we're building off snapshots.

                                  Problem 2:
                                  Even if the AS components had an official release, they're tied to the AS build itself. So EJB3 depends on AS, which depends on EJB3. That's the circular dependency.

                                  Problem 3:
                                  Minimal time/resources to address everything in the most clean fashion before CR1. In tandem with TCK.

                                  So the goals as I see it, in order of priority:
                                  1) Resolve circular dependency issues (cannot cut proper release with status-quo)
                                  2) TCK
                                  3) Cleanliness

                                  I'll map out my out-of-scope Utopian wishlist as a jump-off point for debate and cutting back until we have something manageable for CR1.

                                  * Each AS component owner responsible for their own build
                                  * AS Build policy dictating that artifacts must be deployed to Maven repo
                                  * Each AS component in separate release cycle
                                  * Reorganization of AS in SVN to something like: jbossas/components/componentName/[trunk|branches|tags]
                                  * Creation of new AS Assembly Project to bring in all dependent components and package into distribution (The EJB3 Plugin does this currently)
                                  * Centralized POM tracking current versions of each AS component, to be used both by AS Assembly and outside projects (ie. EJB3)

                                  AFAICT, discussion of artifact names and embedded versions seems minute compared with the overall task of defining how we can resolve the above for CR1.

                                  Carlo's suggestion to treat the existing AS trunk as a unified "component", adding the additional AS Assembly has a lot of merit. This way we trim the fat of separately componentizing each module and have a well-defined version upon which EJB3 can depend. The cleanup and reorg can then be shuffled off to post-CR1.

                                  S,
                                  ALR

                                  • 14. Re: JBossAS CR1 build plan
                                    starksm64

                                     

                                    "wolfc" wrote:

                                    Even if both were mavenized it still wouldn't work, because jbossas and ejb3 run with different aggregators. It needs to become and will from point on always be a multi-stage rocket.

                                    I don't follow why. Sure, both product releases would require separate runs of the top level aggregation entry point, but if both were mavenized, projects properly separated, running the top level jbossas release pom target should be able to build everything.

                                    In practice I think we should separate the release project from the base artifacts and one would have to run the build in stages. Is that what you mean?


                                    1 2 Previous Next