1 2 Previous Next 25 Replies Latest reply on Jun 2, 2005 5:29 PM by Adrian Brock

    Requirements - Top level build - versioning

    Adrian Brock Master

      This gets a bit complicated because there are lots of different versions,
      I'll try to enumerate them:

      1) The top level build version e.g. 4.0.2
      2) The top level build branch e.g. JBoss4
      3) The cvs version e.g. JBoss_4_0_2
      4) The cvs branch e.g. Branch_4_0
      5) The component version (if it is standalone or thirdparty)
      6) The component specification version e.g. JMS1.1
      7) Patch versions e.g. 4.0.2-JBAS1234
      8) Different builds for different JVMs 1.3/1.4/5

      When defining versions, the "head" description wants to be unspecific as possible
      where the top level is a federation of direct components like JBossAS and naming/transaction/jbossmq

      In other cases, the versions need to be specific. i.e. the JBossAS head branch uses log4j-1.2.8 until we say otherwise.

      Then there are the issues of compatiblity. e.g.
      JBossAS needs log4j-1.2.8 or does it need log4j-1.2.8+ or
      log4j-1.2.8-log4j-1.2.11
      or maybe JBossAS 4.0.2 needs log4j-1.2.8+ but not log4j-1.2.10
      etc.

      These issues fall into a number of different categories:
      1) Who is reponsible for the versions used
      a) the top level buld
      b) the component
      c) something else
      2) How are conflicts resolved
      a) by changing the top level build
      b) by changing the offending component
      c) other
      3) How does this version information propogate
      a) in the build scripts
      b) in the artifacts
      4) Who is responsible for verifying the information and how are problems signalled
      to the user
      a) the top level build
      b) cruise control
      c) testing

        • 1. Re: Requirements - Top level build - versioning
          Adrian Brock Master

          When it comes to the final build of a project however, you want it to be as
          explicit as possible.

          You want to record exactly which versions were used to build the components
          such that you can reproduce it later.

          Currently, this is done by tagging things in CVS including all the thirdparty jars.

          One equivalent to this would be to add a target to the build that allows one to
          "materialize" the dependencies. i.e. build scripts are constructed that explcitly state
          their thirdparty versions and CVS tags.

          • 2. Re: Requirements - Top level build - versioning
            Scott Stark Master

            The "materialize" target is what I want to try initially with errors/warnings about detected version differences being a byproduct of that target until we have a solid dependency mechanism with versioning that we trust to not get in the way of development.

            • 3. Re: Requirements - Top level build - versioning
              ryan.campbell Expert

               

              "adrian@jboss.org" wrote:

              1) Who is reponsible for the versions used
              a) the top level buld
              b) the component
              c) something else


              The solution we've discussed elsewhere is that a graph of dependencies should be built based on the union of dependencies from each component. So:

              1. Components define their version compatibilities
              2. The toplevel build chooses the version acceptable to all components?
              3. It also seems we have a requirement for the toplevel build to "peg" a component at a given version. For instance log4j as above, and of course any directly referenced components such as jms.



              2) How are conflicts resolved
              a) by changing the top level build
              b) by changing the offending component
              c) other


              There are 2 kinds of conflicts:

              1. Conflicts which occur during synchronization. That is, that two components can not agree on a depedency's version -- preventing synchronization from completing (Or would a conflict not prevent synchronization?).

              In this case, you could either resolve it by modifying one of the components to be more agreeable (ie, test it with the new version). Or you could "peg" the toplevel build to a given version of that dependency, which would remove the ambiguity?

              2. Conflicts that occur during materialization. This would produce warnings if a component is incompatible with a pegged version of another component, as a result of #1.



              • 4. Re: Requirements - Top level build - versioning
                Adrian Brock Master

                The problem with pegging a version in the top level build
                is that the components will move forward to later versions
                while the top level build remains on an old version.

                There is a conflict here:
                1) You need the pegging to communicate how a conflict is being resolved
                to other developers.
                2) You need the pegging at release time to unambigously describe the release.
                3) You also need to be aware that it needs unpegging later.
                4) You can later change the release to record that although it was originally
                tested with log4j-1.2.8, it also works just as well with 1.2.9

                This was my argument against holding versions in the build files
                and instead recording which artificats have been tested twith which versions
                of a dependency.
                Who is going to (or even knows when) a version should be unpegged.

                • 5. Re: Requirements - Top level build - versioning
                  ryan.campbell Expert

                  Ok, so let me try to make this concrete to see if I understand.

                  1. Only direct dependencies are stored in the toplevel build (eg. jms1.1)

                  2. A developer can call the "materialize" target to traverse all the component-infos and generate a materialized-versions.xml. If there are conflicts, they will need to be resolved manually. This file is not source controlled, it is a product of the build.

                  3. Once conflicts are resolved, this file can then be copied to a source controlled file, versions.xml. This file effectively pegs components at specific versions. So usually, a developer will just check out this file and not worry about the above process.

                  4. We have an additional target "versions-report" which compares the materialized versions to the pegged versions. Cruisecontrol calls this at the end of each build and includes the results in the build email:

                  Error: log4j is pegged at 1.2.8, but aop (1.0.2) is compatable with log4j 1.2.6, 1.2.7 or 1.2.9.
                  Warning: log4j is pegged at 1.2.8, but cache (1.2.30), remoting (1.1.3) are also compatible with 1.2.9.

                  So this emailed report would trigger the discussion and would then be resolved by a developer starting with step #1.

                  • 6. Re: Requirements - Top level build - versioning
                    Adrian Brock Master

                     

                    "ryan.campbell@jboss.com" wrote:

                    1. Only direct dependencies are stored in the toplevel build (eg. jms1.1)


                    Ideally, only primary components are in the top level build during normal development.
                    e.g. JBossAS contains naming, jbossmq, etc. it does not define log4j because
                    that is a secondary dependency introducted by common/logging.
                    (I haven't created a usecase discussion for component builds yet...)


                    • 7. Re: Requirements - Top level build - versioning
                      Adrian Brock Master

                       

                      "ryan.campbell@jboss.com" wrote:

                      2. A developer can call the "materialize" target to traverse all the component-infos and generate a materialized-versions.xml. If there are conflicts, they will need to be resolved manually. This file is not source controlled, it is a product of the build.


                      No, it is very important that this is versioned in cvs. It defines the release.
                      The "materialize" should be done as a part of the release process.

                      I haven't discussed the release use cases yet either, but...

                      e.g.
                      1) 4.0.3beta - developers happily continue developing, maybe there are conflicts
                      which they discover through warnings from the build or through testing problems

                      2) pre-4.0.3final - the release manager or qa, materialize the dependencies in their own
                      local workspace - this fixes the release versions and also helps to discover conflicts
                      At this point, a discussion can start on conflict resolution

                      3) 4.0.3 final - the release manager branches cvs (allowing other developers to continue to use the lax build scripts for the next release - goto 5)
                      4) 4.0.3 final - the release manager commits the materialized build scripts and tags everything with the release id in cvs

                      5) 4.0.4 beta - see (1)

                      It is fairly obvious to me that steps 2 through 4 can be done with the help of build
                      targets.

                      (3) and (4) can be pretty much automated, e.g.
                      cd jbossas
                      ant make-release -Dtag=4.0.3final -Dnext=4.0.4beta
                      which is only done when the release manager is happy that (2) is complete
                      It could even upload the final binaries to sourceforge.

                      Of course, knowing cvs, and the fact that this is a human process (people make
                      mistakes) it is never quite that simple :-)

                      • 8. Re: Requirements - Top level build - versioning
                        ryan.campbell Expert

                         

                        "adrian@jboss.org" wrote:

                        Ideally, only primary components are in the top level build during normal development.
                        e.g. JBossAS contains naming, jbossmq, etc. it does not define log4j because
                        that is a secondary dependency introducted by common/logging.
                        (I haven't created a usecase discussion for component builds yet...)


                        Right, but during normal development, developers are still going to need to be able to build a release image for testing. Which means there will need to be one version of log4j selected somehow. So this target is not just for calling immediately before a release.

                        • 9. Re: Requirements - Top level build - versioning
                          Adrian Brock Master

                           

                          "ryan.campbell@jboss.com" wrote:

                          3. Once conflicts are resolved, this file can then be copied to a source controlled file, versions.xml. This file effectively pegs components at specific versions. So usually, a developer will just check out this file and not worry about the above process.

                          4. We have an additional target "versions-report" which compares the materialized versions to the pegged versions. Cruisecontrol calls this at the end of each build and includes the results in the build email:

                          Error: log4j is pegged at 1.2.8, but aop (1.0.2) is compatable with log4j 1.2.6, 1.2.7 or 1.2.9.
                          Warning: log4j is pegged at 1.2.8, but cache (1.2.30), remoting (1.1.3) are also compatible with 1.2.9.

                          So this emailed report would trigger the discussion and would then be resolved by a developer starting with step #1.


                          Yes, one of the original aims of the new build is to have a declartive build
                          such that additional tools can do extra analysis or processing.

                          • 10. Re: Requirements - Top level build - versioning
                            Adrian Brock Master

                             

                            "ryan.campbell@jboss.com" wrote:
                            "adrian@jboss.org" wrote:

                            Ideally, only primary components are in the top level build during normal development.
                            e.g. JBossAS contains naming, jbossmq, etc. it does not define log4j because
                            that is a secondary dependency introducted by common/logging.
                            (I haven't created a usecase discussion for component builds yet...)


                            Right, but during normal development, developers are still going to need to be able to build a release image for testing. Which means there will need to be one version of log4j selected somehow. So this target is not just for calling immediately before a release.


                            I'll talk about release issues in a separate thread, but in general developers will want to
                            build multiple release images (for testing different configurations)
                            and we will probably be moving to an installer based release where the user
                            will be creating the final release image themselves.
                            unzip jboss-4.0.4
                            cd jboss-4.0.4
                            install -config=default
                            or
                            install -config=all
                            or
                            install -config=jms
                            or
                            install // use a gui to create a custom config

                            • 11. Re: Requirements - Top level build - versioning
                              Scott Stark Master

                              On the point of where the version of an included component is defined, I don't see anywhere but the master build to define this. Another possible requirement for the repository is a notion of the LATEST version of a component, but this is just an optimisitic view that the next version, even if restricted to the latest point release of a series, will work. This is actually the mode we run in now as whatever has been checked into thirdparty on a given branch is what is going into the release.


                              4) You can later change the release to record that although it was originally tested with log4j-1.2.8, it also works just as well with 1.2.9

                              This was my argument against holding versions in the build files
                              and instead recording which artificats have been tested twith which versions of a dependency. Who is going to (or even knows when) a version should be unpegged.


                              I don't view this future compatibility issue as a useful requirement. So the 4.0.1 of jbossas was tested with log4j-1.2.8 and a user wants to upgrade to log4j-1.2.9. Say we test this as part of a support case and validate log4j-1.2.9 works. We certainly are not going to re-release 4.0.1 with an updated compatibility version stamp on the log4j component.


                              • 12. Re: Requirements - Top level build - versioning
                                Adrian Brock Master

                                 

                                "scott.stark@jboss.org" wrote:

                                I don't view this future compatibility issue as a useful requirement. So the 4.0.1 of jbossas was tested with log4j-1.2.8 and a user wants to upgrade to log4j-1.2.9. Say we test this as part of a support case and validate log4j-1.2.9 works. We certainly are not going to re-release 4.0.1 with an updated compatibility version stamp on the log4j component.


                                No we are not going to release the base package it is fixed for all time.
                                But users of the JBoss Network can get access to this information.

                                i.e. "log4j-1.2.9 is now available it contains the following fixes/enhancements, it has
                                been tested/validated against 4.0.1, 4.0.2, etc. click here to install"

                                Although, this usecase is probably more relevent for our packages.

                                • 13. Re: Requirements - Top level build - versioning
                                  Scott Stark Master

                                  Sure, but this is another layer of the as yet undefined patch release process that is adding information on top of the cast in stone jbossas 4.0.1 release build. I'm saying that right now I don't see a reason to not express the version dependencies in the master build. Certainly this should be overridable via local properties, and the LATEST notion can be useful for development builds, but there really is no other place for this version information.

                                  • 14. Re: Requirements - Top level build - versioning
                                    Adrian Brock Master

                                    So we are actually talking about two different issues.

                                    The first is to record in the repository that jgroups-2.2.8 has been tested with log4j-1.2.8 and log4-1.2.9
                                    The second that to reproduce the jboss-4.0.3 package release you need
                                    jgroups-2.2.8 and log4j-1.2.8

                                    I think "materializing" the versions in the top level build was the solution I proposed
                                    for the second requirement?
                                    My only issue was whether those versions remain fixed until somebody updates the
                                    top level build, or whether interim/development checkouts use "LATEST".
                                    I preferred the second option, but either will work.

                                    1 2 Previous Next