1 2 Previous Next 16 Replies Latest reply on Apr 15, 2005 1:06 PM by starksm64

    New Build - Standalone module - multiple builds

      Hi Ryan,

      Is this the task for changing the build so it doesn't depend
      upon the top level build?
      http://jira.jboss.com/jira/browse/JBBUILD-16

      i.e. the project "imports" the dependent projects (including binary projects
      downloaded from the repository into thirdparty).

      e.g. For Tom wants to create a standalone JBossRemoting
      project that will include other JBoss projects, but these projects
      reference the top level JBossAS for their artifact definitions.

      Did you test the ant parsing of other build.xml?

        • 1. Re: New Build - Standalone module - multiple builds

          Yes, that is the one, but it doesn't include our convesation.

          Yes, I did test the parsing of the build.xml. The main problem I ran into was that of the ids of the various imported components conflicting. IE, every component has an source with an ID of "main", so trying to import a half dozen components is going to end up with Ant overriding the reference to main a half dozen times.

          The solution to this, I think, is to have two files in each component directory. One is the component-info.xml (better name?) which contains the external metadata (interface) of each component, as outlined in the issue you refer to. It would also include the "exports" that we talked about at the dev conference. The other would be the build.xml which includes the componentdefs as now.

          Do you see any problems with seperating them as I suggest? IE, instead of remoting importing common/build.xml, it would import common/component-info.xml instead?

          • 2. Re: New Build - Standalone module - multiple builds

            Yes, the split makes sense.

            1) It clearly separates the external definition from the internal implementation

            2) Nobody is likely to be interested in how a jar is constructed or which source
            is used, only the end product. So the duplicate

            <source id="main"/>
            

            should be irrelevent.

            3) It means the repository is not being continually updated with minor
            internal tweaks to the module builds.



            The other solution would be to use xml namespaces.

            • 3. Re: New Build - Standalone module - multiple builds

               

              "adrian@jboss.org" wrote:

              The other solution would be to use xml namespaces.


              I actually think namespace is technically a much better solution. However, I think will be more difficult for people to learn/use, so would not go with it because will lead to more build errors.

              So where does this leave things for JBossRemoting build? I obviously can't take the jbossbuild.xml from jboss-head/remoting/ directory and expect it to work as is.


              • 4. Re: New Build - Standalone module - multiple builds

                Ok, here is the plan for the standalone remoting build:

                1. Implement the export/import funcationality described above.
                2. Add remoting's thirdparty deps to cruisecontrol.jboss.com/repository
                3. Create a standalone remoting cvs alias which includes remoting and tools (for jbossbuild)
                4. Create a release.xml in the jboss-remoting module which describes the release structure of remoting.

                This will allow Tom to create a standalone release of remoting. I think all of this will take about a week.

                This doesn't address the issue of integrating the remoting jar back into head. As far as I can tell, this will have to wait until the new build is complete in head and is downloading dependencies. This looks like the beginning of May to me (milestone-3).

                • 5. Re: New Build - Standalone module - multiple builds

                  For the export functionality, we want to keep the syntax as simple as possible. So to include the jmx project in your classpath you would want to do:

                  <source id="main">
                   <include component="jmx"/>
                   </source>

                  This would look for a component-info.xml first in ../jmx and then in ../thirdparty/jmx. The component-info would need to have an export declaration:
                  <component id="jmx" ... >
                   <artifact id="jboss-jmx.jar"/>
                   <artifact id="jboss-jmx-core.jar"/>
                   <export>
                   <include input="jboss-jmx.jar"/>
                   </export>
                  </component>

                  So that jboss-jmx.jar would be included in the classpath of the compilation. However, I think you will still want the ability to use artifacts or other includes besides the exported ones:
                  <source id="main">
                   <include component="jmx" input="jboss-jmx-core.jar"/>
                  </source>
                  
                  This would cause the parsing as above, but would use the designated reference id instead of the component's exports. So the export statement is the default, but you retain the ability to reference an abitrary input.

                  To do this, I will need to add a component-info.xml for thirdparty/* in cvs. I assme there is no problem with this? We'll need these eventually, anyway. Basically, it will involve copying the existing thirdparty component declarations from jbossas/jbossbuild.xml into the component-info.xml and changing the {includes} to {export}.

                  • 6. Re: New Build - Standalone module - multiple builds

                     

                    "ryan.campbell@jboss.com" wrote:
                    For the export functionality, we want to keep the syntax as simple as possible.
                    <snip/>
                    This would cause the parsing as above, but would use the designated reference id instead of the component's exports. So the export statement is the default, but you retain the ability to reference an abitrary input.
                    [/qoute]

                    The includes or export is just a convenience such that each consuming project
                    does not need to be changed when the exported artifact definition changes.

                    The consuming project still has the option to be more explicit about what it
                    actually uses, but then that would point to a project that probably needs
                    splitting up????


                    To do this, I will need to add a component-info.xml for thirdparty/* in cvs. I assme there is no problem with this? We'll need these eventually, anyway. Basically, it will involve copying the existing thirdparty component declarations from jbossas/jbossbuild.xml into the component-info.xml and changing the {includes} to {export}.



                    Yes, that is exactly what I want. Each component (whether it is one our projects
                    or a true thirdparty project) should define its own artifacts
                    rather than the top level build. Doing it on the top level build stops
                    the project being included in multiple integration projects because it has
                    references back to a specific top level build.

                    • 7. Re: New Build - Standalone module - multiple builds

                       

                      "adrian@jboss.org" wrote:
                      "ryan.campbell@jboss.com" wrote:
                      For the export functionality, we want to keep the syntax as simple as possible.
                      <snip/>
                      This would cause the parsing as above, but would use the designated reference id instead of the component's exports. So the export statement is the default, but you retain the ability to reference an abitrary input.


                      The includes or export is just a convenience such that each consuming project
                      does not need to be changed when the exported artifact definition changes.

                      The consuming project still has the option to be more explicit about what it
                      actually uses, but then that would point to a project that probably needs
                      splitting up????


                      To do this, I will need to add a component-info.xml for thirdparty/* in cvs. I assme there is no problem with this? We'll need these eventually, anyway. Basically, it will involve copying the existing thirdparty component declarations from jbossas/jbossbuild.xml into the component-info.xml and changing the {includes} to {export}.



                      Yes, that is exactly what I want. Each component (whether it is one our projects
                      or a true thirdparty project) should define its own artifacts
                      rather than the top level build. Doing it on the top level build stops
                      the project being included in multiple integration projects because it has
                      references back to a specific top level build.

                      • 8. Re: New Build - Standalone module - multiple builds

                         

                        "ryan.campbell@jboss.com" wrote:
                        For the export functionality, we want to keep the syntax as simple as possible.
                        (snip)
                        This would cause the parsing as above, but would use the designated reference id instead of the component's exports. So the export statement is the default, but you retain the ability to reference an abitrary input.


                        The includes or export is just a convenience such that each consuming project
                        does not need to be changed when the exported artifact definition changes.

                        The consuming project still has the option to be more explicit about what it
                        actually uses, but then that would point to a project that probably needs
                        splitting up????


                        To do this, I will need to add a component-info.xml for thirdparty/* in cvs. I assme there is no problem with this? We'll need these eventually, anyway. Basically, it will involve copying the existing thirdparty component declarations from jbossas/jbossbuild.xml into the component-info.xml and changing the {includes} to {export}.



                        • 9. Re: New Build - Standalone module - multiple builds

                           

                          "ryan.campbell@jboss.com" wrote:
                          For the export functionality, we want to keep the syntax as simple as possible.
                          <snip/>
                          This would cause the parsing as above, but would use the designated reference id instead of the component's exports. So the export statement is the default, but you retain the ability to reference an abitrary input.


                          The includes or export is just a convenience such that each consuming project
                          does not need to be changed when the exported artifact definition changes.

                          The consuming project still has the option to be more explicit about what it
                          actually uses, but then that would point to a project that probably needs
                          splitting up????


                          To do this, I will need to add a component-info.xml for thirdparty/* in cvs. I assme there is no problem with this? We'll need these eventually, anyway. Basically, it will involve copying the existing thirdparty component declarations from jbossas/jbossbuild.xml into the component-info.xml and changing the {includes} to {export}.


                          Yes, that is exactly what I want. Each component (whether it is one our projects
                          or a true thirdparty project) should define its own artifacts
                          rather than the top level build. Doing it on the top level build stops
                          the project being included in multiple integration projects because it has
                          references back to a specific top level build.

                          • 10. Re: New Build - Standalone module - multiple builds

                            One result of not importing a toplevel build is that attributes which were previously set on the toplevel build element now must be set elsewhere. Examples of this are the thirdparty location, implTitle, targetdefs, repository location, etc. For most of these, I have created properties, stored in tools/etc/jbossbuild.properties.

                            However, for the manifest data like implTitle and implVersion, I am less sure how to proceed. One natural place is to set these in the component-info for a given component. However, when we do a release, we want the jars to contain the same manifest information -- not to vary by component. Perhaps we could use http://jira.jboss.com/jira/browse/JBBUILD-5 to modify all jar manifests, including non-thirdparty?

                            This is not stopping any work, just looking for input while its on my mind.

                            • 11. Re: New Build - Standalone module - multiple builds

                              Ok. So the problem now is that we need somewhere to specify project defaults,
                              like the repository location and top level release id/version.

                              In some circumstances, you probably wouldn't want the top level defining the version,
                              e.g. aop-1.1 in 3.2.x or 4.0.x is still aop-1.1
                              You probably still want to know the top level build that built/included the file?
                              I think we need to define what information we need in the manifest and who
                              puts it there.
                              e.g. If I download a thirdparty jar, do I still tag it with my main release id? Probably yes.
                              e.g.2. does jbossmq get tagged with its own version besides the top level release
                              ids it is included in, probably yes
                              etc.

                              Don't forget that some projects/jars although source projects will actually
                              get included in multiple top level builds/versions as binaries, e.g. AOP or Cache.

                              In terms of the repository, I don't see a problem with each module defining its own
                              repository/cvs. The issue is keeping that in step with reality.
                              e.g. Suppose I copy a binary from one repository to another, who updates the xml?
                              Same problem with the cvs location.

                              I think this will be more important for the full synchronize?
                              e.g.
                              1) Download jbossas and tools
                              2) cd jbossas
                              3) ant synchronize
                              4) Download from the repository component info for each source project which gives the cvs info
                              5) checkout each project from cvs

                              e.g. Does a problem occur if the component info from cvs disagrees with the repository's version. Probably?

                              If everything is specfied correctly, there will be no problem.

                              • 12. Re: New Build - Standalone module - multiple builds

                                So just to rephrase your comments: Each component sets the implVersion and implTitle to the component level values (eg. aop 1.1). We want additional values for the toplevel release, maybe releaseVersion and releaseTitle.

                                I'm guessing the best place to set these two manifest values is as a part of the release target, using the Ant task from JBBUILD-5.


                                "adrian@jboss.org" wrote:

                                In terms of the repository, I don't see a problem with each module defining its own repository/cvs. The issue is keeping that in step with reality. e.g. Suppose I copy a binary from one repository to another, who updates the xml?


                                I guess you are right, components could theoretically have different repositories. However, this would seem rare for our usage at least. Since we don't want to have this duplicated in each file, maybe the default should be a property which can be overriden with an attriibute in component info.

                                "adrian@jboss.org" wrote:

                                e.g. Does a problem occur if the component info from cvs disagrees with the repository's version. Probably?


                                There is a problem if they differ, but not a fatal one. I give preference to the component-info from cvs.


                                • 13. Re: New Build - Standalone module - multiple builds
                                  starksm64

                                  One issue that is not clear in this discussion is do I have sufficient info to go obtain the source for every jar I find in the dist? Ultimately I want this for patches as well. The ideal roundtrip behavior is that I would point a ant task to a release and have it spit out the jboss-build.xml that would obtain the source and thirdparty repository to rebuild the possibly patched dist.

                                  • 14. Re: New Build - Standalone module - multiple builds

                                     

                                    "scott.stark@jboss.org" wrote:
                                    One issue that is not clear in this discussion is do I have sufficient info to go obtain the source for every jar I find in the dist? Ultimately I want this for patches as well. The ideal roundtrip behavior is that I would point a ant task to a release and have it spit out the jboss-build.xml that would obtain the source and thirdparty repository to rebuild the possibly patched dist.


                                    Right, you might have a distribution which has potentially reflects multiple patches, and now you want the source for this distribution. I think this is possible with the current design. How it would work:

                                    1. Iterate over all the jars in a release, collecting the component id & version from each jar manifest.
                                    2. Verify there are no conflicts between jars. IE, jboss-common.jar and jboss-common-client.jar must agree on their version. Otherwise, the source is not resolvable.
                                    3. Based on the above, you have enough information to create a toplevel build. *
                                    4. Upon calling synchronize for this toplevel build, the component id's and versions will be used to resolve the component-info from the build repository.
                                    5. The component-info will contain the cvsroot, module, and tags for each component. These data will then be used to checkout the source for each component.

                                    * This does not address non-archive artifacts, such as text files. We would need to add "manifests" to these text files if we wanted to be able to resolve their source components. Is this necessary for this use case (or in general)?


                                    1 2 Previous Next