14 Replies Latest reply on Feb 27, 2008 1:55 PM by adrian.brock

    ClassLoadingMetaData ease-of-use

      I'd like to hear people's opinions on the following issues
      with the new OSGi style classloading metadata.

      First some explanation.

      The classloading metadata looks like this:

      <classloading xmlns="urn:jboss:classloading:1.0">
      
      <!-- capabilities are classloading exports -->
       <capabilities>
      <!-- This deployment can be referenced by either of these two module names -->
       <module name="export2" version="2.0.0"/>
       <module name="export1" version="1.0.0"/>
      <!-- It exports these packages with the given versions -->
       <package name="export2" version="2.0.0"/>
       <package name="export1" version="1.0.0"/>
       </capabilities>
      <!-- requirements are classloading imports -->
       <requirements>
      <!-- It needs to import the following modules -->
       <module name="import2" from="2.0.0"/>
       <module name="import1" from="1.0.0"/>
      <!-- and packages -->
       <package name="import2" from="2.0.0"/>
       <package name="import1" from="1.0.0"/>
       </requirements>
      </classloading>
      


      OFF-TOPIC: The above is the deployment metadata, i.e. META-INF/jboss-classloading.xml where the "classpath" is already known
      from the deployment structure.
      There's a similar one for use in a -beans.xml, except
      in a different namespace because there you have to specify the
      roots as well, e.g.
      <deployment>
       <classloader><inject bean='MyClassLoader"/></classLoader>
      
       <classloader name="MyClassLoader" xmlns="urn:jboss:classloader:1.0">
       <!-- similar to above -->
      
       <root>${jboss.lib.url}/some.jar</root>
       <root>${jboss.lib.url}/another.jar</root>
       </classloader>
      
       <!-- beans here -->
      </deployment>
      


      For ease-of-use and "backwards compatibity" there are some shorthands.

      Module name and version shorthand which automatically
      creates a single module capability.
      <classloading name="test" version="1.0.0" xmlns="urn:jboss:classloading:1.0"/>
      


      Exporting all packages
      <classloading name="test" version="1.0.0" export-all="ALL"
      xmlns="urn:jboss:classloading:1.0"/>
      



      Exporting all packages with an inclusion filter
      i.e. only the packages listed are part of the classloader and exported
      <classloading name="test" version="1.0.0" export-all="ALL" included="org.jboss.package1,org.jboss.package2"
      xmlns="urn:jboss:classloading:1.0"/>
      


      Exporting all packages with an exclusion filter
      i.e. all packages are included in the classloader and exported except
      the ones listed
      <classloading name="test" version="1.0.0" export-all="ALL" excluded="org.jboss.package1,org.jboss.package2"
      xmlns="urn:jboss:classloading:1.0"/>
      


      Exporting all packages with an excluded export filter
      i.e. all packages are included in the classloader but the given classes
      are not exported
      <classloading name="test" version="1.0.0" export-all="ALL" excludedExport="org.jboss.package1,org.jboss.package2"
      xmlns="urn:jboss:classloading:1.0"/>
      


      In the above examples with filters, the packages are automatically
      created as "capabilities" with the same version as the module.

      Note: the exportAll and explicitly listing capabilities are mutually exclusive.
      ExportALL and its related filters are ignored if you list capabilities explicity

      NOW FOR THE QUESTIONS.

      1) Is the above "Note" about mutual exclusion of exportAll and listing
      capabilities correct? If not, it could get very messy trying to merge them. ;-)

      2) In the filter case, the packages automatically get the version of the module,
      but this doesn't happen for listing capabilities explicitly

      <classloading xmlns="urn:jboss:classloading:1.0">
       <capabilities>
       <module name="export1" version="1.0.0"/>
       <package name="export2"/>
      

      The package export2 has version 0.0.0 if you are explicit.

      3) Similar to (2) there is no fixup on the explicit capabilities.

      <classloading xmlns="urn:jboss:classloading:1.0">
       <capabilities>
       <package name="export2"/>
      

      This deployment has NO module capability.

      4) Should I create an included export filter?
      This would mean all classes are in the classloader, but only
      the listed packages are exported.
      While this might seem obvious, it is effectively the same as listing the
      packages in the capabilities and therefore redundant.
      It would also mean that "exportAll" doesn't really have that meaning anymore,
      instead a better name would be something like "autoDiscoverPackages". :-)

      SUMMARY

      The above questions boil down to two issues.

      a) Do you think we should "fixup" the capabilities, i.e.
      missing module capabilities, missing package versions
      or should we just do what the user says?

      b) Is it worth trying to go through the "hard problem" of merging
      the simple but not very explicit exportAll processing with explicit capabilities?

        • 1. Re: ClassLoadingMetaData ease-of-use

          OFF-TOPIC: Also for completeness.
          There's other metadata for the classloader.

          Old "big ball mud" (import everything in the same domain)

          <classloading importAll="true" xmlns="urn:jboss:classloading:1.0"/>
          


          Servlet spec style
          <classloading parent-first="false" xmlns="urn:jboss:classloading:1.0"/>
          


          Domain hierarchies
          <classloading domain="MyDomain" parentDomain="AnotherDomain" xmlns="urn:jboss:classloading:1.0"/>
          


          Caching and blacklisting (enabled by default)
          <classloading cache="false" blackList="false" xmlns="urn:jboss:classloading:1.0"/>
          


          And you can also "write your own classloader dependency"
          since the metadata supports wildcards on the Capability/Requirement interfaces.
          <classloading xmlns="urn:jboss:classloading:1.0">
           <capabilities>
           <module name="test" version="1.0.0"/>
           <test xmlns="urn:jboss:test:classloading:capability:1.0" name="foo" version="1.0.0"/>
           </capabilities>
           <requirements>
           <test xmlns="urn:jboss:test:classloading:requirement:1.0" name="bar" from="1.0.0"/>
           </requirements>
          </classloading>
          


          • 2. Re: ClassLoadingMetaData ease-of-use
            starksm64

             

            "adrian@jboss.org" wrote:

            NOW FOR THE QUESTIONS.

            First, does module name equate to the osgi bundle symbolic name? We would also need additional policy information that equates to the Require-Bundle header parameters.

            "adrian@jboss.org" wrote:

            1) Is the above "Note" about mutual exclusion of exportAll and listing
            capabilities correct? If not, it could get very messy trying to merge them. ;-)

            I guess, I can't see why one would want to use both syntaxes.

            "adrian@jboss.org" wrote:

            2) In the filter case, the packages automatically get the version of the module,
            but this doesn't happen for listing capabilities explicitly

            <classloading xmlns="urn:jboss:classloading:1.0">
             <capabilities>
             <module name="export1" version="1.0.0"/>
             <package name="export2"/>
            

            The package export2 has version 0.0.0 if you are explicit.

            Sure, this would be a reason to use the expanded syntax.

            "adrian@jboss.org" wrote:

            4) Should I create an included export filter?
            This would mean all classes are in the classloader, but only
            the listed packages are exported.
            While this might seem obvious, it is effectively the same as listing the
            packages in the capabilities and therefore redundant.
            It would also mean that "exportAll" doesn't really have that meaning anymore,
            instead a better name would be something like "autoDiscoverPackages". :-)

            I don't see the difference between this and the export-all="ALL" included="..." notion. What is the difference?

            "adrian@jboss.org" wrote:

            a) Do you think we should "fixup" the capabilities, i.e.
            missing module capabilities, missing package versions
            or should we just do what the user says?

            What would the version be fixed up to? In terms of modules, unless its mapped to something usable like the deployment name, how would the fixed up name be usable?

            "adrian@jboss.org" wrote:

            b) Is it worth trying to go through the "hard problem" of merging
            the simple but not very explicit exportAll processing with explicit capabilities?



            • 3. Re: ClassLoadingMetaData ease-of-use

               

              "scott.stark@jboss.org" wrote:

              First, does module name equate to the osgi bundle symbolic name? We would also need additional policy information that equates to the Require-Bundle header parameters.


              As I understand it, yes. Although I allow multiple names, i.e.
              multiple module capabilities/aliases.
              I'm using a "context name" internally, which you can either specifiy explicitly
              or it will generate one for you as "name:version"


              "adrian@jboss.org" wrote:

              1) Is the above "Note" about mutual exclusion of exportAll and listing
              capabilities correct? If not, it could get very messy trying to merge them. ;-)

              I guess, I can't see why one would want to use both syntaxes.


              I've erronously tried to do so in the tests which is why I thought about implementing it. :-)


              "adrian@jboss.org" wrote:

              4) Should I create an included export filter?
              This would mean all classes are in the classloader, but only
              the listed packages are exported.
              While this might seem obvious, it is effectively the same as listing the
              packages in the capabilities and therefore redundant.
              It would also mean that "exportAll" doesn't really have that meaning anymore,
              instead a better name would be something like "autoDiscoverPackages". :-)

              I don't see the difference between this and the export-all="ALL" included="..." notion. What is the difference?


              This would be exportAll="ALL" includedExport="p1,p2,p3".

              The "included" by itself also controls what you can see of yourself in the module,
              an included export or the package capabilities control what others can see,
              i.e. your exports.

              NOTE: included and excluded are still valid with explicit capabilities
              since they only act as an additional filter that applies to both
              you and others rather than as a potentially conflicting one. ;-)


              "adrian@jboss.org" wrote:

              a) Do you think we should "fixup" the capabilities, i.e.
              missing module capabilities, missing package versions
              or should we just do what the user says?

              What would the version be fixed up to? In terms of modules, unless its mapped to something usable like the deployment name, how would the fixed up name be usable?


              Possible fix ups would be:

              * Even if capabilities are explicit always add a module capability from the name
              given in the top level element. e.g. for deployments it could be the deployment
              name.

              * If there is a version in the top level element then add that to any capability
              that would otherwise have 0.0.0 as its version (the default, unspecified version).


              "adrian@jboss.org" wrote:

              b) Is it worth trying to go through the "hard problem" of merging
              the simple but not very explicit exportAll processing with explicit capabilities?

              I think we need to go through the resolution process when there are amiguities to see if there are defaults we should fixup to minimize problems.


              The issues would be around contradictory specifications, e.g.
              <classloader exportAll="ALL" excludeExport="p1">
              <capability>
               <package name="p1"/>
              ...
              


              So is p1 in the exports or not? ;-)

              There's probably other more complicated examples.
              Especially since I didn't mention that although these filters are shown
              as lists of packages, I'm potentially allowing any ClassFilter and wildcard
              capabilities so I don't necessarily know exactly what they mean,
              although I do know if the Capability implements ExportPackages
              which is part of the way.

              • 4. Re: ClassLoadingMetaData ease-of-use

                I'm going to use this thread to complete the description of all the classloader metadata
                in one place.

                RE-EXPORTS

                Beyond what is mentioned above, a reqiurement can take a reExport attribute.
                This allows you to build up "super modules" for ease of use, e.g.

                <classloader xmlns="urn:jboss:classloader:1.0">
                 <capabilities>
                 <module name="javaee" version="1.5"/>
                 </capabiltiies>
                 <requirements>
                 <module name="javax.jms" version="1.1" reExport="true"/>
                 <module name="javax.ejb" version='3.0" reExport="true/>
                 etc.
                


                Then somebody can do the following to get the entire javaee api.
                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="javaee" version="1.5"/>
                 </requirements>
                </classloading>
                


                NOTE: This is only works for module requirements. Other types of requirements
                like individual packages don't pick up re-exports.

                OPTIONAL

                You can specify that a requirement is optional, e.g.
                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="foo" optional="true"/>
                 </requirements>
                </classloading>
                


                If the module "foo" exists it will be imported, otherwise the requirement will be ignored.

                USES

                This is similar to optional, where you can specify that you want
                to use somebody else's export of a package if it is exists, but otherwise
                you will use your own version (included in your classpath).
                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <uses name="com.acme.blah"/>
                 </requirements>
                </classloading>
                


                EXPLICIT VERSION

                A requirement can take an explicit version constraint
                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="com.acme.blah" version="1.0.0"/>
                 </requirements>
                </classloading>
                


                In mathematical language :

                1.0.0 <= version <= 1.0.0

                FROM VERSION

                You can specify a from version which is inclusive by default

                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="com.acme.blah" from="1.0.0"/>
                 </requirements>
                </classloading>
                


                1.0.0 <= version

                or not inclusive

                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="com.acme.blah" from="1.0.0" from-inclusive="false"/>
                 </requirements>
                </classloading>
                


                1.0.0 < version

                TO VERSION

                You can specify a to version which is exclusive by default

                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="com.acme.blah" to="1.0.0"/>
                 </requirements>
                </classloading>
                


                version < 1.0.0

                or inclusive

                <classloading xmlns="urn:jboss:classloading:1.0">
                 <requirements>
                 <module name="com.acme.blah" to="1.0.0" to-inclusive="true"/>
                 </requirements>
                </classloading>
                


                version <= 1.0.0

                CONTEXT NAME

                In the classloader namespace that is used by the -beans.xml
                you'd use it something like this;

                <deployment xmlns="urn:jboss:bean-deployer:2.0">
                 <classloader><inject bean="MyClassLoader:1.0.0"/></classloader>
                 <classloader xmlns="urn:jboss:classloader:1.0" name="MyClassLoader" version="1.0.0">
                


                NOTE, by default the context name in the MC is name:version.
                But you can override this if you don't like it, e.g.

                <deployment xmlns="urn:jboss:bean-deployer:2.0">
                 <classloader><inject bean="Fred"/></classloader>
                 <classloader xmlns="urn:jboss:classloader:1.0" name="MyClassLoader" version="1.0.0" context="Fred">
                


                It's just a name after all ;-)

                The deployment version, just uses the deployment controller context name.

                CLASSLOADERSYSTEM

                By default, the classloader (-beans.xml usage) will get registered in a ClassLoaderSystem
                called .... "ClassLoaderSystem" :-)
                But you can change it.

                 <classloader xmlns="urn:jboss:classloader:1.0" name="MyClassLoader" version="1.0.0" system="SomethingElse">
                


                I doubt anybody would want to change it, since you'd have to change
                it consistently across all classloaders that want to export/import each other.

                For the deployment processing, the ClassLoaderSystem is injected
                into the deployer configuration.

                DONE

                I think that's about it? If I find I missed something, I'll add it here.

                • 5. Re: ClassLoadingMetaData ease-of-use
                  starksm64

                   

                  "adrian@jboss.org" wrote:

                  1.0.0 <= version <= 1.0.0
                  ...
                  FROM VERSION
                  ...
                  TO VERSION
                  ...

                  Can we use the osgi/mathematical syntax of a version range to specify from/to behaviors?
                  <... version="[1.2.3, 4.5.6)" />
                  

                  meaning the version must be in the range, 1.2.3 <= version < 4.5.6


                  • 6. Re: ClassLoadingMetaData ease-of-use

                     

                    "scott.stark@jboss.org" wrote:

                    Can we use the osgi/mathematical syntax of a version range to specify from/to behaviors?
                    <... version="[1.2.3, 4.5.6)" />
                    

                    meaning the version must be in the range, 1.2.3 <= version < 4.5.6


                    I can add it as an option.It is probably more prone to error
                    and easily forgotten by non-mathematicians though? :-)

                    And as you'll see in this example, it's not really any terser.
                    <requirements>
                     <!-- This -->
                     <module name="test" range="[1.2.3, 4.5.6)"/>
                     <!-- is the same as ->
                     <module name="test" from="1.2.3" to="4.5.6"/>
                    


                    NOTE: The toString() of VersionRange already uses that syntax.

                    • 7. Re: ClassLoadingMetaData ease-of-use
                      starksm64

                       

                      "adrian@jboss.org" wrote:

                      The issues would be around contradictory specifications, e.g.
                      <classloader exportAll="ALL" excludeExport="p1">
                      <capability>
                       <package name="p1"/>
                      ...
                      


                      So is p1 in the exports or not? ;-)

                      There's probably other more complicated examples.
                      Especially since I didn't mention that although these filters are shown
                      as lists of packages, I'm potentially allowing any ClassFilter and wildcard
                      capabilities so I don't necessarily know exactly what they mean,
                      although I do know if the Capability implements ExportPackages
                      which is part of the way.


                      So we have,
                      exportAll="ALL|NON_EMPTY" - an alias for includedExport="*|NonEmptyPackageFilter"?
                      included - define what is visible as input to the module itself via inclusion rules.
                      excluded - define what is visible as input to the module itself via exclusion rules.
                      includedExport,capability - define what can be exported from module's available content.

                      So in the example above, the overlapping p1 specifications could mean an empty p1 package was exported. If it was exportAll="NON_EMPTY", then p1 would not show up as a capability. This leads to the question: what does it mean to resolve a package to a module that only has an empty version of it? Generally this would not be meaningful. The only possible usage would be to obtain package annotations, but then is that really an empty package?

                      This really seems like a management interface issue where I want to be able to ask, what are the package export filters that apply to the class loader. From that perspective, the includedExport is just an alias for a type of capability (package, custom). However, since you can't use a namespace in an attribute value, how would you specify something other than a package filter? Given that, I would say the includedExport is only used for package filtering, and we can decide if we want better detection of conflicting statements that end up in an empty export set in the management interface.


                      • 8. Re: ClassLoadingMetaData ease-of-use
                        starksm64

                         

                        "adrian@jboss.org" wrote:

                        I can add it as an option.It is probably more prone to error
                        and easily forgotten by non-mathematicians though? :-)

                        There is no question that it is more error prone.


                        • 9. Re: ClassLoadingMetaData ease-of-use

                           

                          "scott.stark@jboss.org" wrote:

                          The only possible usage would be to obtain package annotations, but then is that really an empty package?


                          No there is a package-info.class


                          This really seems like a management interface issue where I want to be able to ask, what are the package export filters that apply to the class loader. From that perspective, the includedExport is just an alias for a type of capability (package, custom).


                          I'm working on it. I can't do everything at once. ;-)

                          Once I've done the management features you requested,
                          then I'll add ManagedOperations/Statistics to the ClassLoadingMetaData
                          ManagedObject. This can include the packages exported by a classloader,
                          classes/resources loaded, etc., although I'm only going to do a basic
                          set for the initial release as a "proof of concept", then you, me or somebody else
                          can add extra features.


                          However, since you can't use a namespace in an attribute value, how would you specify something other than a package filter? Given that, I would say the includedExport is only used for package filtering, and we can decide if we want better detection of conflicting statements that end up in an empty export set in the management interface.


                          I haven't enabled the full ClassFilter functionality in the xml or ManagedObject yet,
                          because the ClassFilter stuff needs fixing to be serializable.

                          I'll do them as elements so you can properly customize it.
                          With xml wildcards, etc. to define your own.

                          WARNING THIS DOES NOT EXIST YET

                          e.g. something like:
                          <classloading ...>
                           <!-- builit in-->
                           <include type="RegExp">javax\.ejb\..+</include>
                           <!-- or equivalently -->
                           <include type="Glob">javax.ejb.**</include>
                           <!-- user defined -->
                           <include type="Custom"><myfilter xmlns="whatever" ..../></include>
                          


                          • 10. Re: ClassLoadingMetaData ease-of-use
                            starksm64

                             

                            "adrian@jboss.org" wrote:

                            I'll do them as elements so you can properly customize it.
                            With xml wildcards, etc. to define your own.

                            Ok, but then if the attribute form still exists, won't it just be an alias for the most common element construct? To get back to the original question, it seems both included/excluded attributes and include/capability elements make sense. The attributes being shorthand for common element forms.

                            In the case where attributes/elements overlap, we have to consider understanding the filter type, even if only in the management interface to help diagnose conflicts like declaring a p1 capability when p1 was not made visible to the class loader itself. For that we would need access to the raw packages, local packages, and exported packages.


                            • 11. Re: ClassLoadingMetaData ease-of-use

                               

                              "scott.stark@jboss.org" wrote:
                              "adrian@jboss.org" wrote:

                              I'll do them as elements so you can properly customize it.
                              With xml wildcards, etc. to define your own.

                              Ok, but then if the attribute form still exists, won't it just be an alias for the most common element construct? To get back to the original question, it seems both included/excluded attributes and include/capability elements make sense. The attributes being shorthand for common element forms.

                              In the case where attributes/elements overlap, we have to consider understanding the filter type,


                              Well actually I coded it as running both filters if you specify both,
                              using a CombiningClassFilter. i.e. it will apply both the package list and
                              any custom filter you specify.

                              ClassLoadingMetaData
                              
                               /**
                               * Get a filter for the included packages
                               *
                               * @return the included packages
                               */
                               public ClassFilter getIncluded()
                               {
                               ClassFilter packageFilter = null;
                               if (includedPackages != null)
                               packageFilter = PackageClassFilter.createPackageClassFilterFromString(includedPackages);
                              
                               if (packageFilter == null)
                               return included;
                               if (included == null)
                               return packageFilter;
                               return CombiningClassFilter.create(true, packageFilter, included);
                               }
                              



                              even if only in the management interface to help diagnose conflicts like declaring a p1 capability when p1 was not made visible to the class loader itself. For that we would need access to the raw packages, local packages, and exported packages.


                              If you declare that you export a package and it doesn't exist, then that's your fault. ;-)

                              But this only comes up with the explicit package capabilties. The filters are not
                              absolute declarations, they are, well, filters on what it finds when scanning
                              the "file system".

                              I suppose I could validate the capabilities that say they "ExportPackages"
                              by running the ExportALL processing even if you explicitly list the package capabilities,
                              but I'm not sure it really buys much?
                              You'll still get a ClassNotFoundException on javax.jms.Queue
                              if you export the package, include it in your filesystem but don't have that class. ;-)

                              • 12. Re: ClassLoadingMetaData ease-of-use
                                starksm64

                                 

                                "adrian@jboss.org" wrote:

                                I suppose I could validate the capabilities that say they "ExportPackages"
                                by running the ExportALL processing even if you explicitly list the package capabilities,
                                but I'm not sure it really buys much?
                                You'll still get a ClassNotFoundException on javax.jms.Queue
                                if you export the package, include it in your filesystem but don't have that class. ;-)

                                Right, but what we want to be able to debug is the question about why javax.jms.Queue is not found when it sitting in my-jms.jar of classloader x. Its going to be common to have classes from the server domain hidden from the application deployment class loader, so why the hidden class is not seen in the question.


                                • 13. Re: ClassLoadingMetaData ease-of-use

                                   

                                  "scott.stark@jboss.org" wrote:
                                  "adrian@jboss.org" wrote:

                                  I suppose I could validate the capabilities that say they "ExportPackages"
                                  by running the ExportALL processing even if you explicitly list the package capabilities,
                                  but I'm not sure it really buys much?
                                  You'll still get a ClassNotFoundException on javax.jms.Queue
                                  if you export the package, include it in your filesystem but don't have that class. ;-)

                                  Right, but what we want to be able to debug is the question about why javax.jms.Queue is not found when it sitting in my-jms.jar of classloader x. Its going to be common to have classes from the server domain hidden from the application deployment class loader, so why the hidden class is not seen in the question.


                                  The management interface of the classloader will show the package is not exported,
                                  but you'd need to see the filter(s) and decode them to understand why.

                                  I guess I could always add some more trace logging to say which filter excludes a package? :-)

                                  • 14. Re: ClassLoadingMetaData ease-of-use

                                    I just implemented the final major feature required for OSGi.

                                    DYNAMIC

                                    You can now do

                                    <classloading ...>
                                     <requirements>
                                     <package name="foo" dynamic="true"/>
                                     </requirements>
                                    </classloading>
                                    


                                    This is a bit similar to optional, except unlike optional, it will continue
                                    to try to resolve import even after the classloader is constructed.

                                    Since this feature requires the caching and blacklisting to be disabled
                                    the dynamic imports are sorted towards the bottom of the import list
                                    (which looks to me like what is required for OSGi as well).

                                    When it is eventually resolved, then any caching and blacklisting will be re-enabled.