7 Replies Latest reply on Jun 10, 2009 8:41 AM by Ales Justin

    Classloading and versioning

    Ales Justin Master

      While hacking the demo, I had 'problems' creating this 'expected' behavior:

      I have foo.jar which contains FooBar service.
      This FooBar service uses two other services - AlesService and ScottService.

       <bean name="FooBarService" class="org.jboss.foo.FooBarService">
       <property name="alesService"><inject/></property>
       <property name="scottService"><inject/></property>
       </bean>
      


      Then I have two similar jars - acme1 and acme2.
      Containing AlesService(Impl) and ScottService(Impl).
      acme1 has all classes in version 1.
      Where acme2 has ales package in version 2 and scott package in version 3.

      Then I have the following classloading metadata in foo.jar:
      <classloading xmlns="urn:jboss:classloading:1.0" export-all="NON_EMPTY">
       <capabilities>
       <package name="org.jboss.acme.foo" version="1"/>
       </capabilities>
       <requirements>
       <package name="org.jboss.acme.ales" from="1" to="2" from-inclusive="true"/>
       <package name="org.jboss.acme.scott" from="3" to="4" from-inclusive="true"/>
       </requirements>
      </classloading>
      


      Expecting that FooBar would use AlesService(Impl) from acme1 and ScottService(Impl) from acme2.

      But that's not what happened when using the bootstrap configuration similar to what we use in AS5_trunk. :-(

      Either I don't know how to properly configure this, or it's broken. ;-)

      The problems that I've found:

      When ClassLoaderPolicy returns non-null getExported delegate loader, this gets cached in BaseClassLoaderDomain::classLoadersByPackageName
       if (packageNames != null && info.getExported() != null)
       {
       for (String packageName : packageNames)
       {
       List<ClassLoaderInformation> list = classLoadersByPackageName.get(packageName);
       if (list == null)
       {
       list = new CopyOnWriteArrayList<ClassLoaderInformation>();
       classLoadersByPackageName.put(packageName, list);
       }
       list.add(info);
      

      and this is then used when looking up the class in other bundles as well, but w/o knowing the right version:
      BaseClassLoaderDomain::findLoaderInExports
       List<ClassLoaderInformation> list = classLoadersByPackageName.get(packageName);
       if (trace)
       log.trace(this + " trying to load " + name + " from all exports of package " + packageName + " " + list);
       if (list != null && list.isEmpty() == false)
       {
       for (ClassLoaderInformation info : list)
       {
       BaseDelegateLoader exported = info.getExported();
      
       // See whether the policies allow caching/blacklisting
       BaseClassLoaderPolicy loaderPolicy = exported.getPolicy();
       if (loaderPolicy == null || loaderPolicy.isCacheable() == false)
       canCache = false;
       if (loaderPolicy == null || loaderPolicy.isBlackListable() == false)
       canBlackList = false;
      
       if (exported.getResource(name) != null)
       {
       if (canCache)
       globalClassCache.put(name, exported);
       return exported;
       }
       }
       }
      

      So, when attempting to load class with the same name from foo.jar's classlaoder, but expecting it in diff version, I still get the previous one.

      OK, then I disabled this exported usage - either by returning null on getExported or null getPackageNames.
      Still didn't work as expected. :-(
      Now the version-unaware delegates were kicking in:
      BaseClassLoaderDomain::findLoaderInImports
       for (DelegateLoader delegate : delegates)
       {
       if (trace)
       log.trace(this + " trying to load " + name + " from import " + delegate + " for " + info.getClassLoader());
       if (delegate.getResource(name) != null)
       {
       info.cacheLoader(name, delegate);
       return delegate;
       }
       }
      

      where by default in CLPolicy we return exported, which is just filtered by package names - no version awareness.
      So again the previous classloader was picked.

      What I had to do then - and I don't see why this isn't default behavior - is the following piece of code:
       public DelegateLoader getDelegateLoader(Module module, Requirement requirement)
       {
       List<Capability> capabilities = getCapabilities();
       if (capabilities != null && capabilities.isEmpty() == false)
       {
       for(Capability capability : capabilities)
       {
       if (capability.resolves(this, requirement))
       return new FilteredDelegateLoader(getPolicy(), new CapabilityFilter(capability));
       }
       // none of the capabilities match - don't put it as delegate
       return null;
       }
       return super.getDelegateLoader(module, requirement);
       }
      
      public class CapabilityFilter implements ClassFilter
      {
       private Capability capability;
      
       public CapabilityFilter(Capability capability)
       {
       if (capability == null)
       throw new IllegalArgumentException("Null capability.");
      
       this.capability = capability;
       }
      
       public boolean matchesClassName(String className)
       {
       return matchesPackageName(ClassLoaderUtils.getClassPackageName(className));
       }
      
       public boolean matchesResourcePath(String resourcePath)
       {
       return matchesPackageName(ClassLoaderUtils.getResourcePackageName(resourcePath));
       }
      
       public boolean matchesPackageName(String packageName)
       {
       if (capability instanceof ExportPackages)
       {
       ExportPackages ep = (ExportPackages)capability;
       Set<String> packages = ep.getPackageNames(null);
       if (packages != null && packages.contains(packageName))
       return true;
       }
      
       return false;
       }
      }
      

      This finally got me to my expected behavior. :-)

      All this code can be found here:
      - http://anonsvn.jboss.org/repos/jbossas/projects/demos/osgi/


        • 1. Re: Classloading and versioning
          Adrian Brock Master

          This configuration is invalid (but the part that checks it is the outstanding issue in JBCL).

          acme1 exports ales version 1 and scott version 1
          acme2 exports ales version 2 and scott version 3
          foo wants to import ales version 1 from acme1 and scott version 3 from acme2

          Tthis will lead to incompatibilities between the versions in the linked classloaders.

          foo, acme1 and acme2 are in the same "classloading space" so they must agree
          on a common instance of the classes in the exported ales package.

          It should be failing at deployment time with an "conflicting package resolution" error for the
          ales package.

          If foo wants to use scott version 3 (provided by acme2) then it must also use
          ales version 2 to be consistent.

          • 2. Re: Classloading and versioning
            Adrian Brock Master

             

            "adrian@jboss.org" wrote:

            If foo wants to use scott version 3 (provided by acme2) then it must also use
            ales version 2 to be consistent.


            Another alternative would be for acme2 to declare a uses constraint
            such that it could fall back to ales version 1 when required. Then everybody
            would use ales version 1 with acme2's export of ales ignored.

            But that would fail in this example because it would still conflict on the scott package.

            • 3. Re: Classloading and versioning
              Ales Justin Master

               

              "adrian@jboss.org" wrote:

              If foo wants to use scott version 3 (provided by acme2) then it must also use
              ales version 2 to be consistent.

              This then defeats the purpose of versioning, or at least that's how I see the real useful case.
              For me it's totally legit to have this example kind of configuration, in case you know that classes in ales and scott packages are never gonna exchange classes among each other.
              But you're saying this is not what OSGi considers legit?

              • 4. Re: Classloading and versioning
                Ales Justin Master

                 

                "adrian@jboss.org" wrote:

                Another alternative would be for acme2 to declare a uses constraint
                such that it could fall back to ales version 1 when required. Then everybody
                would use ales version 1 with acme2's export of ales ignored.

                But that would fail in this example because it would still conflict on the scott package.

                OK, this makes sense, and it's more logical.
                But I still don't see why it would fail on scott?
                Or you're assuming scott package uses something from ales?

                Even if it would, why would that be a problem?
                Shouldn't there be a way to tell that in-bundle usage is a black box, and scott's usage of ales is impl detail.

                Would then there be a way to get runtime CCE if something ales's was exposed through scott and used in foobar?
                Or how do you detect this at deployment, exposing the possible conflict?

                • 5. Re: Classloading and versioning
                  Adrian Brock Master

                   

                  "alesj" wrote:

                  Even if it would, why would that be a problem?
                  Shouldn't there be a way to tell that in-bundle usage is a black box, and scott's usage of ales is impl detail.

                  Would then there be a way to get runtime CCE if something ales's was exposed through scott and used in foobar?
                  Or how do you detect this at deployment, exposing the possible conflict?


                  The way to tell the classloader that they are shared and could conflict is by
                  expressing the export. Once the package is exported anything that imports it
                  could load it.

                  Think about it...

                  foo imports ales from acme1
                  foo imports scott from acme2
                  but acme1 also exports scott and acme2 exports ales

                  The classloader can't make the assumptions you are trying to make.
                  it doesn't know that ales and scott won't leak across the two different
                  deployments.
                  They are exported so it must assume that they will since importers
                  must need the classes for a reason.

                  If they are not exported, there's no consistency check required.
                  The modules are assume to only use the classes as private implementation detail.

                  Even if they were the same physical classes on disk (and defined as the same
                  version) you'd get a CCE since the classes are loaded by different classloaders.

                  The fundamental problem is caused by an incorrect modularisation.
                  You want to version ales and scott independently so foo
                  can choose them independently. But you've tied them into
                  the same bundle making this impossible.

                  There be no problem if there was an
                  acme1-ales:1.0.0
                  acme1-scott:1.0.0
                  acme2-ales:3.0.0
                  acme2-scott:4.0.0

                  foo could then choose acme1-ales:1.0.0 and acme1-scott-4.0.0
                  as long as acme1-scott-4.0.0 doesn't need to make an incompatible
                  decision on what version of ales to use (and vice versa).

                  • 6. Re: Classloading and versioning
                    Ales Justin Master

                    Yeah, I though about it some more after my posts last night, and came to similar conclusions. I though it was more flexible, but like your clarification shows, that's the only way to do it.
                    But thanks anyway for shedding some light on the subject.

                    Basically, what I did, was what I was asking you in email - an example of some hacky classloading policy. ;-)

                    • 7. Re: Classloading and versioning
                      Ales Justin Master

                       

                      "alesj" wrote:

                      Basically, what I did, was what I was asking you in email - an example of some hacky classloading policy. ;-)

                      This is now properly restricted, by ClassLoadingSpace::join.
                      (just tried to hack it again @J1, with same example, but failed :-))