5 Replies Latest reply on Sep 26, 2010 11:00 AM by alesj

    Lazy modules resolving / linking

    alesj

      A simple brain dump about lazy module resolution -- as I'm yet to fully understand the code.

       

      Lets say you could immediately map resource to owning module.

      e.g. based on the package ->  org.jboss.foo is the module name, my class is org.jboss.foo.bar.Baz

      Or any similar mechanism would do.

       

      We could then lazy load the first level / near module dependencies,

      only fully initialize it on demand -- when first resource lookup is hit,

      and again have its first level / near dependencies lazy.

       

      This could be done optional, by default, as we currently do, resolve the whole graph at start.

      Perhaps pushing it a bit :-), we could even specify the depth / level of resolving.

       

      A few observations:

      * OSGi has this similar notion of lazy resolve - how do we do it now?

      * if I'm not mistaken, and I think Flavia was thinking about the same issue, we can reduce visiting

        --> simply (recursively) reuse paths from own children, no need to visit them

       

      Wdyt? Or, how much work this would be?

      I can try to hack something, but any input is appreciated.

        • 1. Re: Lazy modules resolving / linking
          dmlloyd

          OSGi dynamic import is implemented using a "fallback loader", which basically gets queried whenever a class or package is unknown to the loader.

           

          You could use the same idea for lazy loading.  In your fallback loader, attempt to load the corresponding module.  If it succeeds, add it as a dependency to the Module, and then relink the module; then retry the load.

          • 2. Re: Lazy modules resolving / linking
            alesj
            You could use the same idea for lazy loading.  In your fallback loader, attempt to load the corresponding module.  If it succeeds, add it as a dependency to the Module, and then relink the module; then retry the load.

            OK, this would work.

             

            protected LocalLoader doUpdate(String[] tokens)
               {
                  Node<Import> current = root;
                  StringBuilder path = new StringBuilder();
                  for (String token : tokens)
                  {
                     current = current.getChild(token);
                     if (current == null)
                        return null;

             

                     if (path.length() > 0)
                        path.append(".");
                     path.append(token);

             

                     Import i = current.getValue();
                     if (i != null)
                     {
                        ModuleIdentifier mi = ModuleIdentifier.create(path.toString(), i.getVersion().toString());
                        ModuleDependencySpec.Builder mdsb = ModuleDependencySpec.build(mi).setOptional(i.isOptional());
                        if (i.isExport()) // TODO -- relink depending modules ...
                           mdsb.setExportFilter(PathFilters.acceptAll());
                        ModuleDependencySpec mds = mdsb.create();
                        DependencySpec dependencySpec = dependencyBuilder.addModuleDependency(mds).create();
                        try
                        {
                           Module module = Module.getModule(target);
                           loader.updateModule(module, dependencySpec);
                           return module.getModuleLocalLoader(); // <--------- HERE
                        }
                        catch (ModuleLoadException ignored)
                        {
                           return null;
                        }
                     }
                  }
                  return null;
               }

             

            but in order not to hack around too much, I would need somthing like "HERE".

            Exposing too much if we expose ModuleClassLoader's LocalLoader?

             

            btw: any tip on how to relink dependeing modules from TODO?

            • 3. Re: Lazy modules resolving / linking
              alesj
              but in order not to hack around too much, I would need somthing like "HERE".

              Exposing too much if we expose ModuleClassLoader's LocalLoader?

              OK, this doesn't work anyway.

              I'm looking into properly solving this.

              • 4. Re: Lazy modules resolving / linking
                alesj
                I'm looking into properly solving this.

                After updating Module, I now simply try to reload based on Module's ClassLoader:

                 

                Module module = Module.getModule(target);
                loader.updateModule(module, dependencySpec);
                return new ClassLoaderLocalLoader(module.getClassLoader());

                 

                but this creates a deadlock:

                 

                main@1, prio=5, in group 'main', status: 'waiting'

                  java.lang.Thread.State: WAITING

                      at java.lang.Object.wait(Object.java:-1)

                      at java.lang.Object.wait(Object.java:485)

                      at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:257)

                      at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:52)

                      at net.something.xyz.Module$1.run(Module.java:43)

                      at org.jboss.as.modules.plugins.run.DefaultRuntime.execute(DefaultRuntime.java:64)

                      at org.jboss.as.modules.Main.execute(Main.java:67)

                      at org.jboss.asz.test.modules.ModulesTest.execute(ModulesTest.java:123)

                      at org.jboss.asz.test.modules.ModulesTest.testArchive(ModulesTest.java:94)

                      at org.jboss.asz.test.modules.ondemand.test.OnDemandTestCase.testBasic(OnDemandTestCase.java:47)

                      at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)

                      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

                      at java.lang.reflect.Method.invoke(Method.java:597)

                 

                and

                 

                ClassLoader Thread@1179 daemon, prio=5, in group 'main', status: 'runnable'

                  java.lang.Thread.State: RUNNABLE

                     blocks main@1

                      at java.lang.Class.forName0(Class.java:-1)

                      at java.lang.Class.forName(Class.java:247)

                      at org.jboss.modules.ClassLoaderLocalLoader.loadClassLocal(ClassLoaderLocalLoader.java:49)

                      at org.jboss.as.modules.plugins.run.OnDemandLocalLoader.loadClassLocal(OnDemandLocalLoader.java:99)

                      at org.jboss.modules.Module.loadModuleClass(Module.java:727)

                      at org.jboss.modules.Module.loadModuleClass(Module.java:692)

                      at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:186)

                      at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:268)

                      at org.jboss.modules.ConcurrentClassLoader.access$400(ConcurrentClassLoader.java:42)

                      at org.jboss.modules.ConcurrentClassLoader$LoaderThread.run(ConcurrentClassLoader.java:337)

                • 5. Re: Lazy modules resolving / linking
                  alesj

                  Adding and using something like this helps:

                   

                  public final class ModuleLocalLoader extends ClassLoaderLocalLoader {

                   

                      private final Module module;

                   

                      public ModuleLocalLoader(Module module) {
                          super(check(module));
                          this.module = module;
                      }

                   

                      private static ClassLoader check(Module module) {
                          if (module == null)
                              throw new IllegalArgumentException("Null module");
                          return module.getClassLoader();
                      }

                   

                      public Class<?> loadClassLocal(final String name, final boolean resolve) {
                          return module.loadModuleClass(name, false, resolve);
                      }
                  }

                   

                  It's public, but it exposes only read only stuff.

                  Wdyt?