13 Replies Latest reply on Oct 24, 2006 6:59 AM by Morten Haavaldsen

    JNI, System.loadLibrary() and class loader interactions

    Bob Handsaker Newbie

      If this is the wrong forum, please redirect this.

      I have been debugging a problem with JBoss and JNI native code.
      I have sample code that runs outside of JBoss, but when run in JBoss
      it throws an UnsatisfiedLinkError referencing the method name on the
      first call to a native method. I have verified (at the OS level) that
      the DLLs (happens to be Windows) are in fact being loaded in to the
      process.

      I believe I've finally found the cause of my problem, which I believe
      is that the native libraries are loaded in the context of a particular
      ClassLoader. In my case, I'm loading the DLLs in one ClassLoader and
      the classes that define the native methods are being loaded into a
      different ClassLoader.

      These are third-party java classes and JNI DLLs, so I don't have control
      over the design. The classes to not load the DLLs automatically, the
      caller has to do it.

      Questions:

      1. Does my suspect cause above sound correct? If the DLL is loaded in
      a different ClassLoader than the class declaring the native method,
      should I expect it to not "see" the loaded native method?

      2. I was initially somewhat surprised that this seems to happen even if
      I load the DLLs in a ClassLoader that is the parent of the ClassLoader
      that loads the class declaring the native method. For example, I have
      tried loading the DLLs in the boot class loader (the one that loads
      org.jboss.Main), hoping that all descendent class loaders would be
      able to see the DLLs, but this did not work. (I loaded just the DLLs
      in the boot class loader, not the classes declaring the native methods.)

      I haven't been able to find definitive documentation on this in terms
      of Java language specs. Is it correct that child class loaders do not
      inherit visibility to native libraries loaded by thier parents?

      Or is this some kind of bug in the JBoss ClassLoader implementations?

      3. If I'm on the right track, what I think I would like to do is to have
      my MBean load the DLLs once and then also have the MBean force-load
      the classes that declare the native methods. These would not be hot
      deployable. Then I would like to be able to reference these classes
      from other jars and EJBs that are deployed separately and have my
      code be hot deployable.

      Should this work?

      What class loader is used for an MBean in JBoss?

      I read the JMX 4.x class loader documentation but couldn't find
      a specific reference to which class loader is used for MBeans.

        • 1. Re: JNI, System.loadLibrary() and class loader interactions
          David Robison Novice

          Did you ever solve this problem? I'm having a simular problem.
          David Robison

          • 2. Re: JNI, System.loadLibrary() and class loader interactions
            Charles Kreps Newbie

            I am having the same problem with LD_LIBRARY_PATH on linux and an MBean that should be able to invoke a shared library defined on that path. Could someone please post a response? Maybe a hint on where to look in the JBoss documentation?

            • 3. Re: JNI, System.loadLibrary() and class loader interactions
              Bob Handsaker Newbie

              I was on the right track in what I wrote above. I finally found the
              following reference in the JNI spec
              (http://java.sun.com/j2se/1.5.0/docs/guide/jni/spec/design.html):

              "The programmer may use a single library to store all the native methods
              needed by any number of classes, as long as these classes are to be
              loaded with the same class loader. The VM internally maintains a list
              of loaded native libraries for each class loader. Vendors should choose
              native library names that minimize the chance of name clashes."


              The key point here is "same class loader". Not a parent, not a child:
              Same class loader.

              We have resolved this problem by loading the native libraries from an MBean
              and then invoking code during the initialization of the MBean that loads the
              library *and* touches each class that uses native methods, forcing the classes
              to be loaded by the MBean class loader. Once we do this, everything works
              correctly.

              Note that if you do not force a class with native methods to be loaded by the
              MBean, it will be successfully loaded later by a different class loader,
              (for example, an EJB class loader), but the native methods will not be visible.

              One side effect of this solution is that we cannot hot deploy the classes
              with the native methods and our code that calls them. We could perhaps
              achieve some level of hot deployment if we carefully separated our code,
              but we didn't try.



              • 4. Re: JNI, System.loadLibrary() and class loader interactions
                Charles Kreps Newbie

                Thanks bhandsaker...this is probably the solution to my problem. If you have any example code of just how you are doing the ClassLoading from the MBean could you post it? Would help me get started.

                • 5. Re: JNI, System.loadLibrary() and class loader interactions
                  Bob Handsaker Newbie

                  Here's what we're doing. Basically you need to touch the class
                  to force it to be loaded in the thread that initializes the MBean.

                  The AHelpers.loadGCOSLibraries method is called from the MBean.
                  Then we touch one class from each jar file that references native
                  methods. Once the jar file is bound to a class loader, I believe
                  that class loader will load all classes from that jar, regardless
                  of which thread/class requests the class to be loaded later.

                  When debugging, we also log the value of getClassLoader() for
                  the MBean class and each of the classes with native methods.
                  If they don't match, you're in for trouble.

                  Hope this helps,
                  Bob Handsaker


                  public class AHelpers {
                  
                   public synchronized static boolean loadGCOSLibraries() {
                  
                   // Prevent multiple initialization.
                   if (m_gcosInitialized) {
                   return true;
                   }
                  
                   log.info("Loading GCOS libraries...");
                   log.info("System PATH: " +
                   System.getProperty("java.library.path"));
                  
                   try {
                  
                   System.loadLibrary("GcdoAffyJava");
                   log.info("System.loadLibrary(\"GcdoAffyJava\") is successful.");
                   System.loadLibrary("GcdoAnalysisController");
                   log.info("System.loadLibrary(\"GcdoAnalysisController\") is successful.");
                   System.loadLibrary("GdacFilesJava");
                   log.info("System.loadLibrary(\"GdacFilesJava\") is successful.");
                  
                   log.info("About to check Classes in GCOSSDK.jar ...");
                   checkClass(GcdoConnection.class);
                   log.info("GcdoConnection passed.");
                   checkClass(GcdoSample.class);
                   log.info("GcdoSample passed.");
                  
                   log.info("About the check Classes in GdacFiles.jar ...");
                   checkClass(GDACCELFile.class);
                   log.info("GDACCELFile passed.");
                   checkClass(GDACCHPFile.class);
                   log.info("GDACCHPFile passed.");
                  
                   } catch(Throwable t) {
                   log.error("GCOS API initialization failed: " + t.getMessage(), t);
                   return false;
                   }
                  
                   m_gcosInitialized = true;
                   return true;
                   }
                  
                   private static void checkClass(Class c) throws Exception {
                   if (c.getClassLoader() != AHelpers.class.getClassLoader()) {
                   throw new Exception ("Class " + c.getName() + " not loaded by correct class loader");
                   }
                   }
                  
                   private static Log log = LogFactory.getLog(AHelpers.class);
                  
                   private static boolean m_gcosInitialized = false;
                  }
                  


                  • 6. Re: JNI, System.loadLibrary() and class loader interactions
                    Chris Moran Newbie

                    I have just tried this but it didn't solve the problem I'm seeing. Was this tried with JBoss 4.0.3 or with some other version?

                    • 7. Re: JNI, System.loadLibrary() and class loader interactions
                      damonf Newbie


                      We are also having this problem in JB 4.03sp1. A .dll is used both by a .war and also in some .jars. My impression is that the .war classloader will be separate from the classloaders for the jars in the deploy and lib directories.

                      I am not clear after reading the above posts about how to "find" the right classloader to load everything and still have it accessible to all parts of our system.

                      • 8. Re: JNI, System.loadLibrary() and class loader interactions
                        Bob Handsaker Newbie

                        If the DLL is being used by different native methods defined in the .war and
                        in the .jar, then you may be in trouble. I suspect you will need to get all
                        of the native methods that use the DLL into one place (e.g. one jar).

                        We are still using 4.0.2. I don't know if 4.0.3 will affect this.
                        We are initializing (calling AHelpers.loadGCOSLibraries()) from an MBean,
                        in the start() method of the MBean. Classes loaded by an MBean are generally
                        pretty visible to other JBoss classloaders. We are calling them from .jars
                        and ejbs. I don't think we've specifically tried .wars.

                        (I don't remember exactly which class loader is used for the MBean, you
                        can dig into the JBoss documentation for details. It looks like maybe now
                        you can even specify the class loader you want to use.
                        See http://docs.jboss.org/jbossas/jboss4guide/r4/html/
                        particularly sections 2.2 and 2.4.2.)

                        The disadvantage of using an MBean is that it is not hot deployable
                        (although the native DLL itself will never be hot deployable).

                        • 9. Re: JNI, System.loadLibrary() and class loader interactions
                          Scott Stark Master

                          MBeans have been hot deployable since 3.0.x.

                          • 10. Re: JNI, System.loadLibrary() and class loader interactions
                            Debbie Collins Newbie

                            I have implemented your MBean solution and the dll's are being loaded, but the methods are not visible even from the MBean. All classes have the same class loader. I'm using JBOSS 4.0.1sp1.

                            Any ideas?

                            • 11. Re: JNI, System.loadLibrary() and class loader interactions
                              Debbie Collins Newbie

                              Got it! I forgot to go back and change the .h files when I moved the methods.

                              • 12. Re: JNI, System.loadLibrary() and class loader interactions
                                Morten Haavaldsen Newbie

                                Hi,

                                we have same situation and follows above advice but it does not work.

                                We load library from MBean.start()
                                We do teh check class loader which confirms same classloader...

                                But, when calling an operation from JMX console which uses the native interface method, the unsatisifed link error comes....

                                So...we are a bit puzzled why this does not work....

                                Does there exist any test DLL and som JBoss code which actually works and shows how to do this?

                                • 13. Re: JNI, System.loadLibrary() and class loader interactions
                                  Morten Haavaldsen Newbie

                                   

                                  "bhandsaker" wrote:
                                  I was on the right track in what I wrote above. I finally found the
                                  following reference in the JNI spec
                                  (http://java.sun.com/j2se/1.5.0/docs/guide/jni/spec/design.html):

                                  "The programmer may use a single library to store all the native methods
                                  needed by any number of classes, as long as these classes are to be
                                  loaded with the same class loader. The VM internally maintains a list
                                  of loaded native libraries for each class loader. Vendors should choose
                                  native library names that minimize the chance of name clashes."


                                  The key point here is "same class loader". Not a parent, not a child:
                                  Same class loader.

                                  We have resolved this problem by loading the native libraries from an MBean
                                  and then invoking code during the initialization of the MBean that loads the
                                  library *and* touches each class that uses native methods, forcing the classes
                                  to be loaded by the MBean class loader. Once we do this, everything works
                                  correctly.

                                  Note that if you do not force a class with native methods to be loaded by the
                                  MBean, it will be successfully loaded later by a different class loader,
                                  (for example, an EJB class loader), but the native methods will not be visible.

                                  One side effect of this solution is that we cannot hot deploy the classes
                                  with the native methods and our code that calls them. We could perhaps
                                  achieve some level of hot deployment if we carefully separated our code,
                                  but we didn't try.



                                  How do you force the class loading?