1 2 Previous Next 19 Replies Latest reply on Oct 26, 2006 6:08 AM by Sacha Labourey

    VirtualFile.toURL() returning vfsfile:

    Bill Burke Master

      Make a bunch of changes on my box so that VirtualFile.toURL() returns a vfsfile: based URL. The Handler's toURL()/getURL() will return the real URL (if possible) of the file, while VF returns the vfsfile: based one.

      One huge problem I've run into is that vfsfile: based urls seem to REALLY slow down URL classloaders. Looking into the problem now. It may be that I have to cache VFs based on URL. Don't know yet.

        • 1. Re: VirtualFile.toURL() returning vfsfile:
          Bill Burke Master

          looks like VirtualFiles and their handlers do not cache any references to their children.

          Maybe we should change this? And have a clear() and/or refresh?

          • 2. Re: VirtualFile.toURL() returning vfsfile:
            Scott Stark Master

            Some do cache children (jars). Why can't the existing close be used to releases resources? Unless as part of profiling there is a reason to refresh a deployment dir/top-level deployment its something that I don't know we want to support.

            • 3. Re: VirtualFile.toURL() returning vfsfile:
              Bill Burke Master

              hmmm...

              Added a cache for things created under:

              FileHandler.createChildHandler()

              So, JarHandlers would only get created once. Problem is, URLClassLoaders don't like this reuse of the JarHandler. If I remove the cache, everything boots up find, but if I add the cache, then it fails with NoClassDefFound errors with no nice stack trace I can pinpoint it to although I did see some failures in ClassLoader.declareClass.

              Any guesses would be appreciated.

              My guess is that it doesn't like how JarHandler caches the JarEntries.

              • 4. Re: VirtualFile.toURL() returning vfsfile:
                Bill Burke Master

                BTW, I have no problem testing this outside of JBoss with using a regular URL classloader with a vfsfile: based url.

                • 5. Re: VirtualFile.toURL() returning vfsfile:
                  Bill Burke Master

                  For example:

                  13:06:18,406 ERROR [AbstractKernelController] Error installing to Create: name=jboss.management.local:j2eeType=J2EEDomain,name=Manager state=Configured mode=Manual requiredState=Create
                  java.lang.NoClassDefFoundError: org/jboss/management/j2ee/statistics/StatisticsProvider
                   at java.lang.ClassLoader.defineClass1(Native Method)
                   at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
                   at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
                   at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
                   at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
                   at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
                   at java.security.AccessController.doPrivileged(Native Method)
                   at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
                   at org.jboss.mx.loading.RepositoryClassLoader.findClassLocally(RepositoryClassLoader.java:566)
                  



                  • 6. Re: VirtualFile.toURL() returning vfsfile:
                    Scott Stark Master

                    If there is a declareClass failure you won't be able to load it latter so why declareClass is failing needs to be resolved. The main problem I have seen with caching/reusing JarFile elements is that once any etnry stream is closed all others are unsuable. Let's separate out optimization from the initial implementation though.

                    • 7. Re: VirtualFile.toURL() returning vfsfile:
                      Bill Burke Master

                       


                      Let's separate out optimization from the initial implementation though.


                      switching VirtualFile.toURL() to vfsfile: based makes JBoss unusable if we don't optimize.

                      • 8. Re: VirtualFile.toURL() returning vfsfile:
                        Scott Stark Master

                        Then we probably need more control of how jars are handled as the JarFile implementation has a bunch of native code for the stream. Marshall mentioned this project before:

                        https://truezip.dev.java.net/

                        • 9. Re: VirtualFile.toURL() returning vfsfile:
                          Bill Burke Master

                          Found out some things about classloading and URLClassLoader:

                          * a sun URLClassPath class is used to get .class files as a resource
                          * If the URL ends in '/' a "directory" based approach seems to be used and the URLClassPath class concatenates the resource name to the base URL to open the file.

                          So, when a resource is looked up it will do

                          vfsfile:/foo.jar/org/jboss/SomeClass.class

                          to find the resource.

                          My current guess is that JarFile's are not reentrant nor threadsafe. I took a look at truezip, but it is currently totally based upon java.io.File and a real file system. So you can't just pass in any old URL to get access to the JAR file.

                          • 10. Re: VirtualFile.toURL() returning vfsfile:
                            Bill Burke Master

                            i'm looking to see if truezip is reentrant and threadsafe. It says its threadsafe, don't know about reentrant or if it matters that it is reentrant or not.

                            • 11. Re: VirtualFile.toURL() returning vfsfile:
                              Bill Burke Master

                               

                              "bill.burke@jboss.com" wrote:
                              Found out some things about classloading and URLClassLoader:

                              * a sun URLClassPath class is used to get .class files as a resource
                              * If the URL ends in '/' a "directory" based approach seems to be used and the URLClassPath class concatenates the resource name to the base URL to open the file.

                              So, when a resource is looked up it will do

                              vfsfile:/foo.jar/org/jboss/SomeClass.class

                              to find the resource.

                              My current guess is that JarFile's are not reentrant nor threadsafe. I took a look at truezip, but it is currently totally based upon java.io.File and a real file system. So you can't just pass in any old URL to get access to the JAR file.



                              I'm going to remove the "/" from the end of any JAR vfsfile: URL so that URLClassLoaders will treat the vfsfile: URL as a JAR rather than raw URLs. I couldn't figure out why using raw URLs with the VFS does not work. This leads me to believe we will initially not be able to support an exploded archive nested within an unexploded archive and this will effect things like a WAR's WEB-INF/classes.



                              • 12. Re: VirtualFile.toURL() returning vfsfile:
                                Bill Burke Master

                                Found out more things:

                                * If I read the JarEntries stream into a ByteArrayInputStream, then everything is cool

                                to be able to read into a BAIS, for some reason, I have to loop on the read or it doesn't work:

                                 ByteArrayOutputStream baos = new ByteArrayOutputStream((int)size);
                                 int wasRead = 0;
                                 byte[] tmp = new byte[1024];
                                 while( is.available() > 0 )
                                 {
                                 int length = is.read(tmp);
                                 if( length > 0 )
                                 baos.write(tmp, 0, length);
                                 wasRead += length;
                                 }
                                 is.close();
                                 byte[] bytes = baos.toByteArray();
                                


                                * A better way to get a unique/copy of the InputStream for the JarEntry. I used URL's for this:

                                URLConnection con = jarEntryUrl.openConnection();
                                con.setUseCaches(false);
                                return con.getInputStream();
                                


                                This works beautifully. Now the question is, how slow is this MOFO?


                                • 13. Re: VirtualFile.toURL() returning vfsfile:
                                  Scott Stark Master

                                  I think we are butting up against JarFile implementation details here. Wouldn't it be better to get InputStream support into truezip?

                                  The reading the entry contents into memory is something I had to do for nested jars elements. We are essentially pulling all jar contents into memory. I would think its more of a startup time/memory issue than performance. We won't be able to lazily read entry contents with the JarFile. Maybe we can with truezip, although hopefully its not needed.

                                  • 14. Re: VirtualFile.toURL() returning vfsfile:
                                    Bill Burke Master

                                    I can only get it to work by doing both the useCaches(false) and pre-reading the stream into a BAIS. But it boots twice as slow as normal. I have to do this trick for FileHandlers, JarHandlers, and JarEntry handlers.

                                    I'm really nervous that basing everything on a VFS URL will not work with other VMs. There's all this funky shit caching going on behind the scenes that I can only get a glimpse at by decompiing Sun specific JDK code. Some of the problems are hidden within native methods as well. I got all this to work so far by guessing...

                                    1 2 Previous Next