7 Replies Latest reply on Nov 3, 2008 8:42 AM by Ales Justin

    Heavy usage of VirtualFileURLConnection

    Ales Justin Master

      When dealing with VFS URLs pointing to some nested resources,
      things get horribly slow due to huge VirtualFileURLConnection usage.
      Here is where things are explained a bit more on real Seam example:
      - http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4184779#4184779

      The possible solutions I see.

      1) try to impl those Seam use cases directly on VFS - same way we do it for Deployment and CL
      Q1: can we really do it for all cases?
      Q2: what about other non-integrated frameworks in JBoss5?

      2) change how we handle VFS URLs - same way we do it in pre-JBoss5
      Q1: what's the reason we don't return VFS URLs with temp paths?

      3) implement some simple VFS caching
      Q1: use real cache, e.g. LRU, or is SoftRef enough?
      Q2: we need to cache intermediate paths - getting different resources from same parent faster (this requires VFURLConnection impl change)
      Q3: transparent AOP usage - caching is an aspect?

      Any other ideas?

        • 1. Re: Heavy usage of VirtualFileURLConnection
          Marko Strukelj Master

           

          "alesj" wrote:

          2) change how we handle VFS URLs - same way we do it in pre-JBoss5
          Q1: what's the reason we don't return VFS URLs with temp paths?


          VFS URL handling delegates to VFS handlers and contexts. The existence of a temp path is currently not guaranteed - depends on vfs settings and is an implementation detail. If contexts (handlers) were reused between URLs then there would be no performance problem. But as things are now, every getInputStream() request creates a new context.

          "alesj" wrote:

          3) implement some simple VFS caching
          Q1: use real cache, e.g. LRU, or is SoftRef enough?

          What would you cache? The most sensible thing to cache would be VFSContext ...


          "alesj" wrote:

          Q2: we need to cache intermediate paths - getting different resources from same parent faster (this requires VFURLConnection impl change)

          As long as you hold a context 'open', intermediate paths are automatically kept 'open' for fast access.

          "alesj" wrote:

          Q3: transparent AOP usage - caching is an aspect?

          We would have to identify the most appropriate join points for this. This opens another question - do we want to put cache interception only on URL VFS integration code or at VFS API level itself.

          - marko


          • 2. Re: Heavy usage of VirtualFileURLConnection
            Ales Justin Master

            I've implemented simple vfs cache:

            public interface VFSCache
            {
             /**
             * Get the file.
             *
             * @param uri the file's uri
             * @return virtual file instance
             * @throws IOException for any error
             */
             VirtualFile getFile(URI uri) throws IOException;
            
             /**
             * Get the file.
             *
             * @param url the file's url
             * @return virtual file instance
             * @throws IOException for any error
             */
             VirtualFile getFile(URL url) throws IOException;
            
             /**
             * Put vfs context to cache.
             *
             * @param context the vfs context
             */
             void putContext(VFSContext context);
            
             /**
             * Remove vfs context from cache.
             *
             * @param context the vfs context
             */
             void removeContext(VFSContext context);
            
             /**
             * Start the cache.
             *
             * @throws Exception for any error
             */
             void start() throws Exception;
            
             /**
             * Stop the cache.
             */
             void stop();
            }
            


            As you can see it caches VFSContex.
            Every time VFSContext is created, it is put into cache.
            This creation only happens - if you use VFS API - in VFS class.

            When you demand VirtualFile for given URI/URL,
            I match VFSContex's rootURI to get the right context for URI/URL.
            Then it's plain navigation from VFSContext to the actual relative VirtualFile.
            Relative path == URI/URL path - context's path.

            VFSCache comes in different impls: LRU, Timed, Soft, Weak.
            It's all configurable over jboss.vfs.cache System property.

            There is also optional CacheStatistics interface,
            which knows how to return:
            - cached contexts (unfortunately not for CachePolicy based cache)
            - cache size
            - last insert

            I expect only a dozen contexts in this cache when running JBossAS.
            e.g. deployers and deploy dir should be among them

            • 3. Re: Heavy usage of VirtualFileURLConnection
              Sacha Labourey Master

              and does it improve performance? how much?

              • 4. Re: Heavy usage of VirtualFileURLConnection
                Ales Justin Master

                 

                "sacha.labourey@jboss.com" wrote:
                and does it improve performance?

                No, it's there just for fun. :-)

                The way it's designed it should definitely improve performance.
                It's aimed at URL usage outside direct VFS usage.
                e.g. 3rd party libs that mostly work on CL::getResource(s)
                will now re-use existing VFSContext --> existing VirtualFile instances
                In simpler words, there won't be duplicated un-packaging. ;-)

                "sacha.labourey@jboss.com" wrote:

                how much?

                The original cause for this impl - Seam example redeploy,
                now takes ~10sec instead of previous 2min+.
                Is that good enough? ;-)

                • 6. Re: Heavy usage of VirtualFileURLConnection
                  Ales Justin Master

                  I added JMX statistics about this to JBossAS.
                  And I'm seeing some minor issue - nothing than cannot be fixed. ;-)

                  VFSContext Root URI
                  conf file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/conf/
                  jboss-aop-asintegration-core.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-asintegration-core.jar
                  lib file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/lib/
                  jboss-deployers-vfs.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-vfs.jar
                  jboss-mbeans.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-mbeans.jar
                  jboss-aop-asintegration-mc.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-asintegration-mc.jar
                  deploy file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deploy/
                  jboss-bindingservice.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/lib//jboss-bindingservice.jar
                  jboss-system-jmx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-system-jmx.jar
                  jaxb-impl.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jaxb-impl.jar
                  jboss-service.xml file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/conf/jboss-service.xml
                  jboss-deployers-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-spi.jar
                  jboss-aop-asintegration-jmx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-asintegration-jmx.jar
                  jboss-deployers-impl.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-impl.jar
                  jboss-deployers-client-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-client-spi.jar
                  wstx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//wstx.jar
                  jboss-deployers-structure-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-structure-spi.jar
                  jboss-system.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-system.jar
                  deployers file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deployers/
                  jboss-deployers-core-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-core-spi.jar
                  jboss-j2se.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-j2se.jar
                  jboss-jmx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-jmx.jar
                  jboss-deployers-client.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-client.jar
                  jboss-deployers-vfs-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-vfs-spi.jar
                  dom4j.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//dom4j.jar
                  jboss-deployers-core.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-core.jar
                  jboss-aop-jboss5.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-jboss5.jar


                  Just need to find out where that double // is coming in.
                  Otherwise it looks good == behaves as expected. :-)

                  • 7. Re: Heavy usage of VirtualFileURLConnection
                    Ales Justin Master

                     

                    "alesj" wrote:

                    Just need to find out where that double // is coming in.
                    Otherwise it looks good == behaves as expected. :-)

                    Down to just this:
                    VFSContext - root URI
                    file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib/
                    file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/conf/
                    file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deploy/
                    file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deployers/
                    file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/lib/

                    ;-)