13 Replies Latest reply on Sep 8, 2006 12:27 PM by starksm64

    VFS issues

      I'm currently semi-stalled on VFS issues.

      To illustrate some of the problems (which might be that I'm using it wrong :-)
      I've added an org.jboss.test.vfs.SundryVFSTests to the container module.

        • 1. Re: VFS issues
          starksm64

          I'll take a look when I get a chance latter tonight. I assume this is in the MC_VDF_WORK branch?

          • 2. Re: VFS issues

            I've created an alternative version of the VFS prototype in
            org.jboss.virtual
            (I didn't want to break EJB3's usage of the org.jboss.vfs).

            The main effort is to try to make it consistent
            in how it creates virtual files rather having repeated
            (but inconsistent) code across the retrieval mechanism.

            The org.jboss.virtual is a refactoring of org.jboss.vfs
            (I left the original authors in place where relevant).

            The other changes are:

            1) A split into user wrappers and the real implementation.
            User (api) -> Real (spi)
            VFS -> VFSContext (old ReadOnlyVFS)
            VirtualFile -> VirtualFileHandler (old VirtualFile)
            The user api is made of concrete classes that delegate to the spi

            The user api is in org.jboss.virtual,
            the spi is in org.jboss.virtual.spi

            2) VFS also serves as the initial factory (wrapping the locator
            with a simpler static api).

            3) Abstract support for implementions, basically all you need
            to implement is getting the root in the VFSContext
            and then (besides the attributes) getChildren() and findChild()
            in the VirtualFileHandler.
            There is also some simple support for findChild based on
            two models:

            a) StructureVirtualFileHandler - which bumps through
            each context of the path, i.e. com/acme/Blah
            is effectively (but more efficiently) turned into

            start.findChild("com").findChild("acme").findChild("Blah");
            

            This is what the file vfs does.

            b) Simple - where all the entries are known to the root
            this is what the jar stuff does.

            You don't have to use either of these of course. :-)

            4) A cleaner and more expressive query api based on the visitor
            pattern. Other simpler queries in the api are wrappers for this.
            The default visit impl is based on the getChildren() from the spi
            so you don't need to write any of this yourself.

            5) Paths work whether they end in a / or not
            com/acme/directory/ == com/acme/directory
            the only reason for an explicit slash is when you want to
            express "contents of directory" rather than the directory,
            which isn't relevant for this api.
            There is isDirectory() if you need to know what type it is.

            Things missing that are in org.jboss.vfs
            1) Serialization of the VFS objects
            2) Internal caching
            3) Support for "search context"
            4) Nested jars are unpacked into temp files rather than
            in memory byte buffers.

            Finally, I've done a lot manual testing, but there is no
            complete testsuite. I did port the original tests where relevant
            and modified to the new api.

            • 3. Re: VFS issues

              The slight difference due to the simplification of the api can be
              seen in this example:

              Old test

               VFSFactory factory = VFSFactoryLocator.getFactory(rootURL);
               ReadOnlyVFS vfs = factory.getVFS(rootURL);
              
               // Check resolving the root file
               VirtualFile root = vfs.resolveFile("");
              
               // Find the outer.jar
               VirtualFile outerJar = vfs.resolveFile("outer.jar");
              


              New test
               VFS vfs = VFS.getVFS(rootURL);
              
               // Check resolving the root file
               VirtualFile root = vfs.findChildFromRoot("");
              
               // Find the outer.jar
               VirtualFile outerJar = vfs.findChildFromRoot("outer.jar");
              


              Additionally, the new stuff supports the follow factory methods
              which are really just wrappers to the other primitives.
              // establish a context and get a file
              vfs.getVirtualFile(rootURL, "outer.jar");
              
              // Simplified access to the root
              vfs.getRoot(); == vfs.findChildFromRoot("");
              
              // Relative access
              VirtualFile outer = vfs.getVirtualFile(rootURL, "outer.jar");
              vfs.findChild(outer, "META-INF/MANIFEST.MF");
              
              // The latter is really just the simpler
              outer.findChild("META-INF/MANIFEST.MF");
              


              • 4. Re: VFS issues

                Since this impl is working, I'm going to get on with writing the
                deployers based on it.

                I'll come back to write the full regression tests when I have more time.

                Since the deployers only uses the public api (which is nearly the same)
                it is easy to change it either way.

                The two impls need to be merged when EJB3 is ready.

                • 5. Re: VFS issues
                  starksm64

                  I'm having trouble checking out the mc contents to pick this up:

                  [sstark@sstark JBossMC]$ svn co https://svn.jboss.org/repos/jbossas/projects/microcontainer/trunk/ jbossmc >svn.log
                  
                  svn: Can't copy 'jbossmc/kernel/src/resources/xml-test/org/jboss/test/kernel/con
                  fig/test/.svn/tmp/text-base/testCustomMapPreinstantiated.xml.svn-base' to 'jbossmc/kernel/src/resources/xml-test/org/jboss/test/kernel/config/test/testCustomMap
                  Preinstantiated.xml.tmp': No such file or directory
                  [sstark@sstark JBossMC]$
                  



                  • 6. Re: VFS issues
                    bill.burke

                    EJB3 does not use VFS. If you are talking about the small deployer prototype I did, then that is not used by EJB3. Due to negative feedback from yourself, I didn't go forward with it.

                    • 7. Re: VFS issues
                      starksm64

                      I have merged the nested jar capability into the org.jboss.virtual tree, and to support this added a notion of an options map to the VFSContext. This is curretnly parsed from the rootURL query string. It could be made an explicit map of options that does not require a String key and value.

                      I'm also working on the Serialization support, but the hiearchy in the abstract plugin classes is not consistent with the data that needs to be written out. The problem is that the AbstractVirtualFileHandler has the VFSContext reference, but it does not have the URL needed to recreate this. This exists in the AbstractURLHandler. Either the rootURL is needed at the AbstractVirtualFileHandler, or the handler url is pushed to this level, in which case I don't see much point to the AbstractURLHandler.

                      • 8. Re: VFS issues

                        The AbstractVirtualFileHandler does have the reference,
                        it is just via a virtual method: getRootURL();

                        The idea is that we might want to support virtual files
                        that are not directly URL based, but do provide a unique identifier.

                        In fact, the api should really be URI based to make this a real possibility.
                        With only those handlers that are based on a real URL having
                        a URI that can be used to recreate the URL.

                        In anycase, don't you have a problem that each deserialized
                        virtual file handler will end up in its own copy of the context?
                        That is unless the locator holds some kind of a map of rootURL -> context.

                        Also, I think you should make VirtualFileHandler implement Serializable
                        as a part of the intended contract rather doing it on each implementation.

                        • 9. Re: VFS issues
                          starksm64

                           

                          "adrian@jboss.org" wrote:
                          The AbstractVirtualFileHandler does have the reference,
                          it is just via a virtual method: getRootURL();

                          The idea is that we might want to support virtual files
                          that are not directly URL based, but do provide a unique identifier.

                          Ok, so I need to write out the rootURL to be able to read it back in.

                          "adrian@jboss.org" wrote:

                          In fact, the api should really be URI based to make this a real possibility.
                          With only those handlers that are based on a real URL having
                          a URI that can be used to recreate the URL.

                          In anycase, don't you have a problem that each deserialized
                          virtual file handler will end up in its own copy of the context?
                          That is unless the locator holds some kind of a map of rootURL -> context.

                          When I originally prototype the vfs code, I did start out with URI. The problem was that you always had to go to the URL in order to get the io streams, or come up with another URI handler framework. Its a decision we need to make.

                          Yes, we would need a mapping in context factory to avoid explosion of vfs contexts on deserialization.

                          "adrian@jboss.org" wrote:

                          Also, I think you should make VirtualFileHandler implement Serializable
                          as a part of the intended contract rather doing it on each implementation.

                          No problem with that.


                          • 10. Re: VFS issues

                             

                            "scott.stark@jboss.org" wrote:

                            When I originally prototype the vfs code, I did start out with URI. The problem was that you always had to go to the URL in order to get the io streams, or come up with another URI handler framework. Its a decision we need to make.


                            But isn't that the whole point. If the URI really is a URL
                            then the handler extends AbstractURLHandler and gets
                            (or it can reimplement it).

                             public InputStream openStream() throws IOException
                             {
                             checkClosed();
                             return url.openStream();
                             }
                            


                            But if it is not really URL based then the handler
                            has to provide its own implementation.
                            i.e. a mapping to the real resource it is trying to represent with the
                            more logical URI.

                            The real issue is whether you also need
                            to write a URLStreamHandler besides the virtual file stuff.

                            • 11. Re: VFS issues
                              starksm64

                               

                              "adrian@jboss.org" wrote:

                              But isn't that the whole point. If the URI really is a URL
                              then the handler extends AbstractURLHandler and gets
                              (or it can reimplement it).
                              ...
                              But if it is not really URL based then the handler
                              has to provide its own implementation.
                              i.e. a mapping to the real resource it is trying to represent with the
                              more logical URI.

                              The real issue is whether you also need
                              to write a URLStreamHandler besides the virtual file stuff.

                              Yes, good point. Separating out the URL/URI nature should cleanup the issues. I should look at adding the URI support before going on with serialization then.


                              • 12. Re: VFS issues
                                alesj

                                 

                                "scott.stark@jboss.org" wrote:
                                I'm having trouble checking out the mc contents to pick this up:

                                
                                [sstark@sstark JBossMC]$ svn co https://svn.jboss.org/repos/jbossas/projects/microcontainer/trunk/ jbossmc >svn.log
                                
                                svn: Can't copy 'jbossmc/kernel/src/resources/xml-test/org/jboss/test/kernel/con
                                fig/test/.svn/tmp/text-base/testCustomMapPreinstantiated.xml.svn-base' to 'jbossmc/kernel/src/resources/xml-test/org/jboss/test/kernel/config/test/testCustomMap
                                Preinstantiated.xml.tmp': No such file or directory
                                [sstark@sstark JBossMC]$
                                



                                This one is mine.
                                I had some problems with renaming the file (from small i to big I in PreInstantiated).
                                I hope this is resolved now.

                                • 13. Re: VFS issues
                                  starksm64

                                   

                                  "alesj" wrote:

                                  This one is mine.
                                  I had some problems with renaming the file (from small i to big I in PreInstantiated).
                                  I hope this is resolved now.

                                  Yes. I deleted the testCustomMapPreinstantiated.xml when Alex pointed out this was a known problem on case insensative file systems.