1 2 Previous Next 27 Replies Latest reply on Feb 22, 2010 5:27 PM by alrubinger Go to original post
      • 15. Re: Proposal: tree structure for Archive
        germanescobar

        aslak wrote:

         

        Formatter is API while Node is SPI, so strictly speaking it shouldn't know about it.

         

        The Formatters can use Archive.getContent(), as they do today?

         

        Well, they can. But we will have the same problems we have today. Take a look at FullFormatter or JdkZipExporterDelegate. It is not a problem of the implementations, is a problem with the API. Sometimes you want plain access to the archive contents, but other times you need transversal access. We should provide both to the clients.

         

        What we can do here is leave the API as it is now and let time decide. It would be pointless to add the NodeArchive implementation if no one can benefit from it. I would like to upload the code somewhere just for future reference though. But I really think is a problem that should be address from the API itself. 

        • 16. Re: Proposal: tree structure for Archive
          alrubinger

          Right, the current thing is workable but scales geometrically with respect to number of assets in the archive.

           

          I think the only solutions are:

           

          1. Promote Node to API
          2. Let the API impls assume that a specific SPI provider will be present under the hood, and runtime cast to it (ugly, bad assumptions, but hides stuff from the user)
          3. Make "FullFormatter" for instance an interface, and its got a NodeArchive-based implementation in base-impl.  Same for JdkZipExporterDelegate: it'd have to accept only NodeArchive.  Other impls could use other mechanisms.

           

          Does 3 sound do-able, German?

           

          S,

          ALR

          • 17. Re: Proposal: tree structure for Archive
            germanescobar

            I've have promoted Node to API with the following def:

             

            public interface Node
            {
               /**
                * @return The {@link Asset} this node holds, null if it is a directory
                */
               Asset getAsset();
            
               /**
                * @return The child nodes of this node or, an empty set if it has no 
                *           children or holds an asset. This method will never return null.
                */
               Set<Node> getChildren();
            
               /**
                * @return The path where this node is placed within the {@link Archive}
                */
               ArchivePath getPath();
            
            }
            

             

            I've also changed the following 4 methods of the Archive interface so they return nodes:

             

            Node get(ArchivePath path) throws IllegalArgumentException;
            
            Node get(String path) throws IllegalArgumentException;
            
            Map<ArchivePath, Node> getContent();
            
            Map<ArchivePath, Node> getContent(Filter<ArchivePath> filter);
            

             

            I changed implementations and tests accordingly. I had to change some tests because they expected one node but now every directory is a node so you should expect more entries. All tests pass except:

             

            • ZipExporterTestCase.testExportThrowsArchiveExceptionOnAssetWriteFailure() that hangs and haven´t figure out why (some deadlock scenario). It is commented.
            • The formatter tests. I haven´t fixed them accordingly. They also expect less entries. They are commented.
            • The Glassfish extension tests.

             

            I fixed the MemoryMapArchiveBase but I haven´t added NodeArchiveBase there yet. I also fixed JdkZipExporterDelegate and FullFormatter which are much more cleaner now.

             

            Implementation details

             

            When an Archive is created, it adds a Node with path "/". You can then get the root node using get("/") or get(""). So, now you can either retrieve the contents in a flat presentation with Archive.getContent() method or in a full or partial tree presentation with get("...").

             

            The convention I'm using is that every entry starts with "/" and ends without a trailing "/". On export, the preceding "/" is removed.

             

            I think that's it. I know we talked about not changing the Archive API but I just want to show you this code so you can tell me what you think. In my opinion, it is much more cleaner and flexible.I'm attaching the patch here until we decide whether we want to make this change or not.

             

            German.

            • 18. Re: Proposal: tree structure for Archive
              alrubinger

              German:

               

              I apologize your hard word here has gone unreviewed these past days; I've been down with the flu.  Will try to give it a close look today.

               

              S,

              ALR

              • 19. Re: Proposal: tree structure for Archive
                germanescobar

                Don't worry, get better first!

                 

                Besides, I've continued to work on this and I'm almost done. All the tests pass and it throws an Exception if an Illegal Path is introduced.

                 

                I've also fixed SHRINKWRAP-133 (the ZIP deadlock). I just add a dummy entry if two conditions are satisfied:

                 

                1. An exception was thrown.
                2. The ZIP file is empty.

                 

                I don´t know if it is too hacky.

                 

                I'll upload the path again soon with everything.

                • 20. Re: Proposal: tree structure for Archive
                  alrubinger

                  Cool, and I'll be at the Arquillian IRC meeting today.

                   

                  When you're making patches, mind executing:

                   

                  svn di > /path/to/patch

                   

                  From shrinkwrap/trunk?  Then all modules will be accounted for in one go.

                   

                  S,

                  ALR

                  • 21. Re: Proposal: tree structure for Archive
                    germanescobar

                    Ok, attached is my final proposal to this subject. I really think it's the right path to follow.

                     

                    List of changes

                     

                    1. A new Node interface (see previous posts).
                    2. Changes to the API and Impl to support the new Node interface.
                    3. DirectoryAsset was deleted.
                    4. Changes to the ZipImporter implementation classes to take advantage of the new transversal access to the archive.
                    5. Changes to the FullFormatter to take advantage of the new transversal access to the archive.
                    6. Minor changes to the tests to support the Node interface. They all pass now.
                    7. Changes to the glassfish extension to support the Node interface. Tests passing as well.
                    8. Propossed solution to SHRINKWRAP-133.
                    9. A new exception IllegalArchivePathException was created and it's thrown on an invalid path access. We are still discussing the name. More information: Illegal path

                     

                    Benefits of the solution

                     

                    You now have transversal access to the archive. The Archive.get("/") method will return the root Node. Then you can recursively call Node.getChildren() to navigate the whole tree. Plain access is still supported using the Archive.getContent() method. If the solution is accepted, we could introduce a new Archive.getRootNode() instead of the Archive.get("/") to obtain the root Node.

                     

                    You can also have partial access to the tree with Archive.get("/somepath...") which will return a Node.

                     

                    This problem is solved: Access to directories

                     

                    TODO

                     

                    1. When using Archive.addDirectories(String ... paths), it may throw an IllegalArchivePathException and some directories might not be added.
                    2. We need to remove either VerboseFormatter or FullFormatter. They both do the same. Or make a decision about what each should show.
                    • 22. Re: Proposal: tree structure for Archive
                      aslak

                      With this change we should support a visitor on the Archive interface to avoid everyone having to do the recursive looping on the outside..

                       

                      Something in the lines of:

                      {code:java}

                       

                      public interface Archive<T extends Archive<T>> extends Assignable {

                      ...

                       

                        void traverse(ArchiveTraverser traverser, Filter<Node> node);

                       

                      ...

                      }

                       

                      public interface ArchiveTraverser {

                       

                        void do(Node node);

                       

                      }

                      {code}

                       

                      {code:java}

                      archive.traverse(new BeansXMLInserter(), Filters.include(".*/META-INF/"));

                      {code}

                       

                      With this, Formatters and Exporters can be rewritten as:

                      {code:java}

                       

                      public void exportZip()

                      {

                         archive.traverse(new ZipExporter(), Filters.includeAll());

                      }

                       

                      public String format(Archive<?> archive)

                      {

                         StringBuffer output = new StirngBuffer();

                         archive.traverse(new VerboseFormater(output), Filters.includeAll());

                         return output.toString()

                      }

                      {code}

                      • 23. Re: Proposal: tree structure for Archive
                        alrubinger

                        aslak wrote:

                         

                        With this change we should support a visitor on the Archive interface to avoid everyone having to do the recursive looping on the outside..

                        Using a guideline of "make things hard on yourself if easier for the user", I'm not sure I agree.  In other words, exposing an API method in Archive that the typical end-user is unlikely to use isn't very attractive at first glance, even though it might make the impl code for Formatters/Exporters slimmer.

                         

                        S,

                        ALR

                        • 24. Re: Proposal: tree structure for Archive
                          alrubinger
                          Finally got a chance to bring in this patch and review.  Looks just as we discussed.

                          germanescobar wrote:


                          TODO

                           

                          1. When using Archive.addDirectories(String ... paths), it may throw an IllegalArchivePathException and some directories might not be added.
                          2. We need to remove either VerboseFormatter or FullFormatter. They both do the same. Or make a decision about what each should show.

                           

                          All operations must remain atomic, so 1) must either succeed or fully fail.

                           

                          I think "VerboseFormatter" is a better name, and probably should keep the "FullFormatter" mechanics.  I guess my original intent was to show only Assets that had been explicitly added (and not empty parent dirs), but that doesn't match "jar -tvf".  So I guess we should strike "FullFormatter".

                           

                          Couple minor notes:

                           

                          * String literal "/" in FullFormatter and VerboseFormatter could be a staticy finaly thing.

                          * The author email address has a stray "c" at the end?

                           

                          Should I be committing; I can't remember if you have access?

                           

                          S,

                          ALR

                          • 25. Re: Proposal: tree structure for Archive
                            germanescobar
                            Should I be committing; I can't remember if you have access?

                             

                            No, I don't have.

                             

                            I think "VerboseFormatter" is a better name, and probably should keep the "FullFormatter" mechanics.

                             

                            Ok, I'll change that.

                             

                            String literal "/" in FullFormatter and VerboseFormatter could be a staticy finaly thing.

                             

                            Ok, I'll also change that.

                             

                            The author email address has a stray "c" at the end?

                             

                            Yes, my GMail address is german.escobarc@gmail.com. The one without the c was already taken.

                            • 26. Re: Proposal: tree structure for Archive
                              alrubinger
                              • 27. Re: Proposal: tree structure for Archive
                                alrubinger

                                German, should should have RW access now; try a commit and let me know if there's an auth error.

                                 

                                S,

                                ALR

                                1 2 Previous Next