1 2 Previous Next 15 Replies Latest reply on Jun 26, 2008 6:12 AM by dimitris

    VFS HDScanner test

    mstruk

      The first reason this test is failing is because the default mode for vfszip is asynchronous lock reaping. That means the archive remains locked for 5 seconds after it was last used.

      Also, this test forces archive reuse (and consequently a file lock) continuously in two ways:


      a) MockProfileServiceRepository creates a new VFSContext on every deployment scan:

       VirtualFile deployDir = VFS.getRoot(applicationDir.toURI());
      

      instead of holding on to one instance of root VirtualFile, thereby forcing reinitialization of arhive metadata on every scan, which causes a file lock to be acquired and also slows things down.


      b) The test itself forces redeploy by continuously changing lastModified - again causing a file lock to be acquired.

      Every time a file lock is acquired it takes 5 seconds (after no longer needed - which in most cases is a split second after it was acquired) for it to clear.



      Possible solutions are:

      1) put the following at the beginning of testDeleteWhileScanning() to activate synchronous lock release (access to archive entries becomes a bit slower).
       System.setProperty("jboss.vfs.forceNoReaper", "true");
      


      2) Stop the scanner first, wait for 5 seconds and then do a delete (but I guess the whole purpose of the test is to do a delete while the scanner is still active)

      3) Rewrite MockProfileServiceRepository to reuse deployDir VirtualFile. Tweak scan, and wait periods to bigger values + try archive.delete() multiple times - to catch a window when archive is unlocked.

       VirtualFile deployDir = getDeploymentRoot(applicationDir.toURI());
      
      ...
      
       private VirtualFile deploymentRoot;
      
       protected VirtualFile getDeploymentRoot(URI uri) throws IOException
       {
       if (deploymentRoot == null)
       {
       deploymentRoot = VFS.getRoot(uri);
       }
       return deploymentRoot;
       }
      


      This approach relies on correct interplay of lastModified changes, and scanner and reaper periods which are internally hardcoded and can possibly change without notice - it's not a reliable approach.

      I opted for approach one. It's commited.


      Cheers,

      - marko

        • 1. Re: VFS HDScanner test
          alesj

           

          "mstruk" wrote:

          I opted for approach one. It's commited.

          This is wrong. :-)
          Since this will only work if you run this test first (or just this test) and no-one touches the reaper initialization code before.

          You probably need something similar to what I did in AbstractVFSTest to get option on (no)copy usage.

          • 2. Re: VFS HDScanner test
            mstruk

            Hmm, I see.

            Currently there is only one test method in HDScannerTestCase, I run the tests through CLI, and each test is run forked ... It works for all these:

            mvn package
            mvn test
            mvn test -Dtest=HDScannerTestCase
            

            So people run these in ways that they don't get forked?


            • 3. Re: VFS HDScanner test
              alesj

              Their correct behavior shouldn't depend on the way they are run. ;-)

              • 4. Re: VFS HDScanner test
                alesj

                So, if I understand this correct, if we're to 'cache' this line

                VirtualFile deployDir = VFS.getRoot(applicationDir.toURI());
                

                there should be no problem in real PS usage where frequency of new deployments is rare?

                • 5. Re: VFS HDScanner test
                  mstruk


                  Correct.

                  • 6. Re: VFS HDScanner test
                    starksm64

                    The point of the test was to illustrate the current usage that causes locking problems. How one thread can reliably delete a file when there are scanner threads that may have recently been active is what needs to be handled. It does not matter if the deployDir(in general there can be multiple dirs) should be cached from the initial point where the deploy dir uris are setup.

                    There should be a better api than having to set a system property to clear all unused locks. Can't this be incorporated into the VirtualFileHandler delete implementation?

                    • 7. Re: VFS HDScanner test
                      mstruk

                       


                      There should be a better api than having to set a system property to clear all unused locks. Can't this be incorporated into the VirtualFileHandler delete implementation?


                      Yes, we could add delete() method to VirtualFileHandler, and VirtualFile.

                      How should this method behave in the case of zip entries? It could be a no-op, or it could throw an exception ...

                      It's possible that someone causes locks outside of jbossas so even with this method a file delete is not guaranteed, we could return false in this case like File.delete() does.

                      Also, it's possible that the file we're trying to delete is in the middle of legitimate use. We can be 'bastards' and forcefully close the ZipFile and delete it. Or we can have a short grace period.


                      VirtualFileHandler:
                       /**
                       * Delete a file represented by this handler
                       *
                       * @param gracePeriod max time to wait for locks (in milliseconds)
                       * @return boolean true if file was deleted, false otherwise
                       * @throws IOException for any error
                       */
                       public boolean delete(int gracePeriod) throws IOException;
                      




                      • 8. Re: VFS HDScanner test
                        starksm64

                        Right, there are a number of issues/usescases that we need tests to validate the expected behavior of. The main issue is that we need to be able to write a piece of deterministic code relative to our lock usage, meaning, if the only accessors of the file is the VFS, I can know whether or not the delete should succeed.

                        We still should return true/false as to whether the delete succeeds so calling code can deal with an unexpected failure, or an expected failure as might occur if there is no guarantee that a legitimate lock may exist.

                        • 9. Re: VFS HDScanner test
                          slaboure

                          Shouldn't we simply use an OS-dependent solution? i.e. depending on which OS JBoss is running on, we would use a different solution. That would mean that on some OSes, JBoss would be faster/more efficient, while on others it wouldn't. But at least, we would have a safe way of handling all situations. When I read that some solutions rely on timers, it freezes my spine.

                          Also, given that JBoss Web will be included in JBoss 5.0 and given that we will provide by default APR libraries for most OS, couldn't we also use this as part of the OS-specific check and rely on OS-specific native calls to detect whether a file is currently being opened/locked or not? Between the presence of APR libraries and OS-specific checks we should be able to offer a performing solution on most if not all OSes, no?

                          • 10. Re: VFS HDScanner test
                            alesj

                            Perhaps in version 3.x (and when we hire Marko to do full time on VFS). ;-)

                            Currently we need to get this working to a decent level, covering all stumbling issues; WS serialization usage, Winz locking, ...

                            Golden rule: "make it work, optimize later'. :-)

                            • 11. Re: VFS HDScanner test
                              slaboure

                              well, yeah, then always copy to /tmp !

                              For me, using things like "5 seconds timers" are not synonym with "make it work"

                              • 12. Re: VFS HDScanner test
                                alesj

                                 

                                "sacha.labourey@jboss.com" wrote:

                                For me, using things like "5 seconds timers" are not synonym with "make it work"

                                Probably because you didn't follow all previous discussions on Winz locking issues. ;-)

                                • 13. Re: VFS HDScanner test
                                  slaboure

                                  so you are saying that we currently have a safe solution (not relying on tricks) for all OSes?

                                  • 14. Re: VFS HDScanner test
                                    alesj

                                    I don't see that (reaper) as a trick.
                                    It can be called pre-mature optimization, and in our case a good one, in more ways:
                                    1) exposed new issues
                                    2) removed locking
                                    3) does fair optimization

                                    How safe the solution is, only good tests can show. ;-)
                                    And we're making sure every issue we encounter is put in VFS's tests.

                                    1 2 Previous Next