10 Replies Latest reply on Jan 6, 2009 10:46 AM by timfox

    Maximum path length easly exceeded

    timfox

      It's easy to get the system to fail by specifying an address that's long - the paging system attempts to create directory even though it's not paging.

      (Why create the directory if not paging?)

      Most OSes have limits on the maximum path length e.g. 4095 in Linux, so clearly using the address name is not going to work.

      A better solution needs to be worked out - e..g store the real address in a file in the paging directory.

        • 1. Re: Maximum path length easly exceeded
          timfox

          Actually on my Linux lap top, it fails at 255 characters. Here's the test program:

          public void testCreateDirs() throws Exception
           {
           String dir = "/tmp/jbm-unit-test/page/";
          
           for (int i = 0; i < 1000; i++)
           {
           dir += "X";
          
           File f = new File(dir);
          
           boolean ok = f.mkdirs();
          
           if (!ok)
           {
           throw new IOException("failed at " + i);
           }
           }
           }
          


          • 2. Re: Maximum path length easly exceeded
            timfox

            Also, bear in mind that base 64 encoding will increase the length of the string, so the actual address might be significantly less than 255 chars.

            • 3. Re: Maximum path length easly exceeded
              clebert.suconic

               

              "timfox" wrote:
              It's easy to get the system to fail by specifying an address that's long - the paging system attempts to create directory even though it's not paging.

              (Why create the directory if not paging?)

              Most OSes have limits on the maximum path length e.g. 4095 in Linux, so clearly using the address name is not going to work.

              A better solution needs to be worked out - e..g store the real address in a file in the paging directory.



              I have changed it to add a record on the BindingJournal (thorugh StorageManager). The record will be hold only while in-page-mode.

              Another change I've done was also to delete the directory when out of the page mode. This is because I didn't want records hanging around on the StorageManager forever.

              • 4. Re: Maximum path length easly exceeded
                timfox

                 

                "clebert.suconic@jboss.com" wrote:

                I have changed it to add a record on the BindingJournal (thorugh StorageManager). The record will be hold only while in-page-mode.

                Another change I've done was also to delete the directory when out of the page mode. This is because I didn't want records hanging around on the StorageManager forever.


                Can't you just store it in a file in the dir as I suggested? It's simpler and we don't have a load of unnecessary records stored in the journal. I am already trying to get rid of all these paging related records and simplify things.

                • 5. Re: Maximum path length easly exceeded
                  timfox

                  And please... remain focussed here. At this stage in the game you haven't got time to spend a whole day doing it some other way, only to have to re-do it the next day.

                  • 6. Re: Maximum path length easly exceeded
                    timfox

                    I have reverted your last commit.

                    To clarify, the way it should be done is as follows:

                    1) Each page store has its own directory, but the directory name is NOT the address. Instead the directory name can just be a simple GUID.

                    2) In each paging directory the actual address is stored in a file "address.txt".

                    3) When reloading page stores, simply open all the adress.txt files and get the address from there

                    • 7. Re: Maximum path length easly exceeded
                      clebert.suconic

                      I can't just store destinations on a TXT. If the user is creating and deleting destinations very frequently (say temporary destinations), when routing we will be creating too many entires on the TXT.

                      Besides the TXT could be easily broken by crashes. That's why I have used the journal.

                      What took some time yesterday was creating the directory lazily... not adding a record on the Journal, and replacing the journal by a TXT on that part would be the same complexity. On the process I had screwed up one thing and it took me some time to figure out.

                      What I have done was only create the entry when it was in page mode, and remove the destinations from the storage as soon as it left the page-mode.

                      I don't think a simpler solution as you suggest would be the best idea because of the first point I'm raising here.


                      I will come online to discuss this in 30 min

                      • 8. Re: Maximum path length easly exceeded
                        timfox

                         

                        "clebert.suconic@jboss.com" wrote:
                        I can't just store destinations on a TXT. If the user is creating and deleting destinations very frequently (say temporary destinations), when routing we will be creating too many entires on the TXT.


                        There will only ever be one address in the tx file -there is one text file per address.


                        Besides the TXT could be easily broken by crashes.


                        The paging files could also be broken by crashes, it's no different.


                        What took some time yesterday was creating the directory lazily... not adding a record on the Journal, and replacing the journal by a TXT on that part would be the same complexity. On the process I had screwed up one thing and it took me some time to figure out.

                        What I have done was only create the entry when it was in page mode, and remove the destinations from the storage as soon as it left the page-mode.


                        Not a good idea - there's no need to do this especially if the address is flipping between paging and not, you will get a lot of thrashing.

                        The text file will only ever be created once - when a destination starts paging.

                        • 9. Re: Maximum path length easly exceeded
                          clebert.suconic

                           

                          The paging files could also be broken by crashes, it's no different.


                          It is different... we have put a lot of effort in making the Journal transactional, validating the health of records.. and etc...

                          If we start having a file per address it's better, but once a destination gets on page-mode, you will have that file on disk until someome manually delete it.

                          Think about if a temporary destination gets in page mode. What will remove that file?

                          • 10. Re: Maximum path length easly exceeded
                            timfox

                             

                            "clebert.suconic@jboss.com" wrote:
                            The paging files could also be broken by crashes, it's no different.


                            It is different... we have put a lot of effort in making the Journal transactional, validating the health of records.. and etc...

                            If we start having a file per address it's better, but once a destination gets on page-mode, you will have that file on disk until someome manually delete it.


                            Reaper thread.