9 Replies Latest reply on May 14, 2008 12:13 PM by Clebert Suconic

    MaxIO, MinFiles and JournalFileSize

    Clebert Suconic Master

      When you initialize libaio, you also initialize the number of AIO positions you will have on the kernel. That is the maximum number of pending writes you could have on the kernel. There is an almost hard limit on the OS, I guess it's 25K (I say almost.. because the variable it's not documented, or if it is.. it's hard to find it).

      After few tests I have done, I've been able to get very good numbers, using the StoreManager directly. But the best number I could get to my disk was 10000. With that we can't have more than 2 files opened with AIO. Since the journal is append only, I guess it would be okay to make it a little bigger, since we won't make any reads on the file.

      I will make the MaxIO configurable on the Configuration, but IMO the default for min files should be 2, and the size of each journal at 100M.

      Putting the file at 100M would minimize the number of closes/opens you would do in production also.

      When you close a file in AIO, you need to wait all the pending writes to complete before you can actually close it and move to the next file.

      (Maybe I should use an executor to close the file to avoid the eventual latency, but if you could minimize the number of closes on AIO it would still a good practice IMO)