3 Replies Latest reply on Apr 24, 2008 5:27 PM by timfox

    JBM 2.0 new journal store

    timfox

      I have completed the first phase of the new fast journal storage for JBM 2.0 :)

      It's a fast append only journal, with full transaction and xa support that can be used to store any arbitrary records.

      The actual IO access is abstracted out so the same journal code can be used against different IO systems. So far I have create an implementation for Java file NIO (file channels), and this is now being used in TRUNK as the default storage mechanism - the entire test suite runs successfully against this.

      This is already showing pretty blinding performance :)

      Next stop is to plugin it into our new Linux async io layer. This is a thin JNI wrapper around Linux async IO (thanks to Clebert), and can plugged into our journal. This should give us even better peformance when running on Linux.

      The idea is we use Java NIO when running on other OSes and Linux async IO when running on Linux.

      What remains to be done is the file reclamation implementation.

      A blog entry is coming up on this.

        • 1. Re: JBM 2.0 new journal store
          slaboure

          Tim, instead of a small JNI wrapper, would it have been possible to use Mladen's APR-based code that he uses in JBoss Web? Does it provide that feature?

          Just a thought...

          • 2. Re: JBM 2.0 new journal store
            clebert.suconic

            It would be possible if we add this code to the APR-based code. But that will be using JNI anyway. If Mladen is opened to add this to the APR-based, maybe it would simplify deployment and binary distribution.

            With the AIO work I'm doing, I can max out my disk IO to its limits. I'm calling aio_submit to perform a write call, and I'm using DMA for that. For the max IO I got on my test I used too little of my CPU (30% of a 2 years old Dell D820).
            .

            • 3. Re: JBM 2.0 new journal store
              timfox

               

              "clebert.suconic@jboss.com" wrote:

              With the AIO work I'm doing, I can max out my disk IO to its limits. I'm calling aio_submit to perform a write call, and I'm using DMA for that. For the max IO I got on my test I used too little of my CPU (30% of a 2 years old Dell D820).
              .


              Yes, this is cool stuff. The performance we are seeing is pretty extra-ordinary. Can't wait until alpha so we can show it to the world :)