1 Reply Latest reply on Jul 3, 2009 10:59 AM by Clebert Suconic

    Journal Compacting committed...

    Clebert Suconic Master

      I just committed the journal compacting into trunk.


      I have tested it on the performance labs, and compacting happens totally in parallel.

      I have added a test that was adding one message per second in a destination (without consuming it):

      num-messages=1000
      num-warmup-messages=0
      message-size=1024
      durable=true
      transacted=true
      batch-size=100
      drain-queue=true
      throttle-rate=10
      address=perfAddress2
      queue-name=perfQueue2
      host=172.16.8.10
      port=5445
      tcp-buffer=1048576
      tcp-no-delay=false
      send-window=1048576
      pre-ack=true
      block-ack=false
      block-persistent=false
      




      Then I started runSender and runListener as usual. Running both transactional (as the next code example) and non transactional.


      num-messages=500000
      num-warmup-messages=20000
      message-size=1024
      durable=true
      transacted=true
      batch-size=1000
      drain-queue=true
      queue-name=perfQueue
      throttle-rate=-1
      address=perfAddress
      queue-name=perfQueue
      host=172.16.8.10
      port=5445
      tcp-buffer=1048576
      tcp-no-delay=false
      send-window=1048576
      pre-ack=true
      block-ack=false
      block-persistent=false
      


      and I could get 28K messages / second if either having the first sender or or not.

      On my desktop, with a more limited disk.. I had 20K messages / second without the first sender.. and 15 messages / second with the first sender.


      I also compared the performance on the regular scenario on with and without those changes, and I could get the same performance as expected.



      Now... as I said on the other thread, we need to implement clean up next.

        • 1. Re: Journal Compacting committed...
          Clebert Suconic Master


          Having the compacting running too frequently will make compacting to compete for resources.

          Messaging-01 has a pretty fast disk, because of that the numbers were good on both scenarios.


          We have to minimize the need for compacting. Having living records on the journal will cause compacting to be executed too frequently. We need to cleanup the links between those files.