3 Replies Latest reply on Jun 6, 2010 11:41 AM by Clebert Suconic

    Don't create new pools

    Tim Fox Master

      Just been looking at JournalImpl - I notice it creates a single thread executor (for files) and a thread pool (for compacting).

       

      Modules shouldn't create their own pools. They should use the main pool from the server. Otherwise we have an unmanageable mess of pools.

       

      This is quite an important point. Take a look at the thread pooling chapter in the user manual.

        • 1. Re: Don't create new pools
          Clebert Suconic Master

          I don't think the journal should share any other pools.

           

          I believe I mentioned this somewhere before.

           

          Say if the server is busy serving requests, and if the journal needs a new file, you would have a starvation...  the requests won't finish because you don't have files, and the journal will be waiting a thread from the main pool.

          • 2. Re: Don't create new pools
            Tim Fox Master

            You could say the same about any component. Why is the journal more special than any other component?

            • 3. Re: Don't create new pools
              Clebert Suconic Master

              First, one of the executors is used to write on AIO. It always need to be the same thread due to how AIO works at the kernel level. If I was using the main pools I would use different threads each time what would decrease performance on AIO...

              wasn't a reason: If you run out of threads during an append(sync) call (done at the bindings journal for instance), the entire system

               

              Second, Maybe I could use a pool for the compactor. However it will be using a thread for a time usually longer than what used by other components since it will be doing a lot of AIO.

               

               

              I will make a more complete analysis of those executors usage on monday and post it here.