3 Replies Latest reply on Sep 4, 2008 11:09 AM by clebert.suconic

    Thread usage in paging

    timfox

      I've just noticed that paging uses a new thread per address.

      This isn't going to work, since we may have many thousands or tens of thousands of addresses.

      Instead you could get a thread from a cached thread pool to perform the depage when PagerImpl::messageDone is called.

      Also having many different PagingStores paging/depaging concurrently to the file system may provide perf problems.

      I'd really like to see some results of this performance. It might be a better to make sure only one page/or depage is done at a time (queue them for all destinations and use a single thread?), otherwise the disk head will be skipping all over the place.

        • 1. Re: Thread usage in paging
          clebert.suconic

          OK... I am changing it to use an executor. I will start with a single-threaded executor. We can change it later if we want more than one depage thread working at any time.

          • 2. Re: Thread usage in paging
            timfox

            Just use a cached thread pool initialised with a JBMThreadFactory - see other places in the code that do this already.

            • 3. Re: Thread usage in paging
              clebert.suconic

              I have one executor being created at PagingManagerFactoryNIO. That executor is shared with all the PagingStores (i.e. all the addresses):

              Before change:

               public PagingManagerFactoryNIO(final String directory)
               {
               this.directory = directory;
               this.executor = Executors.newSingleThreadExecutor();
               }
              



              From what I've seen in other places all I would need to do is change how I initialize the executor? Or I am missing something here?



              After change:
               public PagingManagerFactoryNIO(final String directory)
               {
               this.directory = directory;
               this.executor = Executors.newCachedThreadPool(new JBMThreadFactory("JBM-depaging-threads"));
               }