-
1. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 6:19 AM (in response to timfox)Yesterday, when we discussed the meaning of max io you told me it limited the size of the write queue - but that's not true for AIO.
Based on that information I set max io to a small value as discussed. But of course this won't work for AIO since for AIO it really means number of uncompleted IO operations.
Setting maxio to a small value (10) in my tests yesterday gave an artificially small value for throughput, since it was never allowing more than 10 IO operations in progress. This makes yesterday's results invalid.
I'm going to start again today. -
2. Re: Semantics of maxIO are different for NIO and AIO
clebert.suconic Nov 29, 2009 11:10 AM (in response to timfox)NIOWrites are blocking, so the maxIO should be released as soon as the write operation is complete.
AIOWrite are non-blocking, the maxIO should be released *again* as soon as the write operation is complete which happens on a callback.
It's implemented on different places due to the asynchronous nature. How's that different in semantics? It's releasing when the operation is complete, right? -
3. Re: Semantics of maxIO are different for NIO and AIO
clebert.suconic Nov 29, 2009 12:17 PM (in response to timfox)One thing we could change on NIO, to only use a semaphore when a sync is requested.
The idea is to maximize the number of syncs, so as NIO behaves different than AIO by definition, maybe we could do that.
on AIO a write means go to disk, while on NIO it means go to the OS buffer (and eventually to the disk). -
4. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 1:04 PM (in response to timfox)The semantics are different, since, like I said before, with NIO, the max_io limits the size of the write queue, whereas with AIO it does not.
-
5. Re: Semantics of maxIO are different for NIO and AIO
clebert.suconic Nov 29, 2009 2:04 PM (in response to timfox)Aio would be throwing native errors if that was True
-
6. Re: Semantics of maxIO are different for NIO and AIO
clebert.suconic Nov 29, 2009 2:07 PM (in response to timfox)Keep in mind the write queue on AIO is inside the kernel while on Nio is on the executor.
-
7. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:35 PM (in response to timfox)With NIO, the semaphore is acquired in the writeInternal() method of NIOSequentialFile:
if (writerExecutor == null) { doInternalWrite(bytes, sync, callback); } else { // This is a flow control on writing, just like maxAIO on libaio maxIOSemaphore.acquire();
However, with AIO, the semaphore is not acquired until the request has been pulled off the queue:
AsynchronousFileImpl::write()if (writeExecutor != null) { writeExecutor.execute(new Runnable() { public void run() { maxIOSemaphore.acquireUninterruptibly();
Am I making more sense now?
Summary:
With NIO, you acquire the semaphore *before* you add the request to the write queue, but with AIO you acquire the semaphore *after* you remove the element from the write queue.
This means the semantics of max_io are different for AIO and NIO. -
8. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:37 PM (in response to timfox)"clebert.suconic@jboss.com" wrote:
Keep in mind the write queue on AIO is inside the kernel while on Nio is on the executor.
Nope, there is an executor in Java for AIO too.
AsynchronousFileImpl::writeExecutor -
9. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:38 PM (in response to timfox)The fix here is to acquire the semaphore for AIO before the element is added to the queue.
This will make the semantics of max_io the same for NIO and AIO. -
10. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:40 PM (in response to timfox)"clebert.suconic@jboss.com" wrote:
Aio would be throwing native errors if that was True
No.
The write queue is in Java. So if you added too many requests, too quickly you'd get an Java OOM, not a native error. -
11. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:46 PM (in response to timfox)Ok, this is fixed locally
-
12. Re: Semantics of maxIO are different for NIO and AIO
clebert.suconic Nov 29, 2009 2:48 PM (in response to timfox)Ok you can get the semaphore before adding to the executor on aio however I don't think it will make much difference since we don't wait anything on aioWrite.
The only reason we use an executor on aio is to avoid context switch on the kernel. It's different. -
13. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:51 PM (in response to timfox)It's all about consistency
-
14. Re: Semantics of maxIO are different for NIO and AIO
timfox Nov 29, 2009 2:54 PM (in response to timfox)I think if i said the grass was green, you would argue with me that is was blue.