-
30. Re: Messaging and Remoting
To clarify what I was saying.
If we were to augment the remoting API to add methods that take ByteBuffers[] (or byte[] or whatever) for both receiving and sending, then in the non blocking case we would have to cope with partial sends and receives in the application.
We could of course build intelligence into the API to cope with the partial sends /receives - but this is pretty much the raison d'etre for NIO frameworks such as MINA AFAIK. So we'd be re-inventing the wheel.
If we didn't build that intelligence into the API, then we would have to handle it ourselves in the application layer, which again would be re-inventing the wheel, this time in the application.
If we agree that we don't want to build our own NIO framework then, I suggest we should just use one that we know works already. -
31. Re: Messaging and Remoting
"Tim" wrote:
"Ovidiu" wrote:
Where exactly did you see the word "stream" in the Remoting API extension document?
Why are you presuming I was referring to your posts? I was answering one of Clebert's posts where he *did* propose using streams.
Look at my other post about. The NIO API treats Streams as you could deal with then on Channels hence I thought it would be possible to work with Streams on Non block API. Later on I changed my suggestion to use ByteBuffer or any equivalent to not force us to clone byte arrays. -
32. Re: Messaging and Remoting
"clebert.suconic@jboss.com" wrote:
Look at my other post about. The NIO API treats Streams as you could deal with then on Channels hence I thought it would be possible to work with Streams on Non block API. Later on I changed my suggestion to use ByteBuffer or any equivalent to not force us to clone byte arrays.
Understood. :) -
33. Re: Messaging and Remoting
There are at least two distinct aspects that we need to discuss when talking NIO optimization.
First there is the fact that a non-blocking IO implementation, complete with readiness selection, allows a single thread (or a precisely limited number of threads) to efficiently do what in a thread-per-connection model would take a number of threads equal with the number of connections. This approach a) avoids thread context switching that becomes toxic after the number of threads in use reaches a certain threshold and b) prevents using the thread scheduler as a substitute for a readiness selection mechanism (threads that suddenly get something to read, or write, are un-blocked by the scheduler, which acts as a "de facto" readiness detector).
The second advantage of using NIO is the fact that, thanks to direct buffers, JVM client code can read and write directly physical memory that is also directly accessed by device drivers (the network driver or the DMA subsystem). The java code has no idea that it actually doesn't write JVM heap, but physical memory that is also mapped in the kernel space, so it can be directly accessed by the said devices. Regardless of how it is actually done, the net benefit is avoiding copying bytes over. However, one must keep in mind that direct buffers are much more heavier entities than simple byte arrays, they take a lot more time to allocate, and they are not subject to JVM memory management and garbage collection.
So, the thing I want to point out is that these two different aspects are ORTHOGONAL. I can pass a byte[] to a NIO implementation, and the implementation can make sure the buffer is completely read (or written) the same way it would make sure a ByteBuffer is completely read and written (and yes, that's true, by internally creating a direct ByteBuffer and copying bytes over).
So, I agree that there is an efficiency problem with using byte[] instead of direct ByteBuffers, but saying that an interface that use byte[] cannot inherently work with a NIO runtime is just wrong. A byte[] has nothing to do with non-blocking-ness per se.Tim wrote:
This is what non blocking is all about. A blocking implementation would block until all the bytes had been written. This preventing the thread neing used for anything else while the blocking is occurring.
No, this is not what non blocking is all about. There are two different (and orthogonal) aspects: non-blocking behavior + readiness selection AND direct buffers. You can avoid using direct buffers if you want to, and still get non-blocking behavior at the "lower" level of your framework. In the end, you want to achieve separation between your client code and the transport.Tim wrote:
This also means it's going to be up to the application to piece together requests and responses.
No.
Let's walk through a simple example. Let's say that you want to send a message acknowledgment from the client to the server, saying that message with id = 77 has been successfully processed by the client code.
With the API that I am proposing, you can do this:
You allocate a byte[] of five bytes. The first byte is the operation code. Let's assume for the sake of the example that the acknowledgment operation code is 11. The rest of four bytes contain the message id. So you end with a { 0x0B, 0x00, 0x00, 0x00, 0x4D } byte array, that you just hand over to "remoting" (or whatever superstructure based on NIO you want to use). The "remoting" module would wrap the byte[] in a ByteBuffer and passes it to a SocketChannel, writing it in a non-blocking manner on the channel using the thread it manages.
Your client thread returns immediately, and your { 0x0B, 0x00, 0x00, 0x00, 0x4D } acknowledgment will be, eventually, sent on the wire, by the "remoting" layer thread.
So you this way you achive a separation between your "client" thread, that just hands over the bytes, and the "remoting" thread that actually performs the non-blocking operations on the channel. This is as "SEDA" as you can get.Tim wrote:
It seems to me that our biggest issue is getting an implementation up and running. So far we haven't even got the interface sorted.
We will decide on a solution at the end of the remoting meeting that will take place this week. The solution I propose is one of the possible outcomes. Using an external framework it's another. Nothing is a priori overruled.
-
34. Re: Messaging and Remoting
"ovidiu.feodorov@jboss.com" wrote:
So, I agree that there is an efficiency problem with using byte[] instead of direct ByteBuffers, but saying that an interface that use byte[] cannot inherently work with a NIO runtime is just wrong. A byte[] has nothing to do with non-blocking-ness per se.
Well I don't think I ever said that.
The point I was making is that we shouldn't *prevent* people from using direct buffers if they want, by forcing them to use byte arrays.
I think it's a bad API that doesn't allow the user to exploit the full available power. Certainly the user should be careful, but it's user errror if they screw up.
I think we have now (at least tacitly) agreed that byte buffers should be exposed, not byte[] so this is rather moot.Tim wrote:
This is what non blocking is all about. A blocking implementation would block until all the bytes had been written. This preventing the thread neing used for anything else while the blocking is occurring.
No, this is not what non blocking is all about.
This *is* what non blocking is all about. Certainly NIO has other nice things like byte buffers and charset support, but we're talking about blocking here not NIO in general.
By avoiding blocking on reads or writes it enables a small number of threads to handle a large amound of "connections". If reads or writes were blocking that wouldn't be possible. That's the crux of the matter."Ovidiu" wrote:
There are two different (and orthogonal) aspects: non-blocking behavior + readiness selection AND direct buffers. You can avoid using direct buffers if you want to, and still get non-blocking behavior at the "lower" level of your framework. In the end, you want to achieve separation between your client code and the transport.Tim wrote:
This also means it's going to be up to the application to piece together requests and responses.
No.
Let's walk through a simple example. Let's say that you want to send a message acknowledgment from the client to the server, saying that message with id = 77 has been successfully processed by the client code...
To re-iterate what I said earlier.
Yes, you can handle the partial reads/writes in the framework which is basically what you are proposing.
However, this is one of the major value adds of NIO frameworks like MINA (I don't know about ember but I wouldn't be surprised if it did this too), so we are now getting into the realms of writing our NIO framework, and now the warning bells are ringing. Why not just use one that we know works?
Frameworks also seem to handle other stuff like byte buffer pooling (important with direct buffers as you have pointed out) and allow different selector/worker thread models to be used.
Also SSL support is built in (with MINA at least) although this is only with JDK5."Ovidiu" wrote:
We will decide on a solution at the end of the remoting meeting that will take place this week. The solution I propose is one of the possible outcomes. Using an external framework it's another. Nothing is a priori overruled.
Great :)
My 2c (again):
Facts:
a) We need NIO. (I'm not even mentioning multiplex here)
b) Remoting doesn't support NIO as we require it.
c) Other NIO frameworks exist which support what we want to do.
d) We must have this done in 4 weeks.
Options:
a) Write our own NIO framework. (Alarm bells!!)
b) Use an NIO framework that already exists
c) Wrap remoting around an existing NIO framework, and hope that remoting can somehow provide a better API.
Conclusions:
a) This is far too time consuming and error prone.
b) Is my preferred solution
c) Is something worth exploring, but I fail to see what value remoting would add by wrapping the underlying NIO interface. What could it do better?
I would suggest b), then in phase 2, see if it makes sense to combine this with remoting somehow. -
35. Re: Messaging and Remoting
Tim wrote:
I think we have now (at least tacitly) agreed that byte buffers should be exposed, not byte[] so this is rather moot.
Let's make this explicit. Yes, ByteBuffer should be exposed. I said this may days and many posts ago. The problem in contention is where should they be exposed? At the top level or by one of the "layers". This is what we're trying to decide during the meeting today. -
36. Re: Messaging and Remoting
Tim wrote:
By avoiding blocking on reads or writes it enables a small number of threads to handle a large amound of "connections". If reads or writes were blocking that wouldn't be possible. That's the crux of the matter.
Well, read again my previous post. I am quoting it here for completness:
"First there is the fact that a non-blocking IO implementation, complete with readiness selection, allows a single thread (or a precisely limited number of threads) to efficiently do what in a thread-per-connection model would take a number of threads equal with the number of connections. This approach a) avoids thread context switching that becomes toxic after the number of threads in use reaches a certain threshold and b) prevents using the thread scheduler as a substitute for a readiness selection mechanism (threads that suddenly get something to read, or write, are un-blocked by the scheduler, which acts as a "de facto" readiness detector)."
So I really don't see what we're arguing about. -
37. Re: Messaging and Remoting
For the NIO framework, JGroups has implemented this for one of its transport protocols. It seems to be working quite well. I think it was developed by one of the Novell guys.
-
-
39. Re: Messaging and Remoting
While working on the smoke tests, I realized that as I was incorporating the Remoting http transport into Messaging, I didn't account for the problem of version compatibility. I moved callback handling down into the Remoting layer so that all transports could use the same code, but the result is a change in the format of message-bearing callbacks. Older clients (including the current generation on trunk) wouldn't recognize the new format.
Before I re-incorporate the old code to accomodate socket transport users, I just want to verify that it is necessary. -
40. Re: Messaging and Remoting
I wouldn't spend to much time with this, we gave up guaranteeing compatibilty until 1.2.