-
1. Re: Integrating Netty into JBM2
trustin Jul 1, 2008 8:26 AM (in response to trustin)Even further, I think we can reuse DataInput and DataOutput interface instead of MessagingBuffer. The reasoning behind this is that putInt() is the only random access method in MessagingBuffer and it's only used for writing the length field during encoding. Except that, every access is sequential.
What the current decoder implementation does is to read the length field first and wait until the whole message is received, which means the whole message is read into memory. Therefore, using a stream to read a message will never block once a message is properly split (or assembled) in the networking layer (MINA or Netty level).
Writing a message is a different matter because we need to fill the length field. We could fill the length field in the networking layer (MINA or Netty level) to work around this issue.
The disadvantage of this approach is that each networking layer implementation should know how one or more packets are split or assembled into a message. It means both MINA-based transport and Netty-based transport should know where the length field of a message is located. It's a little bit of code duplication, but I don't think this will change once implemented because their change means the protocol incompatibility.
read and write methods for SimpleString could be provided as static utility methods or member methods of SimpleString.
WDYT? -
2. Re: Integrating Netty into JBM2
ataylor Jul 1, 2008 8:26 AM (in response to trustin)Out of the three i would probably go for 1. 2 is a no go because of the copying involved and 3 also because of the reasons you've mentioned. Maybe you could just have empty flip(), rewind methods etc, i dont know, but it would be good if we could run with both Mina and Netty.
Also, when we implement message chunking auto grow may not be a problem since all messages will be a fixed size, so I would bear that in mind. -
3. Re: Integrating Netty into JBM2
clebert.suconic Jul 1, 2008 8:36 AM (in response to trustin)It would be nice if we could avoid increasing the size of the Buffer.
When we get to zero-copy on the Journal, we will need to use DirectBuffers, and for AIO they need to be sized as multiples of 512. If you do any logic of recreating the buffers, it would make harder to send them to the disk IO without copying it. -
4. Re: Integrating Netty into JBM2
ataylor Jul 2, 2008 4:26 AM (in response to trustin)Trustin,
I think we need to keep the Messaging Buffer for now. For 2 reasons, 1. we may be implementing our own wire transport so will need some sort of abstraction and 2, we'll probably be also need it for messaging chunking when we implement it. -
5. Re: Integrating Netty into JBM2
trustin Jul 2, 2008 7:30 AM (in response to trustin)I think the reason #1 doesn't make much difference because we still have an interface which is independent from any transport implementation.
I don't have much idea how message chunking will be implemented, so I'd like to listen to more idea on it.
At least, I'd like to remove limit and position from MessagingBuffer, and it will only simplify the implementation IMO. -
6. Re: Integrating Netty into JBM2
ataylor Jul 2, 2008 8:51 AM (in response to trustin)At least, I'd like to remove limit and position from MessagingBuffer, and it will only simplify the implementation IMO.
Ok thats fine, but if you can make sure the Mina impl still works that would be good.
We haven't really looked at message chunking yet, its something we are talking about at this meeting. We'll talk once we have a better idea of the implications. -
7. Re: Integrating Netty into JBM2
trustin Jul 2, 2008 8:53 AM (in response to trustin)Thanks. Let me continue implementation / refactoring. :)