7 Replies Latest reply on Jul 2, 2008 8:53 AM by Trustin Lee

    Integrating Netty into JBM2

    Trustin Lee Apprentice

      Hi,

      I've been trying to integrate Netty into JBM2 since the end of last week. I enountered a problem while implementing MessagingBuffer. According to the source code, a MessagingBuffer implementation is supposed to increase its capacity automatically when it runs out of space. It's what exactly MINA IoBuffer does. Netty has a CompositeByteArray which does similar thing, but it's different from IoBuffer in that it doesn't have position and limit, which means there's no need for flip or rewind. Also, automatic expansion of the buffer is implemented very differently - it's not reallocation and copy but adding a new chunk to the end of the list.

      There are a few ways to resolve this differences (in the order of better performance):

      1) Replace MINA stuff with Netty and remove position(), limit(), flip(), rewind() and etc from MessagingBuffer.
      2) Emulate the current behavior with NIO BuyeBuffer classes.
      3) Emulate the current behavior with Netty classes

      Of course, the option 1 involves the biggest change among the three and will require the changes in unit tests, while it would be fun for me.

      Another alternative is the option 2 - most part of the code is done in NIO ByteBuffer so the implementation shouldn't be that difficult, but I still have to deal with auto-expansion, which leads to unnecessary memory copy.

      Option 3 is the worst of all because we are going to build something on top of something built on top of existing buffer (or byte array).

      Which option shall I go with? Whichever option we choose, we should make all the existing tests pass.

        • 1. Re: Integrating Netty into JBM2
          Trustin Lee Apprentice

          Even further, I think we can reuse DataInput and DataOutput interface instead of MessagingBuffer. The reasoning behind this is that putInt() is the only random access method in MessagingBuffer and it's only used for writing the length field during encoding. Except that, every access is sequential.

          What the current decoder implementation does is to read the length field first and wait until the whole message is received, which means the whole message is read into memory. Therefore, using a stream to read a message will never block once a message is properly split (or assembled) in the networking layer (MINA or Netty level).

          Writing a message is a different matter because we need to fill the length field. We could fill the length field in the networking layer (MINA or Netty level) to work around this issue.

          The disadvantage of this approach is that each networking layer implementation should know how one or more packets are split or assembled into a message. It means both MINA-based transport and Netty-based transport should know where the length field of a message is located. It's a little bit of code duplication, but I don't think this will change once implemented because their change means the protocol incompatibility.

          read and write methods for SimpleString could be provided as static utility methods or member methods of SimpleString.

          WDYT?

          • 2. Re: Integrating Netty into JBM2
            Andy Taylor Master

            Out of the three i would probably go for 1. 2 is a no go because of the copying involved and 3 also because of the reasons you've mentioned. Maybe you could just have empty flip(), rewind methods etc, i dont know, but it would be good if we could run with both Mina and Netty.

            Also, when we implement message chunking auto grow may not be a problem since all messages will be a fixed size, so I would bear that in mind.

            • 3. Re: Integrating Netty into JBM2
              Clebert Suconic Master

              It would be nice if we could avoid increasing the size of the Buffer.

              When we get to zero-copy on the Journal, we will need to use DirectBuffers, and for AIO they need to be sized as multiples of 512. If you do any logic of recreating the buffers, it would make harder to send them to the disk IO without copying it.

              • 4. Re: Integrating Netty into JBM2
                Andy Taylor Master

                Trustin,

                I think we need to keep the Messaging Buffer for now. For 2 reasons, 1. we may be implementing our own wire transport so will need some sort of abstraction and 2, we'll probably be also need it for messaging chunking when we implement it.

                • 5. Re: Integrating Netty into JBM2
                  Trustin Lee Apprentice

                  I think the reason #1 doesn't make much difference because we still have an interface which is independent from any transport implementation.

                  I don't have much idea how message chunking will be implemented, so I'd like to listen to more idea on it.

                  At least, I'd like to remove limit and position from MessagingBuffer, and it will only simplify the implementation IMO.

                  • 6. Re: Integrating Netty into JBM2
                    Andy Taylor Master

                     

                    At least, I'd like to remove limit and position from MessagingBuffer, and it will only simplify the implementation IMO.


                    Ok thats fine, but if you can make sure the Mina impl still works that would be good.

                    We haven't really looked at message chunking yet, its something we are talking about at this meeting. We'll talk once we have a better idea of the implications.

                    • 7. Re: Integrating Netty into JBM2
                      Trustin Lee Apprentice

                      Thanks. Let me continue implementation / refactoring. :)