12 Replies Latest reply on Aug 21, 2008 11:02 AM by timfox

    Added Netty transport to the trunk

    trustin

      Hi folks,

      I've just added the Netty transport for JBoss Messaging to the trunk. I copied the unit tests of MINA and all tests seem to pass although I had to make some modifications:

      * A test method for MessagingBuffer.limit() was modified slightly because Netty buffer's initial capacity is always 0 however big buffer you requested.

      * Changed the type of connection ID from Long to Object because Netty uses UUID instead of long. This shouldn't affect MINA. I'm not sure if Netty will have some disadvantage from the performance perspective because of this.

      I didn't touch the integration tests and stress tests. Can someone run it for me and let me know how it goes? Otherwise, I could run the integration tests on my machine with your help.

        • 1. Re: Added Netty transport to the trunk
          timfox

          Thanks Trustin - I'll take a look soon.

          • 2. Re: Added Netty transport to the trunk
            trustin

            I'm running the integration tests by myself now, and found some bug. Let me ping here again once the integration tests pass.

            • 3. Re: Added Netty transport to the trunk
              trustin

              Most tests pass with Netty now. Jeff is looking into the remaining issues. Please let me know if you find something. I also would like to know how Netty transport performs.

              You might also take a look at ChannelBufferWrapper which might need some tuning. I found MessagingBuffer still has position, limit, flip, so I had to emulate its behavior which might hurt the performance slightly.

              • 4. Re: Added Netty transport to the trunk
                trustin

                I've been tracking down a weird performance issue in Netty related with JBoss Messaging. Now I can say Netty transport will perform very well with upcoming Netty 3.0.0.CR2. Let me finish this and get back to fixing any failing tests. If you already have a list of failing tests related with the current Netty transport, please let me know. Such a list would be very helpful.

                • 5. Re: Added Netty transport to the trunk
                  clebert.suconic


                  I have once played with changing how the buffers are created on JBM 2, and I'm sure we don't need auto-expanding buffers.

                  When we send a message, we could verify the getEncodeSize(), and pre-allocate the buffers at the exact size necessary to send them. It shouldn't be too difficult of doing that.

                  So, if you (Trustin) are having issues with the auto-expanding buffer, I'm pretty sure we can get rid of that. (At least on sending.. I'm not sure how the buffers are created on reading).

                  • 6. Re: Added Netty transport to the trunk
                    timfox

                     

                    "clebert.suconic@jboss.com" wrote:

                    I have once played with changing how the buffers are created on JBM 2, and I'm sure we don't need auto-expanding buffers.

                    When we send a message, we could verify the getEncodeSize(), and pre-allocate the buffers at the exact size necessary to send them. It shouldn't be too difficult of doing that.

                    So, if you (Trustin) are having issues with the auto-expanding buffer, I'm pretty sure we can get rid of that. (At least on sending.. I'm not sure how the buffers are created on reading).


                    Yes indeed. Especially with Andy's latest changes there should be no need for auto expanding buffers.

                    Also we should be able to get rid of the cumulative protocol decoder stuff since we can just maintain a list of buffers, instead of munging them into one.

                    • 7. Re: Added Netty transport to the trunk
                      trustin

                      I've just released Netty 3.0.0.CR2 which fixes all bugs found while integrating Netty into JBM. Now the trunk passes all integration tests with NettyAcceptorFactory and NettyConnectorFactory. Are we ready to switch? :-)

                      • 8. Re: Added Netty transport to the trunk
                        timfox

                         

                        "trustin" wrote:
                        I've just released Netty 3.0.0.CR2 which fixes all bugs found while integrating Netty into JBM. Now the trunk passes all integration tests with NettyAcceptorFactory and NettyConnectorFactory. Are we ready to switch? :-)


                        If the all-tests target, using netty, consistently passes, then we can switch over.

                        Are we using Netty for SSL too?

                        • 9. Re: Added Netty transport to the trunk
                          trustin

                          SSL tests are hard-coded to use MinaAcceptor. I modified them temporarily to use NettyAcceptor, and all SSL tests passed.

                          I also would like to run all tests with AIOJ enabled. Is there a documented guide to install AIOJ?

                          • 10. Re: Added Netty transport to the trunk
                            timfox

                             

                            "trustin" wrote:
                            SSL tests are hard-coded to use MinaAcceptor. I modified them temporarily to use NettyAcceptor, and all SSL tests passed.

                            I also would like to run all tests with AIOJ enabled. Is there a documented guide to install AIOJ?


                            http://www.jboss.org/file-access/default/members/jbossmessaging/freezone/docs/userguide-2.0.0.alpha1/html/journal.html

                            • 11. Re: Added Netty transport to the trunk
                              trustin

                              Now all tests passes consistently. Thanks for all the help!

                              • 12. Re: Added Netty transport to the trunk
                                timfox

                                I have switched over trunk to use Netty for TCP now. Thanks Trustin for integrating that.

                                I have also implemented an invm transport which we can use for end to end testing.