3 Replies Latest reply on Jun 29, 2011 5:20 PM by clebert.suconic

    Netty performance with network RTT > 1 second

    bnc119

      Hi, I am running HornetQ 2.1.2 Final on a RHEL 5 machine.  We recently started running an interesting network latency test where we use the unix traffic control "tc" utility to increase network delay on the eth0 interface card HornetQ uses to communicate with subscribers.  The command we use is:

       

      /sbin/tc/ qdisc add dev eth0 root netem delay 1000ms 50ms distribution normal

       

      The short of this is that "pinging" TO or FROM the aforementioned inteface gives us a round trip time (RTT) of approximately 1 second.  (which is bloody forever in the world of networking!)  Under these conditions, the performance of HornetQ (or Netty?) appears to really degrade as compared to the performance of simply using the "netcat" utility to read/write "Hello World!" through TCP sockets.  That is, both tests obviously execute slower, but the netcat test slows down linearly while our JMSPublisher test seems to slow down exponentially.

       

      1.  We are running HornetQ in the standalone/non-clustered configuration with "Netty" defined as our connector and acceptor.  I've read through the HornetQ user manual (Section 16.4.1) on configuring the Netty TCP transport, but I can't seem to find the parameters discussed therein:

       

      • use-nio
      • tcp-no-delay
      • tcp-send-buffer-size

       

      Which configuration file can I tune these parameters in?

       

      2.  How can I configure Netty to use a different protocol such as HTTP or GoogleProtoBuf?

       

      Thanks for answering these high-level questions.  Any suggestions on where you think the extra network bandwidth is being devoted would help my study.

        • 1. Re: Netty performance with network RTT > 1 second
          clebert.suconic

          It seems you need to disable naggle:

           

          http://docs.jboss.org/hornetq/2.2.5.Final/user-manual/en/html_single/index.html#d0e3306

           

          i.e. change tcp-no-delay parameter accordingly to the doc.

           

           

          The doc will also provide you information about other protocols.

          • 2. Re: Netty performance with network RTT > 1 second
            bnc119

            Thanks for the response.  I'm looking at the hornetq-configuration.xml file and have a few follow-up questions:

             

            1.  By default, hornetq came configured with two acceptors defined:  netty, and netty-throughout:

             

            <connectors>

                   <connector name="netty">

                     <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>

                     <param key="host"  value="${jboss.bind.address:localhost}"/>

                     <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>

                  </connector>

                 

                  <connector name="netty-throughput">

                     <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>

                     <param key="host"  value="${jboss.bind.address:localhost}"/>

                     <param key="port"  value="${hornetq.remoting.netty.batch.port:5455}"/>

                     <param key="batch-delay" value="50"/>        

                  </connector>

            </connectors>

             

            Can you tell me what the difference is between these two acceptors?  They are configured the same way, except netty-throughput defines a batch-delay variable.

             

            My other question is:  There are two acceptors defined in the config file.  (And they are also referenced in the hornetq-jms.xml file) How does hornetQ decide which of the two acceptors to use at runtime?

            • 3. Re: Netty performance with network RTT > 1 second
              clebert.suconic

              the throughput will batch multiple writes the same way naggling (or tcp-no-delay) does. We just have an algorithm at our protocol level.

               

              Look closely and you will see that each acceptor is on a different port (5455 and 5445) I bet you did't look closely??