4 Replies Latest reply on Oct 22, 2005 12:24 AM by tom.elrod

    remoting vs raw transport performance

    tom.elrod Oct 21, 2005 12:57 PM

      There was a jira issue logged (JBREM-195) that pointed out the performance difference between raw rmi calls and rmi calls using remoting. Although the performance is better now, it spurred me to write some tests to compare the difference (so far all my performance test have been comparing difference of transports *within* remoting). Can find the tests at http://anoncvs.forge.jboss.com:8080/changelog/JBoss/JBossRemoting?cs=MAIN:telrod:20051021163552 (the readme.txt explains them in more detail).

      The thing is that the results do not make sense to me:

      1st run

      JBoss Remoting (rmi & java serialization) - 74116 milliseconds
      JBoss Remoting (socket & java serialization) - 74888 milliseconds
      JBoss Remoting (socket & jboss serialization) - 70041 milliseconds
      RMI - 79064 milliseconds
      Socket - 82869 milliseconds

      2nd run

      JBoss Remoting (rmi & java serialization) - 77141 milliseconds
      JBoss Remoting (socket & java serialization) - 75078 milliseconds
      JBoss Remoting (socket & jboss serialization) - 70772 milliseconds
      RMI - 78563 milliseconds
      Socket - 84491 milliseconds

      Maybe I am too close to the code to write objective tests like this. Wondering if anyone would be willing to take a look and see where I might have missed something.

      Thanks.

      -Tom

      • 13177 Views
      • Tags:
        • 1. Re: remoting vs raw transport performance
          adrian.brock Oct 21, 2005 1:18 PM (in response to tom.elrod)

          The test is too simple and lacking in diagnostics.
          Where was the bottleneck (90% in the OS network layer?)
          Is ~1 minute enough time to make "random" full GCs insignificant,
          has the JIT even kicked in, etc.?

          Making performance decisions on these kind of simple tests (which are subject
          to arbitrary implementation changes across versions) is stupid.

          It is also worthless without looking at concurrency.

          I know from the JBossMQ tests that UIL2 is significantly faster than RMI, can't remember
          the numbers off the top of my head? But that is from running unit tests (real world
          jms usage), not simple benchmarks.

            • Actions
          • 2. Re: remoting vs raw transport performance
            tom.elrod Oct 21, 2005 2:33 PM (in response to tom.elrod)

            OMG, who invided Adrian? ;)

            As usual, you are correct. However, I don't want to take the time to write the raw transport counterpart tests for my remoting performance tests until I am sure I can write ones that are correctly represented. So I wrote these simple ones to verify my approach was valid, as would be a lot more wasted time if do the full performance testsuite for raw rmi and socket and they are flawed.

              • Actions
            • 3. Re: remoting vs raw transport performance
              starksm64
              starksm64 Oct 21, 2005 4:01 PM (in response to tom.elrod)

              So run for an hour on a loopback interface to make a better first pass on diffs, or was this run over a local loopback.

                • Actions
              • 4. Re: remoting vs raw transport performance
                tom.elrod Oct 22, 2005 12:24 AM (in response to tom.elrod)

                Will just write the full fledge raw transport tests the duplicate the remoting performance tests (so can capture in nightly testsuite run via cruisecontrol and have jrunit benchmark reporting). Already did the rmi one.

                Plan on running different scenarios (number of calls, payload size, etc.) on a reserved qa box to get some real numbers next week.

                  • Actions