6 Replies Latest reply on Aug 10, 2012 9:01 AM by ion_mayank

    High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1

    ion_mayank

      I am facing a low latency issue while invoking a remote EJB using JBOSS remoting.

      On a client I printed the start and end time and similarly I printed the same on remote EJBs start and end of the method.

       

      I am consistently getting on average 2 to 4 millisec delay in a send call and 7 to 10 millisec dealy in receiving the reponse on client. This is total results in almost 10 millisec dealy which is not acceptable for my application.

       

      I am porting my existing application from JBOSS6.0.0 to JBOSS7.1.1. On JBOSS6.0.0 this delay is not more then 2 millisec.

       

      Attached are the stats of:

       

      JBOSS 7.1.1

       

      Client:                                                                                                                                                                    

      Started at Thu Aug 09 11:15:23 IST 2012

      Ended At Thu Aug 09 11:15:23 IST 2012

      Duration  in milliseconds: 78

      Sent: 09.08.2012 11:15:23,312 Received: 09.08.2012 11:15:23,390

       

      Server:

      executeInstructionExecution start time is: 1344491123315

      executeInstructionExecution end time is: 1344491123381

      Time for one message in millis 66

      Sent: 09.08.2012 11:15:23,315 Received: 09.08.2012 11:15:23,381

       

      Total delay of 12 sec, 3 in send call and 9 in receiving

       

       

      JBOSS 6.0.0

      Client:                                                                                                                                                                    

      Test Started at Thu Aug 09 11:18:41 IST 2012

      Test Ended At Thu Aug 09 11:18:41 IST 2012

      Duration in milliseconds: 21

      Sent: 09.08.2012 11:18:41,309 Received: 09.08.2012 11:18:41,330

       

      Server:

      executeInstructionExecution start time is: 1344491321310

      executeInstructionExecution end time is: 1344491321328

      Time for one message in millis 18

      Sent: 09.08.2012 11:18:41,310 Received: 09.08.2012 11:18:41,328

       

      Total delay of 3 sec, 1 in send call and 2 in receiving

       

      What configuration I need to do to get it down to atmost 3millisec?

        • 1. Re: High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1
          jaikiran

          What does the client code look like? Also you seem to have a typo in your posts. I believe you meant millisecond and not second in the timings report. Anyway, this usually isn't the right way to test performance. How many invocations are you doing? And how is the difference by a few milli seconds affecting the application? What kind of an application are you using?

          • 2. Re: High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1
            ion_mayank

            Yes I meant millisecond, sorry for typo. Edit option was not coming for my 1 post.

             

            I have posted the result of only single call. In production there can 1000s of call, where average delay of 9ms/request can result in 9 second delay for 1000 calls which is huge for critical applications.

             

            This can also be checked on JProfiler. I loaded my server with 1000 synchronous calls. My Profiler (which is attached to JBOSS7.1.1) is only capturing 65% of the time.

            For example if I run the utility for 1000 calls which is taking 100 sec to get the response, profiler is showing only 65 seconds which I presume is lost in remote network calls.

             

            To make sure JProfiler results are correct, same application on JBOSS600 with JProfiler shows 99% accurate time as taken by the utility.

             

            If required I can post the results of even 1000 calls.

             

            For performance test we load the server with 1000 synchronous calls.

            On JBOSS6 I am getting result which gives average 19ms/request and on JBOSS7.1.1 it is giving 29 ms/request. This is almost 50% more then JBOSS6.

             

            Test case is simple:

            I am doing a RemoteHome lookup then calls a create method to get the remote interface.

            Next I am calling the a remote business method in a loop 1000 times in loop.

             

            This is a real time finance application.

             

            Thanks,

            Mayank

            • 3. Re: High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1
              jaikiran

              We'll need to see your code to know more about this. Please attach a sample application to investigate. Without that, this is just going to be guess work which isn't useful. Also, please try against the latest nightly build to see if there's a difference there.

              • 4. Re: High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1
                ion_mayank

                We are invoking a remote EJB and passing a XML String with about 1500 characters, in response we get a serializable response object which contains the results for all the business instructions send to server for processing.

                I have created a simple sample application which calls a remote EJB method (without any parameter). Performance on JBOSS6 and JBOSS7 was comparable with these simple invocations.

                 

                Could it be that serialization and deserialization process is making the process slow?

                • 5. Re: High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1
                  jaikiran

                  Like I said, it can't be guessed and since you are worried about milli second difference in the response times, it can even be something like logging which can impact the numbers. So without a proper performance testcase with relevant configurations, it really isn't going to be worth trying to guess what's wrong.

                  • 6. Re: High latency in JBOSS remoting framework for invoking remote EJBs in JBOSS7.1.1
                    ion_mayank

                    ok. I shall try to give you the sample application with similar processing for testing.