My application uses a remoting proxy for remoting all server-side business objects. Thus, it is very easy for us to swap out which remoting technology we use. In the past we have been EJB-based (all remote invocations were done via a Stateless Session EJB), but have recently switched to RMI for performance reasons. The difference in performance between RMI and EJB invocations is astounding, and I feel sure that there is something we could have done to make the EJB calls more performant.
I have been using a profiler to run numerous tests and observe how long remote invocations take. On a particular remote method call:
Using RMI, each remote method call took roughly 1.5-2ms. If I invoked the method 200 times, it would take roughly 300-350ms.
Our framework cached the EJB Home interface, but for each method invocation, it 1) invoked create on the EJB Home interface 2) invoked the method on the EJB object 3) called remove on the EJB. Is this correct or should we also cache the EJB object? If we cache the EJB object, is there any effect on instance pooling, clustering, etc? (we are not currently using clustering, but may in the future)
Calling the above 3 methods looks like it results in 3 separate remote calls to the application server. Obviously if we cached the EJB object, it would improve performance tremendously.
For the three remote invocations:
create() - The create method took an average of 10 ms per invocation (6ms with PooledInvoker). The majority of this time was consistently spent in ClientContainer.readExternal (An average of about 6ms per create was spent in readExternal) Why is this?
method invocation - The method invocation itself took an average of 4 ms (3ms with PooledInvoker). This is still almost 3 times the amount of time a plain RMI call took, but may be normal considering the added complexity of an EJB invocation.
remove() - The remove method took an average of around 3 ms per call (1.5ms with PooledInvoker). This is still double what the entire invocation took using RMI.
Since we primarily only used EJB as a remoting infrastructure, we have now switched to plain RMI. However, I would like to understand if we were doing anything wrong with EJBs or if there is additional tuning we could have done to improve EJB performance.
Summary of my questions:
- Should we call create and remove on each invocation, or should we cache the stateless session EJB object in our remoting infrastructure? What effect does this have on server-side instance pooling, clustering, etc.?
- Why does the create() method take so long? Is it normal for readExternal() to take up so much time?
- Is there additional tuning we can do to speed up EJB performance?
- Another problem we saw was that every now and then a create() call would take 2+ seconds. From the profiler, it appeared that this was caused by the MarshalledInputStream.resolveProxyClass() method called when the MarshalledObject.get() method is called by the ClientContainer.readExternal() method. Has anyone seen this problem or does anyone have any ideas why this could happen?
All testing was done on a local LAN - obviously the performance would be different over a slower network.
JRMPInvoker was used, except where it was noted that PooledInvoker.