Performance saturation regarding jboss-ejb-client
mmois Apr 6, 2013 11:41 AMHello,
at the moment we are developing an ejb client that has to upload hundreds of documents as fast as possible to our ejb application running inside JBoss AS 7.1.1.Final. I have written a client that uses a configurable number of threads to push the documents via ejb remote call to a statefull session bean. The ejb client calls are implemented as described here using jboss-ejb-client: https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+client+using+JNDI
While measuring the performance, I am wondering that there seems to be a specific boundary on upload speed, while the CPU on the client machine as well as the CPU on the JBoss machine don't go to their limits. The network speed is about 10MB/s, and therefore far away from the practical limit of a gigabit connection. Also the write speed of the database is far from its limits, due to the small network throughput. Increasing the number of threads from one step by step only increases the throughput for the first additional threads. After that, adding more threads doesn't help. In the meantime the CPU of the JBoss server nevers goes beyond 20% load. The profiler shows that all worker threads for the stateful ejbs idle most of the time. The same is true for my client application.
On my search for parameters that help me tuning the upload speed, I found this document about the configuration parameters of jboss-ejb-client: https://docs.jboss.org/author/display/EJBCLIENT/Overview+of+Client+properties. I stubmled upon the sentence "
There is only one connection from the client (JVM) to the server to handle all invocations.". But as the document mentioned above about EJB invocations already states, there is no further documentation about the jboss-ejb-client project at the moment. Looking into the sources of the jboss-ejb-client project, I saw that they use only one jboss-remoting3 channel for each configuration connection. I already tried to add the same connection twice, but this didn't increase the speed. I also increased the parameter MAX_OUTBOUND_MESSAGES, with no effect too.
In order to be sure that we have no other effect that lets the performance run into saturation, I setup a simple test project to examine the problem under laboratory conditions, i.e. it uploads a random byte array to a statefull session bean. The byte array isn't stored, so the speed of the harddrive doesn't have an impcat here. You can find the sources on github: https://github.com/mmois/jboss-ejb-client-performance. But the project shows the same effect.
So my question is, are there any specific configuration options that help to speed up the document upload (e.g. options to configure the number of TCP/IP connections per channel)?
On the other hand, what is the recommended way, when you want to do remote ejb calls that have to upload a lot of data? Maybe the jboss-ejb-client approach isn't the right one?
Thanks in advance,
Martin