Don't you want to contribute your tests?
That would help IMO.
Also, you shouldn't take in consideratio any warm up times. Start measuring after some time after.
On the beggining of these tests usually JIT kicks on, pooles are being filled, and that kind of thing.
Also, make sure your test is fair to EJB2 and EJB3. they both have to be under the same caching circunstances. (usually for these scenarios it's better to not use caching at al).
Thanks for the response.
I can't publish the tests, they are very app. specific and probably very top secret ;-)
However, they do cover a wide range of database activity and for us represent a 'relative' benchmark for comparing implementations.
I understand your point about start up etc. To counter this, I ran the tests 10 times from cold for each config., and took the average.
The caching strategy is an issue, and I will investigate this when I get back from hols.We set it specifically in the EJB2, I need to investigate the EJB3 situation.
Again many thanks for your input.
do not test perf with HSQLDB if your target platform is not
This is a entity heavy app I assume? I did some tests on Session beans and remote invocations are faster with EJB3 and local ejb3 interfaces a little bit slower.
You probably do not have your fetch strategies tuned appropriately(FETCH JOINs and EAGER/LAZY loading), nor are you doing any caching.
I've done some tests for ejb3 peristence (in the context of my application).
In most of the case results was good except in some specific cases. I had some problems when a lot of objects where retrieve in the same transaction and when during the transaction I perform queries (due to the auto-flush).
I had another problem: http://jira.jboss.com/jira/browse/EJBTHREE-326
Bug has not been closed yet. I have not tried to see it is is always true with more recent release.