There are certain rules about making public performance figures for ECperf tests. TheServerSide website covers the rules.
When a Test Sponsor decides to publish a ECperf benchmark result, he must generate the Full Dis-closure as described in Clause 7 and submit it to the ECperf Review Committee by emailing it to firstname.lastname@example.org. The ECperf Review Committee meets every Monday at 11 AM PST.
During the review period, the members of the Review Committee will hold all review material in strict confidentiality. Within a member company this information should be shared with only those people necessary to provide a thorough review of the result.
At the end of the review, if there are no open issues related to a given submission, the result is approved and the Full Disclosure is posted on the ECperf results website. Once posted, the infor-mation contained in the disclosure is considered public.
There is also the successor SpecJAppserv2002 http://www.specbench.org/jAppServer2002/ which costs money to obtain the test code but tests EJB 2.0 rather than 1.1. Again publication rules apply for results.
For all these reasons, unless someone is willing to sponsor configuration and tuning and formal submission, results cannot be provided and we continue to abide by the rules set.
Much of these published performance results are based on hardware clusters, extreme tuning for a specific database, network configuration and software tuning. Therefore, this turns out to be a rather subjective assessment. You could listen to Microsoft who claims that JBoss is the fastest J2EE app server for Web services in the Doculabs report, but also shows Microsoft clearly outperforms it. But then Microsoft use the Giga report to show that J2EE is more expensive, contested http://www.smh.com.au/articles/2003/09/19/1063625187016.html.
Choosing an App Server is as much a commodity choice as choosing the OS. If you build your J2EE application without using vendor specific enhancements, you can choose the app server for your client's needs on a case-by-case basis. Some are going to want brand name, some are going to want assurance, some are going to want value for money. JBoss has a place in all of this.
For performance and development, MHO is JBoss is fast and simple - things that tend to go hand-in-hand. The developer and the deployment engineer are pretty close to the JBoss engine so you get a good feel for how things are interacting, you can work out how to tweak things and how to bolt other things on to it.
If you want tightly integrated IDEs, be shielded from the mechanics of building J2EE applications (and hence be removed from when a design or binding is inflexible or load-sensitive), then get a vendor distribution. You'll need to pay later to get some expert advice on tuning your completed application. And you'll probably get lock-in to vendor specific features without knowing it, judging from forum posts in the past. This isn't necessarily bad as long as you are aware of the ramifications.
The license to ECPerf only allows Sun certified app servers to be benchmarked and Sun has shown it is in no hurry to let us certify the server.