Benchmark JBoss performance problem (compared with Orion, Re

[dancri77]

Hi,

Recently I run a benchmark to evaluate JSP performance compared with ASP on several application servers.

I started with an ASP application (around 12 dynamic pages and some static html, js, gif resources). This application is somehow database intensive, as it only displays db data based on simple search criteria.

Server configuration:
Database server: MS SQL 2000
Db connectivity: OLE DB for ASP, and Microsoft JDBC Driver SQL 2000 for JSP
OS: Windows 2000 Server
Java: JDK 1.4.1 and J2EE 1.3.1

To stress test the appliation I used Microsoft Web Application Stress Tool (http://webtool.rte.microsoft.com/).

After stressing the ASP app I converted it to JSP. I tried to keep the same functionality and not to optimize anything. It took me around 24 hours :(.

I first deployed the application on JBoss 3.0.2 + Jetty. Then I deployed the app on Orion 1.5.2 and on Resin-EE 2.0.

These are the benchmark results. I used two machines, one for stress clients and one for web app and sql.

1. MS ASP + IIS5.0
---------
Number of stress threads: 10
Number of hits: 4144
Requests per Second: 34.46

Number of stress threads: 30
Number of hits: 4090
Requests per Second: 34.04

2. JBoss
--------
Number of stress threads: 10
Number of hits: 544
Requests per Second: 24.63

Number of stress threads: 30
Number of hits: 3079
Requests per Second: 25.84

3. Orion
--------
Number of stress threads: 10
Number of hits: 4192
Requests per Second: 34.88

Number of stress threads: 30
Number of hits: 4053
Requests per Second: 34.01

4. Resin
--------
Number of stress threads: 10
Number of hits: 4021
Requests per Second: 33.45

Number of stress threads: 30
Number of hits: 4176
Requests per Second: 34.73


As you can see JBoss has the lowest request per seconds served ~25 req/s compared with others 34 req/s.

I see this as a problems, but I hope it's possible to be improved. I'm not sure which is the cause, JBoss or Jetty. Can someone enlight me on this issue? I preffered JBoss over Orion or Resin, but now I'm not sure.

It's interesting to notice that ASP is not faster than JSP on Orion and Resin. So those that are saying Java is slow should think about this.

Hope this helps for those who are interested to switch from ASP to JSP.

Dan

[gregwilkins]

The culprit is almost certainly a combination of JBoss+Jetty+Jasper. Jasper is the jakarta JSP engine and has frequently been criticized for being slow. Jasper2 has been released and JBoss 3.2 should not be using it. Hopefully that will give some significant performance gains.

Also the JBoss 3.0.x series was a major functional upgrade and as such introduced a few performance issues. Many of these have been resolved in 3.2.x and I think some may be
included in 3.0.3

Finally, Jetty has been also been regaining lots of performance after it's functional upgrade to the 4.0 series. 4.1 is now the stable release and is used in
3.0.3 and 3.2. It gives a good step forward in peformance and 4.2.x series is now in development with another step.

So in short, 3.0.[012] are probably not the fastest
JBoss releases out there. Try the 3.2 series if you want
to see better results.

[joelvogt]

I'll second this. Our testing showed similar results which have been improving very steadily with each point release. The latest and greatest is always the way to go