-
1. Re: Seam performance vs plain JSF
svadu Dec 6, 2007 5:27 PM (in response to gonzalad)Do your tests use JBoss microcontainer in all cases?
-
2. Re: Seam performance vs plain JSF
svadu Dec 6, 2007 5:33 PM (in response to gonzalad)There is also somewhat older article from JBoss about Seam performance, might be interesting as reference information: http://www.dell.com/downloads/global/power/jbossworld_2006_june_jaffe.pdf
-
3. Re: Seam performance vs plain JSF
gonzalad Dec 6, 2007 5:33 PM (in response to gonzalad)Never in fact.
Why ? -
4. Re: Seam performance vs plain JSF
svadu Dec 6, 2007 5:49 PM (in response to gonzalad)I wouldn't be surprised if you don't get very high performance using microcontainer because you're not using full JEE capabilities of application server. But since you didn't use MC I can't comment on your numbers (I am not a performance tuning expert).
Was the JSF (without Seam) application a 'full' equivalent of a Seam application?
I don't have hard numbers but in my case I had performance improved (didn't need to measure to see improvement) a lot mostly due to moving lot's of stuff from request (session scope was not acceptable) scope (JSF application) to conversation scope which reduced amount of database trips a lot. -
5. Re: Seam performance vs plain JSF
gonzalad Dec 6, 2007 5:50 PM (in response to gonzalad)"svadu" wrote:
There is also somewhat older article from JBoss about Seam performance, might be interesting as reference information: http://www.dell.com/downloads/global/power/jbossworld_2006_june_jaffe.pdf
Thanks for this review, I've already looked at it, but read it once more (in case of..).
But doesn't help ;((
page 24 short think time : in fact all vusers involved in all tests had no think time.
logging : no log in any my tests (only at startup).
client vs server state : server state always used (session size was 180 ko ouch ! for all tests).
no ejb : so no call by value problem
vusers where between 5 and 15.
no load balancing
no session replication
and no .css or .js or image requested from my vusers ! -
6. Re: Seam performance vs plain JSF
gonzalad Dec 6, 2007 5:56 PM (in response to gonzalad)"svadu" wrote:
I wouldn't be surprised if you don't get very high performance using microcontainer because you're not using full JEE capabilities of application server.
I really don't see how MC would explain performance gain between plain jsf and seam. But I've never used MC. Could you explain MC benefits for my use case ?"svadu" wrote:
Was the JSF (without Seam) application a 'full' equivalent of a Seam application?
Yes, I've just removed Seam from the spring sample which is included in Seam 2.0.0 GA distribution."svadu" wrote:
I don't have hard numbers but in my case I had performance improved (didn't need to measure to see improvement) a lot mostly due to moving lot's of stuff from request (session scope was not acceptable) scope (JSF application) to conversation scope which reduced amount of database trips a lot.
I fully agree that it would increase my performance if I was using session replication in db. But since I'm not using session replication and I don't have session size problems, i don't see the benefits. -
7. Re: Seam performance vs plain JSF
svadu Dec 6, 2007 6:05 PM (in response to gonzalad)"gonzalad" wrote:
"svadu" wrote:
I wouldn't be surprised if you don't get very high performance using microcontainer because you're not using full JEE capabilities of application server.
I really don't see how MC would explain performance gain between plain jsf and seam. But I've never used MC. Could you explain MC benefits for my use case ?
I didn't mean that MC would give you performance benefits :)
I think if you prepare more details (as you mentioned in the original post) it's likely Seam team will be interested in your case... -
8. Re: Seam performance vs plain JSF
gonzalad Dec 6, 2007 6:10 PM (in response to gonzalad)I'll send more info on Monday then.
Have a good week end and thanks for your help ! -
9. Re: Seam performance vs plain JSF
gonzalad Dec 10, 2007 2:03 PM (in response to gonzalad)So Monday....
Hello !
Here is some more information about the load tests we executed.
If you want more information please tell me. Also, if you have an idea on how we can achieve betters results, I'm interested (of course !).
Otherwise I think you can find some interesting comparative and instructive (ar least for me !) performance results.
Please note, I'm really far from being a bench expert or a system expert - I'm only a humble little developper so once more please - if anyone sees some
interesting optimization to apply just tell me !
Also, those tests were made on a simple application and are not representative for application in production environment
(we didnt' have time to code one for each framework :)).
Extract
From our test:
. Seam consumes 3 times (really average) more cpu than the same app with plain jsf.
. JSF consumes 3 times (really average) more than plain old Struts like app.
I would have really expected at the beginning a performance ratio of 2 times more for Seam app that Struts like but it's like 10 !
arrrgghhhhh !
I - Test Environment
1. Injector platform used :
Pentium IV 3 GHz, 2 Go RAM.
Windows XP Professional SP1 with JMeter 2.3.
2. App Server
Websphere 6.1.0.13
IBM JRE 5.0 SR6 64bits
(Java(TM) 2 Runtime Environment, Standard Edition (build pap64devifx-20071025 (SR6b)).
sxqwas02 : 6 Processors 2.1 GHz Power PC5
40894464 Ko
One JVM
No clustering and no session replication.
JVM Parameters
Initial Heap Size = 256 Mo
Max Heap Size = 512 Mo
Web Container Threads =
Session Timeout = 4 minutes
JDBC Pool = {connectionTimeout=60s, maxConnections=10, minConnections=5, reapTime=60, unusedTimeout=120, aged=0}
Threadpool : minSize = 5,maxSize = 20
3. Web Server
IBM HTTP Server 6.1.0.5
CPU : 1 proc 2812.972 MHz
Dual-Core AMD Opteron(tm) Processor 8220 SE
RAM : 1025372 Ko
IHS Parameters
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
StartServers 5
MaxClients 150
MinSpareThreads 5
MaxSpareThreads 10
ThreadsPerChild 25
MaxRequestsPerChild 10000
II - Test scenario
1. Scenario
User uses jpa sample application bundled with Seam.
All vusers execute the following scenario in loop :
1. go to page /jpa/home.seam
2. loggin
user=demo
pass=demo
3. user press the 'Find Hotel' button
(no search criteria and max results=10).
4. user selects first record in the list
5. user books this hotel
6. user enters booking information
7. user confirms his reservation
8. user cancels his reservation.
2. mesurement points
All version of the application were tested with 15tx/s, 30tx/s, 60tx/s, 120tx/s, 240tx/s.
3. Ramp up period/think time, vuser count, etc...
No think time.
No Ramp up period (we used a constant throughput timer in JMeter to limit the number of request to 15, 30, ...tx/s) depending on the scenario.
4. Applications tested
a. jpa sample modified to remove a4j and RichFaces components & libraries.
uses SUN RI 1.2 (bundled with Seam), facelets, Seam 2.0.0.GA
name:jpa-no-a4j.ear
same as jpa from Seam except rich*.jar removed, el-impl.jar added (if it wasn't already there).
b. jpa sample modified to remove Seam, include Spring for business Layer.
uses SUN RI 1.2, facelets, Spring (bundled with Seam 2.0.0.GA)
name:jpa-standard-jsf-facelets.ear
c. jpa sample modified to remove Seam and facelets, include Spring for business Layer.
uses SUN RI 1.2, JSP, Spring (bundled with Seam 2.0.0.GA)
name:jpa-standard-jsf-sun-1.2.ear
d. jpa sample modified to remove Seam and facelets, include Spring for business Layer.
uses IBM JSF (1.1), JSP, Spring (bundled with Seam 2.0.0.GA)
name:jpa-standard-jsf.ear
e. another demo application which runs with a Struts like framework.
name:demo.ear
5. Notes
Please, note that when we removed Seam, we put hotel list in request scope, not in session scope.
Note also log level was at WARN (so no logs after startup during test execution).
III Results
Result data was retrieved from :
a. JMeter Summary Report (tx/s, page size).
b. topas (cpu usage).
c. Wily Introscope (hea size, session size)
1. Test with 15tx/sScenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo) jpa-no-a4j.ear 15 10 87 5,8 181,4 244,3 jpa-standard-jsf-facelets.ear 15 4 49 6,2 81,1 NA jpa-standard-jsf.ear 15 4 47 6,2 2,0 220 jpa-standard-jsf-sun-1.2.ear 15 4,5 42 5,8 198,2 232,8 demo.ear 15 1,1 13 17 1,9 250
2. Test with 30tx/s
Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
jpa-no-a4j.ear 29 20 79 6,2 184 249,6
jpa-standard-jsf-facelets.ear 30 7 43 6,0 NA NA
jpa-standard-jsf.ear 30 7 52 6,2 2,4 245
jpa-standard-jsf-sun-1.2.ear NA NA NA NA NA NA
demo.ear 30 2 13 17 1,9 253
3. Test with 60tx/s
Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
jpa-no-a4j.ear 51,8 41 77 6,5 182,1 248,8
jpa-standard-jsf-facelets.ear 60 11 45 6,2 2,3 255
jpa-standard-jsf.ear 60 16 43 6,2 2,5 251
jpa-standard-jsf-sun-1.2.ear NA NA NA NA NA NA
demo.ear 62 3,8 13 17 1,9 255
4. Test with 120tx/s
Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
jpa-no-a4j.ear 86 65 102 8,6 182 304
jpa-standard-jsf-facelets.ear 113 22 41 6,8 2,8 262
jpa-standard-jsf.ear 115 30 41 6,2 2,1 248
jpa-standard-jsf-sun-1.2.ear 112 20 50 6,1 201 261
demo.ear 123 7,2 15 17 1,9 246
5. Test with 240tx/s
Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
jpa-no-a4j.ear Saturation Saturation Saturation Saturation Saturation Saturation
jpa-standard-jsf-facelets.ear 205 50 53 6,2 3,3 ko 237
jpa-standard-jsf.ear 206 55 53 6,2 3,1 252
jpa-standard-jsf-sun-1.2.ear NA NA NA NA NA NA
demo.ear 240 14,2 14 17 1,9 251
6. Test with 480tx/s
Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
all jpa samples saturated
demo.ear 440 25 17 17 1,9 255
IV - Configuration files for jpa-no-a4j.ear
components.xml<?xml version="1.0" encoding="UTF-8"?> <components xmlns="http://jboss.com/products/seam/components" xmlns:core="http://jboss.com/products/seam/core" xmlns:persistence="http://jboss.com/products/seam/persistence" xmlns:transaction="http://jboss.com/products/seam/transaction" xmlns:security="http://jboss.com/products/seam/security" xmlns:web="http://jboss.com/products/seam/web" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation= "http://jboss.com/products/seam/core http://jboss.com/products/seam/core-2.0.xsd http://jboss.com/products/seam/persistence http://jboss.com/products/seam/persistence-2.0.xsd http://jboss.com/products/seam/security http://jboss.com/products/seam/security-2.0.xsd http://jboss.com/products/seam/web http://jboss.com/products/seam/web-2.0.xsd http://jboss.com/products/seam/components http://jboss.com/products/seam/components-2.0.xsd"> <core:manager conversation-timeout="120000" concurrent-request-timeout="500" conversation-id-parameter="cid"/> <persistence:entity-manager-factory name="bookingDatabase"/> <persistence:managed-persistence-context name="em" auto-create="true" entity-manager-factory="#{bookingDatabase}"/> <security:identity authenticate-method="#{authenticator.authenticate}"/> <!-- WAS requires the filter to be mapped to a suffix, so can't use built in Seam filter --> <web:ajax4jsf-filter disabled="true" /> </components>
web.xml<?xml version="1.0" encoding="UTF-8"?> <web-app id="WebApp_ID" version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"> <!-- Seam --> <listener> <listener-class>org.jboss.seam.servlet.SeamListener</listener-class> </listener> <filter> <filter-name>Seam Filter</filter-name> <filter-class>org.jboss.seam.servlet.SeamFilter</filter-class> </filter> <filter-mapping> <filter-name>Seam Filter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> <servlet> <servlet-name>Seam Resource Servlet</servlet-name> <servlet-class>org.jboss.seam.servlet.SeamResourceServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>Seam Resource Servlet</servlet-name> <url-pattern>/seam/resource/*</url-pattern> </servlet-mapping> <servlet> <servlet-name>Faces Servlet</servlet-name> <servlet-class>javax.faces.webapp.FacesServlet</servlet-class> <load-on-startup>1</load-on-startup> </servlet> <!-- Faces Servlet Mapping --> <servlet-mapping> <servlet-name>Faces Servlet</servlet-name> <url-pattern>*.seam</url-pattern> </servlet-mapping> <context-param> <param-name>facelets.REFRESH_PERIOD</param-name> <param-value>-1</param-value> </context-param> <context-param> <param-name>javax.faces.STATE_SAVING_METHOD</param-name> <param-value>server</param-value> </context-param> <context-param> <param-name>com.sun.faces.preferXHTML</param-name> <param-value>true</param-value> </context-param> <context-param> <param-name>com.sun.faces.responseBufferSize</param-name> <param-value>70000</param-value> </context-param> <context-param> <param-name>facelets.BUFFER_SIZE</param-name> <param-value>70000</param-value> </context-param> <context-param> <param-name>javax.faces.DEFAULT_SUFFIX</param-name> <param-value>.xhtml</param-value> </context-param> <context-param> <param-name>facelets.DEVELOPMENT</param-name> <param-value>false</param-value> </context-param> <listener> <listener-class>com.sun.faces.config.ConfigureListener</listener-class> </listener> <session-config> <session-timeout>10</session-timeout> </session-config> </web-app>
-
10. Re: Seam performance vs plain JSF
jbalunas Dec 10, 2007 2:53 PM (in response to gonzalad)These are surprising results for me as well. I will review them and compare with our own servers.
I will be adding more tests and comparisons in regards to performance to the Seam project as time goes on. I appreciate your work on this.
Thanks,
-Jay
JBoss Seam QE. -
11. Re: Seam performance vs plain JSF
gonzalad Dec 11, 2007 2:54 AM (in response to gonzalad)Thank you very much for your concern Jay, I'm looking forward for any news you might have on this subject - whatever it would be (configuration change, code change or just different results than me).
I'll also be very interested about your performance results between plain jsf and seam if you make such tests. -
12. Re: Seam performance vs plain JSF
lowecg2004 Dec 11, 2007 4:16 AM (in response to gonzalad)The debug mode for Seam would have a significant impact on your stats. Have you set the debug mode for Seam to false?
In components.xml, try adding:
<core:init debug="false" /> -
13. Re: Seam performance vs plain JSF
gonzalad Dec 11, 2007 9:22 AM (in response to gonzalad)Hi lowecg2004,
Thanks for the tip.
Just modified the jpa-no-a4j.ear application adding :<core:init debug="false" />
But same results than before (looking at Seam code, debug="false" is the default value).
So, for a constant throughput of 60tx/s, I have the following results :
cpu used=40%
elapsed(ms)=94ms
for a 5 minute long load-test.
Thanks you once more anyway ! -
14. Re: Seam performance vs plain JSF
svadu Dec 12, 2007 4:59 PM (in response to gonzalad)"jbalunas@redhat.com" wrote:
These are surprising results for me as well. I will review them and compare with our own servers.
I will be adding more tests and comparisons in regards to performance to the Seam project as time goes on. I appreciate your work on this.
Hi Jay,
It would be nice if after sorting this out an official article about Seam performance was posted. The old article listed at the beginning of this thread is... somewhat old.
And there is much need in more fresh and official info on Seam performance for it to be taken more seriously by enterprises.
Thanks again, I am really glad Red Hat/JBoss team has picked this issue that quickly!