1 2 Previous Next 18 Replies Latest reply on Jan 18, 2008 3:37 PM by jbalunas

    Seam performance vs plain JSF

    gonzalad

      Hello,

      We've just finished some performance test on the jpa sample application from seam 2.0.0.GA, and Seam consumes a lot of CPU.

      Has anyone had a chance to optimize Seam ?
      Is the dev team working on this subject ?
      (I'm really sorry - you're doing a really good work as you know, but those bench results are really scary ! ;(

      Some results
      Seam consumes from 2.2 (for 15 tx/s) to 4.3 (for 60tx/s) times more CPU than same application without Seam.

      Seam powered application saturated at 86 tx/s (cpu saturation).
      JSF application at 205tx/s (cpu saturation).

      Applications tested
      I've made various changes to the jpa sample application and tested each change :
      1 - jpa sample.
      2 - jpa sample without richfaces & a4j.
      3 - spring sample (like jpa sample but with spring).
      4 - spring sample with jsp instead of facelets.
      5 - spring sample with jsp instead of facelets and JSF IBM 1.1 impl.

      - Test Platform -
      All test where executed with Websphere 6.1.0.13 and IBM JVM 5.0 SR 6 64 bits.
      OS : AIX
      Server : 6 CPU (2.1 GHz each) Power PC 5.
      Don't remember RAM size.

      I'll be able to give some more results on Monday - week end time now !

      Bye

        • 1. Re: Seam performance vs plain JSF

          Do your tests use JBoss microcontainer in all cases?

          • 2. Re: Seam performance vs plain JSF

            There is also somewhat older article from JBoss about Seam performance, might be interesting as reference information: http://www.dell.com/downloads/global/power/jbossworld_2006_june_jaffe.pdf

            • 3. Re: Seam performance vs plain JSF
              gonzalad

              Never in fact.

              Why ?

              • 4. Re: Seam performance vs plain JSF

                I wouldn't be surprised if you don't get very high performance using microcontainer because you're not using full JEE capabilities of application server. But since you didn't use MC I can't comment on your numbers (I am not a performance tuning expert).

                Was the JSF (without Seam) application a 'full' equivalent of a Seam application?

                I don't have hard numbers but in my case I had performance improved (didn't need to measure to see improvement) a lot mostly due to moving lot's of stuff from request (session scope was not acceptable) scope (JSF application) to conversation scope which reduced amount of database trips a lot.

                • 5. Re: Seam performance vs plain JSF
                  gonzalad

                   

                  "svadu" wrote:
                  There is also somewhat older article from JBoss about Seam performance, might be interesting as reference information: http://www.dell.com/downloads/global/power/jbossworld_2006_june_jaffe.pdf


                  Thanks for this review, I've already looked at it, but read it once more (in case of..).

                  But doesn't help ;((
                  page 24 short think time : in fact all vusers involved in all tests had no think time.
                  logging : no log in any my tests (only at startup).
                  client vs server state : server state always used (session size was 180 ko ouch ! for all tests).
                  no ejb : so no call by value problem
                  vusers where between 5 and 15.
                  no load balancing
                  no session replication
                  and no .css or .js or image requested from my vusers !


                  • 6. Re: Seam performance vs plain JSF
                    gonzalad

                     

                    "svadu" wrote:
                    I wouldn't be surprised if you don't get very high performance using microcontainer because you're not using full JEE capabilities of application server.

                    I really don't see how MC would explain performance gain between plain jsf and seam. But I've never used MC. Could you explain MC benefits for my use case ?

                    "svadu" wrote:
                    Was the JSF (without Seam) application a 'full' equivalent of a Seam application?

                    Yes, I've just removed Seam from the spring sample which is included in Seam 2.0.0 GA distribution.

                    "svadu" wrote:
                    I don't have hard numbers but in my case I had performance improved (didn't need to measure to see improvement) a lot mostly due to moving lot's of stuff from request (session scope was not acceptable) scope (JSF application) to conversation scope which reduced amount of database trips a lot.

                    I fully agree that it would increase my performance if I was using session replication in db. But since I'm not using session replication and I don't have session size problems, i don't see the benefits.


                    • 7. Re: Seam performance vs plain JSF

                       

                      "gonzalad" wrote:
                      "svadu" wrote:
                      I wouldn't be surprised if you don't get very high performance using microcontainer because you're not using full JEE capabilities of application server.

                      I really don't see how MC would explain performance gain between plain jsf and seam. But I've never used MC. Could you explain MC benefits for my use case ?


                      I didn't mean that MC would give you performance benefits :)

                      I think if you prepare more details (as you mentioned in the original post) it's likely Seam team will be interested in your case...

                      • 8. Re: Seam performance vs plain JSF
                        gonzalad

                        I'll send more info on Monday then.
                        Have a good week end and thanks for your help !

                        • 9. Re: Seam performance vs plain JSF
                          gonzalad

                          So Monday....

                          Hello !

                          Here is some more information about the load tests we executed.

                          If you want more information please tell me. Also, if you have an idea on how we can achieve betters results, I'm interested (of course !).
                          Otherwise I think you can find some interesting comparative and instructive (ar least for me !) performance results.
                          Please note, I'm really far from being a bench expert or a system expert - I'm only a humble little developper so once more please - if anyone sees some
                          interesting optimization to apply just tell me !

                          Also, those tests were made on a simple application and are not representative for application in production environment
                          (we didnt' have time to code one for each framework :)).

                          Extract
                          From our test:
                          . Seam consumes 3 times (really average) more cpu than the same app with plain jsf.
                          . JSF consumes 3 times (really average) more than plain old Struts like app.

                          I would have really expected at the beginning a performance ratio of 2 times more for Seam app that Struts like but it's like 10 !
                          arrrgghhhhh !

                          I - Test Environment

                          1. Injector platform used :
                          Pentium IV 3 GHz, 2 Go RAM.
                          Windows XP Professional SP1 with JMeter 2.3.

                          2. App Server
                          Websphere 6.1.0.13
                          IBM JRE 5.0 SR6 64bits
                          (Java(TM) 2 Runtime Environment, Standard Edition (build pap64devifx-20071025 (SR6b)).
                          sxqwas02 : 6 Processors 2.1 GHz Power PC5
                          40894464 Ko
                          One JVM
                          No clustering and no session replication.
                          JVM Parameters
                          Initial Heap Size = 256 Mo
                          Max Heap Size = 512 Mo
                          Web Container Threads =
                          Session Timeout = 4 minutes
                          JDBC Pool = {connectionTimeout=60s, maxConnections=10, minConnections=5, reapTime=60, unusedTimeout=120, aged=0}
                          Threadpool : minSize = 5,maxSize = 20

                          3. Web Server
                          IBM HTTP Server 6.1.0.5
                          CPU : 1 proc 2812.972 MHz
                          Dual-Core AMD Opteron(tm) Processor 8220 SE
                          RAM : 1025372 Ko
                          IHS Parameters
                          Timeout 300
                          KeepAlive On
                          MaxKeepAliveRequests 100
                          KeepAliveTimeout 15
                          StartServers 5
                          MaxClients 150
                          MinSpareThreads 5
                          MaxSpareThreads 10
                          ThreadsPerChild 25
                          MaxRequestsPerChild 10000


                          II - Test scenario

                          1. Scenario
                          User uses jpa sample application bundled with Seam.
                          All vusers execute the following scenario in loop :
                          1. go to page /jpa/home.seam
                          2. loggin
                          user=demo
                          pass=demo
                          3. user press the 'Find Hotel' button
                          (no search criteria and max results=10).
                          4. user selects first record in the list
                          5. user books this hotel
                          6. user enters booking information
                          7. user confirms his reservation
                          8. user cancels his reservation.

                          2. mesurement points
                          All version of the application were tested with 15tx/s, 30tx/s, 60tx/s, 120tx/s, 240tx/s.

                          3. Ramp up period/think time, vuser count, etc...
                          No think time.
                          No Ramp up period (we used a constant throughput timer in JMeter to limit the number of request to 15, 30, ...tx/s) depending on the scenario.

                          4. Applications tested
                          a. jpa sample modified to remove a4j and RichFaces components & libraries.
                          uses SUN RI 1.2 (bundled with Seam), facelets, Seam 2.0.0.GA
                          name:jpa-no-a4j.ear
                          same as jpa from Seam except rich*.jar removed, el-impl.jar added (if it wasn't already there).
                          b. jpa sample modified to remove Seam, include Spring for business Layer.
                          uses SUN RI 1.2, facelets, Spring (bundled with Seam 2.0.0.GA)
                          name:jpa-standard-jsf-facelets.ear
                          c. jpa sample modified to remove Seam and facelets, include Spring for business Layer.
                          uses SUN RI 1.2, JSP, Spring (bundled with Seam 2.0.0.GA)
                          name:jpa-standard-jsf-sun-1.2.ear
                          d. jpa sample modified to remove Seam and facelets, include Spring for business Layer.
                          uses IBM JSF (1.1), JSP, Spring (bundled with Seam 2.0.0.GA)
                          name:jpa-standard-jsf.ear
                          e. another demo application which runs with a Struts like framework.
                          name:demo.ear

                          5. Notes
                          Please, note that when we removed Seam, we put hotel list in request scope, not in session scope.
                          Note also log level was at WARN (so no logs after startup during test execution).


                          III Results

                          Result data was retrieved from :
                          a. JMeter Summary Report (tx/s, page size).
                          b. topas (cpu usage).
                          c. Wily Introscope (hea size, session size)

                          1. Test with 15tx/s

                          Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
                          jpa-no-a4j.ear 15 10 87 5,8 181,4 244,3
                          jpa-standard-jsf-facelets.ear
                           15 4 49 6,2 81,1 NA
                          jpa-standard-jsf.ear
                           15 4 47 6,2 2,0 220
                          jpa-standard-jsf-sun-1.2.ear
                           15 4,5 42 5,8
                           198,2 232,8
                          demo.ear 15 1,1 13 17
                           1,9 250
                          


                          2. Test with 30tx/s
                          Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
                          jpa-no-a4j.ear 29 20 79 6,2 184 249,6
                          jpa-standard-jsf-facelets.ear 30 7 43 6,0 NA NA
                          jpa-standard-jsf.ear 30 7 52 6,2 2,4 245
                          jpa-standard-jsf-sun-1.2.ear NA NA NA NA NA NA
                          demo.ear 30 2 13 17 1,9 253

                          3. Test with 60tx/s
                          Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
                          jpa-no-a4j.ear 51,8 41 77 6,5 182,1 248,8
                          jpa-standard-jsf-facelets.ear 60 11 45 6,2 2,3 255
                          jpa-standard-jsf.ear 60 16 43 6,2 2,5 251
                          jpa-standard-jsf-sun-1.2.ear NA NA NA NA NA NA
                          demo.ear 62 3,8 13 17 1,9 255

                          4. Test with 120tx/s
                          Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
                          jpa-no-a4j.ear 86 65 102 8,6 182 304
                          jpa-standard-jsf-facelets.ear 113 22 41 6,8 2,8 262
                          jpa-standard-jsf.ear 115 30 41 6,2 2,1 248
                          jpa-standard-jsf-sun-1.2.ear 112 20 50 6,1 201 261
                          demo.ear 123 7,2 15 17 1,9 246

                          5. Test with 240tx/s
                          Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
                          jpa-no-a4j.ear Saturation Saturation Saturation Saturation Saturation Saturation
                          jpa-standard-jsf-facelets.ear 205 50 53 6,2 3,3 ko 237
                          jpa-standard-jsf.ear 206 55 53 6,2 3,1 252
                          jpa-standard-jsf-sun-1.2.ear NA NA NA NA NA NA
                          demo.ear 240 14,2 14 17 1,9 251

                          6. Test with 480tx/s
                          Scenario tx/s CPU used(%) el time moy (ms) page size (ko) session size (ko) bytes used in heap (Mo)
                          all jpa samples saturated
                          demo.ear 440 25 17 17 1,9 255


                          IV - Configuration files for jpa-no-a4j.ear

                          components.xml
                          <?xml version="1.0" encoding="UTF-8"?>
                          <components xmlns="http://jboss.com/products/seam/components"
                           xmlns:core="http://jboss.com/products/seam/core"
                           xmlns:persistence="http://jboss.com/products/seam/persistence"
                           xmlns:transaction="http://jboss.com/products/seam/transaction"
                           xmlns:security="http://jboss.com/products/seam/security"
                           xmlns:web="http://jboss.com/products/seam/web"
                           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                           xsi:schemaLocation=
                           "http://jboss.com/products/seam/core http://jboss.com/products/seam/core-2.0.xsd
                           http://jboss.com/products/seam/persistence http://jboss.com/products/seam/persistence-2.0.xsd
                           http://jboss.com/products/seam/security http://jboss.com/products/seam/security-2.0.xsd
                           http://jboss.com/products/seam/web http://jboss.com/products/seam/web-2.0.xsd
                           http://jboss.com/products/seam/components http://jboss.com/products/seam/components-2.0.xsd">
                          
                           <core:manager conversation-timeout="120000"
                           concurrent-request-timeout="500"
                           conversation-id-parameter="cid"/>
                          
                          
                           <persistence:entity-manager-factory name="bookingDatabase"/>
                          
                           <persistence:managed-persistence-context name="em"
                           auto-create="true"
                           entity-manager-factory="#{bookingDatabase}"/>
                          
                           <security:identity authenticate-method="#{authenticator.authenticate}"/>
                          
                           <!-- WAS requires the filter to be mapped to a suffix, so can't use built in Seam filter -->
                           <web:ajax4jsf-filter disabled="true" />
                          
                          </components>


                          web.xml
                          <?xml version="1.0" encoding="UTF-8"?>
                          <web-app id="WebApp_ID" version="2.4"
                          xmlns="http://java.sun.com/xml/ns/j2ee"
                          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                          xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
                          http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
                          
                          
                           <!-- Seam -->
                          
                           <listener>
                           <listener-class>org.jboss.seam.servlet.SeamListener</listener-class>
                           </listener>
                          
                          
                           <filter>
                           <filter-name>Seam Filter</filter-name>
                           <filter-class>org.jboss.seam.servlet.SeamFilter</filter-class>
                           </filter>
                          
                           <filter-mapping>
                           <filter-name>Seam Filter</filter-name>
                           <url-pattern>/*</url-pattern>
                           </filter-mapping>
                          
                           <servlet>
                           <servlet-name>Seam Resource Servlet</servlet-name>
                           <servlet-class>org.jboss.seam.servlet.SeamResourceServlet</servlet-class>
                           </servlet>
                          
                           <servlet-mapping>
                           <servlet-name>Seam Resource Servlet</servlet-name>
                           <url-pattern>/seam/resource/*</url-pattern>
                           </servlet-mapping>
                          
                          
                           <servlet>
                           <servlet-name>Faces Servlet</servlet-name>
                           <servlet-class>javax.faces.webapp.FacesServlet</servlet-class>
                           <load-on-startup>1</load-on-startup>
                           </servlet>
                          
                           <!-- Faces Servlet Mapping -->
                          
                           <servlet-mapping>
                           <servlet-name>Faces Servlet</servlet-name>
                           <url-pattern>*.seam</url-pattern>
                           </servlet-mapping>
                          
                           <context-param>
                           <param-name>facelets.REFRESH_PERIOD</param-name>
                           <param-value>-1</param-value>
                           </context-param>
                          
                           <context-param>
                           <param-name>javax.faces.STATE_SAVING_METHOD</param-name>
                           <param-value>server</param-value>
                           </context-param>
                          
                           <context-param>
                           <param-name>com.sun.faces.preferXHTML</param-name>
                           <param-value>true</param-value>
                           </context-param>
                          
                           <context-param>
                           <param-name>com.sun.faces.responseBufferSize</param-name>
                           <param-value>70000</param-value>
                           </context-param>
                          
                           <context-param>
                           <param-name>facelets.BUFFER_SIZE</param-name>
                           <param-value>70000</param-value>
                           </context-param>
                          
                          
                           <context-param>
                           <param-name>javax.faces.DEFAULT_SUFFIX</param-name>
                           <param-value>.xhtml</param-value>
                           </context-param>
                          
                           <context-param>
                           <param-name>facelets.DEVELOPMENT</param-name>
                           <param-value>false</param-value>
                           </context-param>
                          
                           <listener>
                           <listener-class>com.sun.faces.config.ConfigureListener</listener-class>
                           </listener>
                          
                           <session-config>
                           <session-timeout>10</session-timeout>
                           </session-config>
                          
                          </web-app>
                          


                          • 10. Re: Seam performance vs plain JSF
                            jbalunas

                            These are surprising results for me as well. I will review them and compare with our own servers.

                            I will be adding more tests and comparisons in regards to performance to the Seam project as time goes on. I appreciate your work on this.

                            Thanks,
                            -Jay
                            JBoss Seam QE.

                            • 11. Re: Seam performance vs plain JSF
                              gonzalad

                              Thank you very much for your concern Jay, I'm looking forward for any news you might have on this subject - whatever it would be (configuration change, code change or just different results than me).

                              I'll also be very interested about your performance results between plain jsf and seam if you make such tests.

                              • 12. Re: Seam performance vs plain JSF
                                lowecg2004

                                The debug mode for Seam would have a significant impact on your stats. Have you set the debug mode for Seam to false?

                                In components.xml, try adding:

                                <core:init debug="false" />

                                • 13. Re: Seam performance vs plain JSF
                                  gonzalad

                                  Hi lowecg2004,

                                  Thanks for the tip.

                                  Just modified the jpa-no-a4j.ear application adding :

                                  <core:init debug="false" />


                                  But same results than before (looking at Seam code, debug="false" is the default value).
                                  So, for a constant throughput of 60tx/s, I have the following results :
                                  cpu used=40%
                                  elapsed(ms)=94ms
                                  for a 5 minute long load-test.

                                  Thanks you once more anyway !

                                  • 14. Re: Seam performance vs plain JSF

                                     

                                    "jbalunas@redhat.com" wrote:
                                    These are surprising results for me as well. I will review them and compare with our own servers.

                                    I will be adding more tests and comparisons in regards to performance to the Seam project as time goes on. I appreciate your work on this.


                                    Hi Jay,

                                    It would be nice if after sorting this out an official article about Seam performance was posted. The old article listed at the beginning of this thread is... somewhat old.
                                    And there is much need in more fresh and official info on Seam performance for it to be taken more seriously by enterprises.

                                    Thanks again, I am really glad Red Hat/JBoss team has picked this issue that quickly!

                                    1 2 Previous Next