1 2 Previous Next 15 Replies Latest reply on Nov 15, 2006 4:00 PM by Viet

    JBoss Portal Stress Results

    Waldemar Baraldi Newbie

      Hi,

      I would like to share with you some results I gathered from stress testing JBoss Portal 2.4 and 2.6DR on JBoss AS 4.0.5.GA.

      My CPU (running the Server) is
      AMD Sempron 2800+ 1.6Ghz. with 1Gb of RAM
      Windows Server 2003 SP1

      Client CPU (running the Stress Tool) is a similar one. They are LAN connected. Results were obtained using Microsoft Web Application Stress Tool.

      The difference between both version is quite high. Is there any other performance results you can point me to?. Do you think they are accurate or I misconfigured something? Are these results close to what we should expect from JBoss Portal with this configuration?

      Thanks in advance for your opinions,

      Waldemar

      Overview ================================================================================
      Report name: JBossPortal_2.4
      Run on: 11/10/2006 6:46:42 PM
      Run length: 00:02:50
      
      Web Application Stress Tool Version:1.1.293.1
      
      Number of test clients: 1
      
      Number of hits: 756
      Requests per Second: 6.30
      
      Socket Statistics
      --------------------------------------------------------------------------------
      Socket Connects: 756
      Total Bytes Sent (in KB): 181.62
      Bytes Sent Rate (in KB/s): 1.51
      Total Bytes Recv (in KB): 8552.99
      Bytes Recv Rate (in KB/s): 71.29
      
      Socket Errors
      --------------------------------------------------------------------------------
      Connect: 0
      Send: 0
      Recv: 0
      Timeouts: 0
      
      RDS Results
      --------------------------------------------------------------------------------
      Successful Queries: 0
      
      
      Script Settings ================================================================================
      Server: 10.152.0.63
      Number of threads: 15
      
      Test length: 00:02:00
      Warmup: 00:00:25
      Cooldown: 00:00:25
      
      Use Random Delay: No
      
      Follow Redirects: Yes
      Max Redirect Depth: 15
      
      
      Clients used in test ================================================================================
      localhost
      
      Clients not used in test ================================================================================
      
      
      Result Codes
      Code Description Count
      ================================================================================
      200 OK 756
      
      
      Page Summary
      Page Hits TTFB Avg TTLB Avg Auth Query
      ================================================================================
      GET /portal/ 756 2324.73 2365.84 No No



      Overview ================================================================================
      Report name: 11/10/2006 5:55:05 PM
      Run on: 11/10/2006 5:55:05 PM
      Run length: 00:02:50
      
      Web Application Stress Tool Version:1.1.293.1
      
      Number of test clients: 1
      
      Number of hits: 116
      Requests per Second: 0.96
      
      Socket Statistics
      --------------------------------------------------------------------------------
      Socket Connects: 116
      Total Bytes Sent (in KB): 27.87
      Bytes Sent Rate (in KB/s): 0.23
      Total Bytes Recv (in KB): 1687.10
      Bytes Recv Rate (in KB/s): 14.02
      
      Socket Errors
      --------------------------------------------------------------------------------
      Connect: 0
      Send: 0
      Recv: 0
      Timeouts: 0
      
      RDS Results
      --------------------------------------------------------------------------------
      Successful Queries: 0
      
      
      Script Settings ================================================================================
      Server: 10.152.0.63
      Number of threads: 15
      
      Test length: 00:02:00
      Warmup: 00:00:25
      Cooldown: 00:00:25
      
      Use Random Delay: No
      
      Follow Redirects: Yes
      Max Redirect Depth: 15
      
      
      Clients used in test ================================================================================
      localhost
      
      Clients not used in test ================================================================================
      
      
      Result Codes
      Code Description Count
      ================================================================================
      200 OK 116
      
      
      Page Summary
      Page Hits TTFB Avg TTLB Avg Auth Query
      ================================================================================
      GET /portal/ 116 14794.99 15167.84 No No
      





        • 1. Re: JBoss Portal Stress Results
          Roy Russo Master

          Its actually not the portal-core. Running the same test app on my devlaptop (WinXPSP2, 2GB, 2.13GHz), I ended up with similar results for 2.6DR. I then went in to the admin screen and removed the CMS Portlet, and here is what I have:

          Overview
          ================================================================================
          Report name: 11/10/2006 5:38:43 PM
          Run on: 11/10/2006 5:38:43 PM
          Run length: 00:02:50
          
          Web Application Stress Tool Version:1.1.293.1
          
          Notes
          --------------------------------------------------------------------------------
          JBoss Portal 2.6DR
          
          Number of test clients: 1
          
          Number of hits: 1649
          Requests per Second: 13.73
          
          Socket Statistics
          --------------------------------------------------------------------------------
          Socket Connects: 1649
          Total Bytes Sent (in KB): 391.32
          Bytes Sent Rate (in KB/s): 3.26
          Total Bytes Recv (in KB): 16923.18
          Bytes Recv Rate (in KB/s): 140.91
          
          Socket Errors
          --------------------------------------------------------------------------------
          Connect: 0
          Send: 0
          Recv: 0
          Timeouts: 0
          
          RDS Results
          --------------------------------------------------------------------------------
          Successful Queries: 0
          
          
          Script Settings
          ================================================================================
          Server: localhost
          Number of threads: 15
          
          Test length: 00:02:00
          Warmup: 00:00:25
          Cooldown: 00:00:25
          
          Use Random Delay: No
          
          Follow Redirects: Yes
          Max Redirect Depth: 15
          
          
          Clients used in test
          ================================================================================
          localhost
          
          Clients not used in test
          ================================================================================
          
          
          Result Codes
          Code Description Count
          ================================================================================
          200 OK 1649
          
          
          Page Summary
          Page Hits TTFB Avg TTLB Avg Auth Query
          ================================================================================
          GET /portal 1649 1076.51 1092.96 No No
          


          We're going to look in to this, as its not behaving as planned.. You didn't say which 2.4 version you were running against, but everything from 2.4.1 to 2.6DR now has Cache available in the CMS.



          • 2. Re: JBoss Portal Stress Results
            Waldemar Baraldi Newbie

            Hi Roy,

            Thanks a lot for your quick answer.

            We have performed the tests deploying JBoss 2.4.0, and 2.6.0-DR1 on JBoss AS 4.0.5.GA.

            What we are wondering is why 2.6 is slower than 2.4. Maybe because of its development state?

            I'll perform a new test hitting a page without the CMS Portlet as soon as I get to the office on Monday. I'll post my results then.

            Best Regards,

            Waldemar

            • 3. Re: JBoss Portal Stress Results
              Roy Russo Master

              Well, keep in mind that 2.6 was pushed out by the portal team developers earlier than planned, because we wanted feedback on the direction we were taking wrt usability.

              As for the CMS, Sohil will be profiling it this week and see where the problem is. I'll tell him to post here if he finds anything.

              • 4. Re: JBoss Portal Stress Results
                Waldemar Baraldi Newbie

                Hi,

                I've performed the test without the CMS Portlet and I've got similar results (13 pages/sec.).

                Besides, I have activated a higher level of logging and found that for rendering '/portal' page it makes extensive use of hibernate, many times going beyond the cache and making actual queries to the database (even without the CMS portlet). I guess this could be a performance bottleneck. I believe that after the first page render there aren't reasons to have new accesses to the DB, layout and such can be cached. I wonder how much we can tune JBoss Portal to get better performance in a production environment. In fact, how much can we push it beyond the 13p/s?.

                Regards,

                Waldemar

                • 5. Re: JBoss Portal Stress Results
                  Roy Russo Master

                   

                  "walbar" wrote:
                  Besides, I have activated a higher level of logging and found that for rendering '/portal' page it makes extensive use of hibernate, many times going beyond the cache and making actual queries to the database (even without the CMS portlet).


                  Depends where in the DB its accessing. Remember, that Hib does not allow cacheing Blobs. Im not sure about others parts of the DB schema, but the CMS (I realize you removed it) does makes extensive use of Blobs (stupid jackrabbit architecture!), so we were forced to add another level cache.

                  Sohil should be able to jprofile things a bit better than my random shots in the dark posts here, and Ill have him post here. ;-)

                  • 6. Re: JBoss Portal Stress Results
                    Waldemar Baraldi Newbie

                    Thanks a lot for your answers, we will be looking forward to see Sohil news :-)

                    Anyway, let's assume we don't care about the CMS Portlet. What about Portal Core performance? Shall we assume that it is difficult to get more than 13 p/s with this configuration? Shall we assume that the database is going to be hit even when the page doesn't have the CMS Portlet? Is there some cache technology that is still to be included?

                    Regards,

                    Waldemar

                    • 7. Re: JBoss Portal Stress Results
                      Roy Russo Master

                      After talking with Julien, he is telling me that 2.6DR has not been fine-tuned. Some tables will likely change... WSRP and CMS security come to mind, so he chose to left the Hibernate level in a development state. I would expect our Beta (not this coming Alpha) to be fine-tuned.

                      • 8. Re: JBoss Portal Stress Results
                        Waldemar Baraldi Newbie

                        Hi,

                        Thanks a lot for your answer. Following your line of thinking I tested 2.4 home page without the CMS Portlet and got 20 p/s. Do you think that 2.6 will give us this number of pages once it is fine-tuned? May we expect a better number after the changes to the tables you mention?

                        Regards,

                        Waldemar

                        • 9. Re: JBoss Portal Stress Results
                          Roy Russo Master

                          Sohil found two things in the 2.6 codebase and they've been fixed (and backported to 2.4):

                          1. CMS regexp I added for removal of everything except *. The regexp was being run every time, over the cached object, regardless if the tag existed.

                          2. Portal ResourceBundle loader seemed to be buggy/slow.

                          He ran the windows test tool on 2.6 and now has this (WITH the CMS):

                          Overview
                          
                          ================================================================================
                          
                          Report name: 11/13/2006 8:35:32 PM
                          
                          Run on: 11/13/2006 8:35:32 PM
                          
                          Run length: 00:02:51
                          
                          
                          
                          Web Application Stress Tool Version:1.1.293.1
                          
                          
                          
                          Number of test clients: 1
                          
                          
                          
                          Number of hits: 2713
                          
                          Requests per Second: 22.57
                          
                          
                          
                          Socket Statistics
                          
                          --------------------------------------------------------------------------------
                          
                          Socket Connects: 2713
                          
                          Total Bytes Sent (in KB): 643.81
                          
                          Bytes Sent Rate (in KB/s): 5.35
                          
                          Total Bytes Recv (in KB): 42361.48
                          
                          Bytes Recv Rate (in KB/s): 352.35
                          
                          
                          
                          Socket Errors
                          
                          --------------------------------------------------------------------------------
                          
                          Connect: 0
                          
                          Send: 0
                          
                          Recv: 0
                          
                          Timeouts: 0
                          
                          
                          
                          RDS Results
                          
                          --------------------------------------------------------------------------------
                          
                          Successful Queries: 0
                          
                          
                          
                          
                          
                          Script Settings
                          
                          ================================================================================
                          
                          Server: localhost
                          
                          Number of threads: 1
                          
                          
                          
                          Test length: 00:02:00
                          
                          Warmup: 00:00:25
                          
                          Cooldown: 00:00:25
                          
                          
                          
                          Use Random Delay: No
                          
                          
                          
                          Follow Redirects: Yes
                          
                          Max Redirect Depth: 15
                          
                          
                          
                          
                          
                          Clients used in test
                          
                          ================================================================================
                          
                          localhost
                          
                          
                          
                          Clients not used in test
                          
                          ================================================================================
                          
                          
                          
                          
                          
                          Result Codes
                          
                          Code Description Count
                          
                          ================================================================================
                          
                          200 OK 2713
                          
                          
                          
                          
                          
                          Page Summary
                          
                          Page Hits TTFB Avg TTLB Avg Auth Query
                          
                          ================================================================================
                          
                          GET /portal 2713 39.32 42.92 No No
                          
                          


                          As we get closer to 2.6Beta, expect throughput to increase.

                          • 10. Re: JBoss Portal Stress Results
                            Viet Master

                            I have added a couple of improvements in 2.6 trunk, would you mind to test it and see what you get ?

                            "walbar" wrote:
                            Hi,

                            Thanks a lot for your answer. Following your line of thinking I tested 2.4 home page without the CMS Portlet and got 20 p/s. Do you think that 2.6 will give us this number of pages once it is fine-tuned? May we expect a better number after the changes to the tables you mention?

                            Regards,

                            Waldemar


                            • 11. Re: JBoss Portal Stress Results
                              Waldemar Baraldi Newbie

                              Hi,

                              I've performed the test on the new version from the trunk. I get:

                              With the CMS Portlet: 15 p/s
                              Without the CMS Portlet: 31p/s

                              Yesterday I was trying to find out where in your code there were the SQL queries logued by hibernate. I was suspecting that they were the bottleneck but the source code is quite big and I had not much success. Anyway I see you have worked on it and now only the cache is hit without actually reaching the DB.

                              I guess that another possible bottleneck are the JTA transactions that are continuously being opened and closed. I would like to try to remove them in order to perform more tests, would you point me to the config or code file where this could be done (if possible)?. I wouldn't mind to lack some functionality during the tests.

                              Regards,

                              Waldemar

                              • 12. Re: JBoss Portal Stress Results
                                Roy Russo Master

                                 

                                "walbar" wrote:

                                I've performed the test on the new version from the trunk. I get:

                                With the CMS Portlet: 15 p/s
                                Without the CMS Portlet: 31p/s


                                Which version?

                                "walbar" wrote:

                                I guess that another possible bottleneck are the JTA transactions that are continuously being opened and closed. I would like to try to remove them in order to perform more tests, would you point me to the config or code file where this could be done (if possible)?. I wouldn't mind to lack some functionality during the tests.


                                umm... that would kill most of the bundled portlets. Do you intend to only test the core?

                                • 13. Re: JBoss Portal Stress Results
                                  Waldemar Baraldi Newbie

                                  Those tests were performed on the 2.6 trunk version Julien mentioned above.

                                  In order to use JBoss Portal in my project I need better performance. I'm trying to guess how much it could give by tuning, caching, etc. From 0.96 p/s (in my first post) to 15p/s on version 2.6 there is a great improve but I would like to know if we would be able to push it further.

                                  You are right, if removing JTA will break most of the protlets, then I guess that test is useless since it would be too far from the real environment.

                                  Regards,

                                  Waldemar

                                  • 14. Re: JBoss Portal Stress Results
                                    Waldemar Baraldi Newbie

                                    Anyway we don't need JTA in production since most of our portlets' logic would be read-only. We would use JTA during Staging, and in that phase we are not interested in performance yet.

                                    So, I'm still interested in knowing where I could remove JTA transactions. Would you help me?

                                    Thanks a lot,

                                    Waldemar

                                    1 2 Previous Next