14 Replies Latest reply on Sep 30, 2003 8:11 PM by jonlee

    A possible sad farewell ...

    kryptontri

      I have been using jboss for a while now, only in the last year i moved to launch a commercial site which will be heavy traffic based. I tried optimizing jboss by:

      - (1)
      <container-configurations>
      <container-configuration extends="Standard CMP 2.x EntityBean">
      <container-name>CMP 2.x and Cache</container-name>
      <commit-option>A</commit-option>
      </container-configuration>
      </container-configurations>
      -(2) example:

      <ejb-name>AddressBean</ejb-name>
      <configuration-name>CMP 2.x and Cache</configuration-name>
      <jndi-name>local/AddressBean</jndi-name>
      <method-attributes>

      <method-name>get*</method-name>
      <read-only>true</read-only>

      </method-attributes>



      Results ..... :-(. ... ... i'm embarressed to say ..... the box has 1G memory, redhat, mysql, local entity + session wrappers, celeron ~1.3G, pipe is fast ...(candidhosting) ... and still ....... *slow*

      Ok so i have not tried 1 last step - compile with optimize on .... (jboss & app) ... but ....my thoughts to rescue this venture are now focussed on orionserver.
      Its fast out of the box.

      Help. Have I left anything else out of the jboss config ? Forget the thread numbers on jetty etc ... 1 user (me) and it still takes time to service a request, I can't seem to figure it out, ... the site is serving jsps, images, and videos ... can *anyone* offer me some advice totry 1 last time before i consider orion ? Please help ....

      K

        • 1. Re: A possible sad farewell ...
          kryptontri

          Forgot to mentilon jboss3.2.0 + jetty + mysql. For now I am on a 56K modem away from home(adsl) .. and even asp based sites are almost 3 x faster. The application is not that intensive, jsps - > wrapper delegate pojos - > session wrappers - > local entity.

          Having been doing java for 5 years +, I find it hard to beleive these are due to 'basic' coding erros/mistakes.

          Can anyone provide and tips or last minute advice before I move on from jboss if i do ? Example ... images ... max 100K size, are served from file system, rather than db access blob/clob etc .... but page renderring .... is ..... in conservative terms .... unbeleivable slow, even though the hosting provider is on a high speed segment t1 or t1++

          K

          • 2. Re: A possible sad farewell ...
            kryptontri

            jonlee any advice? you seem to be most active on this specific topic ?

            • 3. Re: A possible sad farewell ...
              kryptontri

              Sorry for got to add ... java version

              java version "1.4.1_02"
              Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_02-b06)
              Java HotSpot(TM) Client VM (build 1.4.1_02-b06, mixed mode)

              redhat
              [jboss@server jboss]$ ls /boot
              boot.0300 module-info-2.4.18-24.7.x
              boot.b module-info-2.4.18-27.7.x
              chain.b module-info-2.4.20-19.7bigmem
              config-2.4.18-24.7.x os2_d.b
              config-2.4.18-27.7.x System.map
              config-2.4.20-19.7bigmem System.map-2.4.18-24.7.x
              grub System.map-2.4.18-27.7.x
              initrd-2.4.18-24.7.x.img System.map-2.4.20-19.7bigmem
              initrd-2.4.18-27.7.x.img vmlinux-2.4.18-24.7.x
              initrd-2.4.20-19.7bigmem.img vmlinux-2.4.18-27.7.x
              kernel.h vmlinux-2.4.20-19.7bigmem
              lost+found vmlinuz
              map vmlinuz-2.4.18-24.7.x
              message vmlinuz-2.4.18-27.7.x
              module-info vmlinuz-2.4.20-19.7bigmem


              • 4. Re: A possible sad farewell ...
                jonlee

                There are a lot of factors that will impact your system. There are a few things to trial. When you think things are unacceptably slow, you need to start checking for problems (other than JBoss).

                Experiment with different JVMs - they can give varying results. I would suggest using the IBM SDK. Build an optimized JBoss distribution. Define the hsqdb-ds.xml for non-JDBC configuration (either in-process or in-memory). Remove sub-systems you will not be using. If there are still hold ups, you need to start diagnosing where these occur and what is the cause. Optimizeit or a freeware tool will help you track down hold ups.

                If pure Servlet/JSP execution seems slow, try switching between Jetty and Tomcat and see if there is any difference.

                The other thing is with plenty of large static content, use a front-end web server. Native execution is always going to be faster than Java so Apache will always be better for serving your static information, performance wise. It is also faster for dealing with SSL negotiation and transfer - same reason.

                Hope that helps.

                • 5. Re: A possible sad farewell ...

                  If you go through the process of changing out jetty for tomcat that will take care of part of the speed/perfromance problem..

                  Or you may find that after completing the change tha tis solves all your speed/performance problems..

                  I even noticed large difference sin performance on switching from jetty to tomcat on my developer workstation ...

                  • 6. Re: A sad farewell ... [to orion]
                    kryptontri

                    thanks for the tips, for now my tasks are

                    - use jrocket jvm
                    - compile jboss source with local.properties set to production mode (btw - does this need to be in the build folder or build/etc folder, can't see any reference where to put it)
                    - use jboss-3.2.2RC3 (tomcat) - i did see some speed with tomcat a few days ago ..

                    There is not much static content, its mostly dynamic, sorry for my rant earlier ... i was very worried ... i thought i had done everything possible, i'll give these a go and see what happens ...

                    Thanks

                    k

                    • 7. Re: A sad farewell ... [to orion]
                      jonlee

                      There should be a build/local.properties file in the source distribution. Modify that for your build - the javac properties for optimize and debug. The build/output directory will contain the completed binaries.

                      • 8. Re: A sad farewell ... [to orion]
                        kryptontri

                        i have recompiled with the production mode, ie optimize, and d/l jrockit, but for now i'm using suns 1.4+ jvm, i'll be having a look in a few hours and see what else i can do. I am not using the all configuraiton, only the default, so at the moment i am not sure what sar/services i can remove from deploy (from default)

                        - jms
                        -jboss-jca.sar (?)
                        -uuid-key-generator.sar (?)

                        It could be jetty is slower than tomcat. I'm using the 3.22RC3 release, with tomcat ...

                        But still looking for ways to optimize ..



                        • 9. Re: A sad farewell ... [to orion]
                          jonlee

                          If you are not using the scheduler or counter services, you can remove those. If you are not intending to supply invocation over HTTP, remove the http-invoker service. Disable the AJP13 service in your Tomcat/Jetty if you are not using it (it doesn't sound like you are). This is just another listener service adding delay and threads in your JVM. As before, either remove hsqldb (if you are not using it at all) or change it so that you are not performing read/writes over the wire.

                          • 10. Re: A sad farewell ... [to orion]
                            dcartier

                            Hi Kryptontri,

                            I also had some performance problems with JBoss when I first switched to it. I am also using Linux, with Apache in front, and Velocity rather than JSP.

                            I found that using the IBM JVM made an incredible difference and my page generation times went from average of ~30ms down to 3ms in many cases.

                            My sites are less complicated than yours, but even my one site using J2EE also had quite a noticeable improvement.

                            Please note, you may have to patch a fix a few problems relating to the IBM JVM. If you have problems realting to sessions, check the forms for my thread. It might already be fixed in 3.2.2RC3, I am still using 3.0.X branch.

                            Dennis

                            • 11. Re: A sad farewell ... [to orion]
                              kryptontri

                              Hi Dennis,

                              I am no using bea's jrockit :-) and this has made a huge difference, i'm using 3.22RC3, there are also some options for tomcat in production which i will use when i finish bug fixing. But so far, all seems well :-) even when i access the site though a 33k modem, on broadband i have been told its almost instant!

                              Thanks

                              K

                              • 12. Re: A sad farewell ... [to orion]
                                lhankins

                                If the performance slowness doesn't appear to be related to load, I would DEFINITELY run it through a tool like OptimizeIt.

                                A few months back we had to do some optimization and I spent the first several days just optimizing the things I knew were slow. I got kind of stuck on one database query I thought was slow, and come to find out, it wasn't really the query that was slow, but something that appeared harmless that was happening immediately after the query. I spent way longer than I should've trying to track this down. Then I ran the same scenario through OptimizeIt and it was IMMEDIATELY clear where the bottleneck was.

                                OptimizeIt found some big thing's I would NEVER have found. (literally - a few of the items would've been the last place I would've looked).

                                It makes no sense to spend days trying to optimize a part of your application that you is slow when, in reality, its not one of your top hotspots (and without running things through a profiler, you won't know where your hotspots are).

                                Profilers pay for themselves VERY quickly IMO... BTW - I believe they have a freely downloadable trial of OptimizeIT.

                                • 13. Re: A possible sad farewell ...
                                  doc_brown

                                  > If you go through the process of changing out jetty
                                  > for tomcat that will take care of part of the
                                  > speed/perfromance problem..
                                  >
                                  > Or you may find that after completing the change tha
                                  > tis solves all your speed/performance problems..
                                  >
                                  > I even noticed large difference sin performance on
                                  > switching from jetty to tomcat on my developer
                                  > workstation ...


                                  Swapping out jetty for tomcat? So is Tomcat the faster front end ? I would think that jetty is faster because it's lighter weight.. or am I thinking wrong?

                                  • 14. Re: A possible sad farewell ...
                                    jonlee

                                    There is still debate on this topic. Standalone server tests have shown that Jetty performs better under load. Responses seem to be on a par with Tomcat.

                                    There may be mitigating circumstances when combined with JBoss. JBoss processes may compete for processor time. Removing unused processes and their thread overheads will help greatly.

                                    Also, the JDK you use may contribute greatly to performance. There was one case where the employed JDK performed a full GC every cycle, signifcantly impacting response. As Kryptontri discovered, the JDK/SDK has a large bearing on performance.

                                    YMMV. If you feel unhappy with Jetty performance, move to Tomcat and test under those conditions.