1 2 Previous Next 21 Replies Latest reply on Jun 12, 2003 6:41 AM by jonlee Go to original post
      • 15. Re: Configure Connection Thread Pool
        danielagatt

        Hi,

        I thank you again for your very detailed feedback, it was very helpful.

        The following are my conclusions following my last tests:

        1. I confirm that this is not a memory issue, at least it was not obvious using the ps -aux command. The following is the final output before I got the out of memory error.

        root 7279 69.7 11.3 350248 117296 pts/0 S 13:29 5:33 /usr/bin/java/bin

        Pls note that the max heap size is set to 500Mb. Is there any other way to see how this is being consumed?

        2. My JSP is connecting to the database but I'm opening a max of 10 connections to it any one time so I don't think that that is causing a problem. The result set (of 2 records approx 32bytes) is loaded into a vector each time.

        3. I noticed from the pstree that JBoss crashes when 204 threads are opened, I still don't know whether this is a JVM, JBoss or Linux issue.

        init-+-atd
        |-bdflush
        |-crond
        |-gdm-binary---gdm-binary-+-X
        | `-gdmgreeter
        |-gpm
        |-java---java---204*[java]
        |-keventd
        |-khubd
        |-5*[kjournald]
        |-klogd
        |-ksoftirqd_CPU0
        |-ksoftirqd_CPU1
        |-kswapd
        |-kupdated
        |-mdrecoveryd
        |-migration_CPU0
        |-migration_CPU1
        |-6*[mingetty]
        |-ntpd
        |-portmap
        |-rhnsd
        |-rpc.statd
        |-scsi_eh_0
        |-scsi_eh_1
        |-scsi_eh_2
        |-2*[sendmail]
        |-sshd
        |-syslogd
        |-xfs
        `-xinetd-+-in.telnetd---login---bash---su---bash---pstree
        `-in.telnetd---login---bash---su---bash---tail

        Any other ideas?

        Cheers
        Daniela

        • 16. Re: Configure Connection Thread Pool
          jonlee

          Been offline for a while doing some optimizing and testing on the JBoss container trying to squeeze a bit more speed out of our apps.

          OK. I would suggest that you probably start out with jmemprof to do some more analysis to see what is going on. Actually, this page has all the links - I would go with the free profilers first as they are usually enough to give you a heads up of what is going on.

          http://www.javaperformancetuning.com/resources.shtml

          You should be able to keep an eye on the heap consumption and other things. It may be that the Tomcat listeners are leaking. It sounds a bit like a leakage issue as the connector is breaking but the container itself does not seem to crash. I can't see any reason that a standard RedHat Linux would prevent you from creating 204 threads as this is well below the standard limitation.

          Try analysing that first and go from there. You'll probably need to get the Tomcat source to poke around in that.

          If you are still having problems, you can try two other things.

          1) Try switching the JDK - if you can join IBM as a developer partner, you can get the IBM 1.4 SDK. As they test this extensively to ensure that their J2EE products like WebSphere AS 5.0 run flawlessly on RedHat Linux it would be good to get this. I don't believe this will fix your issue but stranger things have happened.

          2) Try using the Jetty build. It is a bit simpler, lightweight and we can more easily modify the listener code to debug if it comes down to it.

          Hope that helps.

          • 17. Re: Configure Connection Thread Pool
            danielagatt

            Hi,

            I'm extremely glad to tell that I have managed to open my 600 connections :-) BUT to manage this...

            I ported my war file to a standalone Tomcat application on a Solaris 8 machine with 512Gb RAM. This is a bigger server than the other (Netra T1)

            I will now be installing JBoss and replicating the configuration on the first machine. If this works then I know it is either a Linux or machine specific problem. I found it so much easier to monitor memory on Solaris!!

            I will let you know how it goes but thks anyhow for yr latest feedback!

            Cheers
            Daniela

            • 18. Re: Configure Connection Thread Pool
              camel

              You may also want to check your ulimit: under bash, type "ulimit -a", and make sure your open files limit is set high, and your max user processes. Each open socket requires a file descriptor I believe. I ran a few load testers and found out that my redhat 7.3 linux maxes out at about 1000 threads, and around 3500 open sockets.

              The volano report cited earlier (http://www.volano.com/report/index.html) has some useful notes.

              Hope this helps

              • 19. Re: Configure Connection Thread Pool
                danielagatt

                core file size (blocks, -c) 0
                data seg size (kbytes, -d) unlimited
                file size (blocks, -f) unlimited
                max locked memory (kbytes, -l) unlimited
                max memory size (kbytes, -m) unlimited
                open files (-n) 1024
                pipe size (512 bytes, -p) 8
                stack size (kbytes, -s) 8192
                cpu time (seconds, -t) unlimited
                max user processes (-u) 7168
                virtual memory (kbytes, -v) unlimited

                There don't seem to be any problems here. I'm suspecting that the -Xmx parameter adjusting the heap size was being ignored because I got the same error on Solaris before I adjusted the heap size. Can this be?? I did make the change as you suggested.

                One small question...is JBoss 4.0 stable or should I stick with the 3.21 release for now. I am using JDK1.4.1.

                Cheers
                Daniela

                • 20. Re: Configure Connection Thread Pool
                  danielagatt

                  I don't believe it...I discovered what the problem was on Linux and I'm fuming!!!

                  I configured JBoss as a Unix Service using the scripts given in the JBoss 3.0 Quick Start Guide. One of these scripts is called go.sh which I assumed was calling run.sh. However, I did originally set the JAVA_OPTS environment variable in go.sh which did not seem to work, so I included it in run.sh.

                  Following the installation on Solaris, I tried to setup the Unix service again but didn't manage because the example scripts are actually for just Linux. So I modified them for Solaris and whilst doing that I realised the run.sh was not being called, therefore I was not setting the heap size at all!!!

                  Anyway, I thank you for yr very good support but JBoss really needs to do something about the lousy documentation. It is practically impossible to work without it!

                  Rgds
                  Daniela

                  • 21. Re: Configure Connection Thread Pool
                    jonlee

                    You mean it is impossible to work without lousy documentation? ;) I know what you mean.

                    Anyway, good to hear that everything is resolved. Now you can go and do the real app development stuff.

                    1 2 Previous Next