This is a JVM bug, you need to report it to Sun.
Are there any work arounds?
Does this mean that if you expect to have a heavy load that you can not use 1.4 or 1.4.1?
We are seeing a very similar thing on Win2k Server with JBoss/Jetty 3.0.2 (and 3.0.3) using 1.4.0.02 as well as 1.4.1 jdks. Its been two days and we have not been able to bring our server up under load.
We get no exceptions and no OS errors. The jvm just shuts down. Even JBoss doesn't do its normal clean shutdown. The jvm just stops.
Is this consistant with what you are seeing?
Thanks for any info.
I see the error like I posted above on Solaris. On windows I get the dialog box that says java has created an error... a report is being..., or something like that.
I have not been able to recreate this problem with Tomcat stand-alone. If it is a jvm bug, it looks like it's a bug that jboss hits.
At least you are seeing something reported from the OS. What we are experiencing is almost as if someone (or something) killed the process immediately with no error trail or execptions or GPF, etc.
I don't even know how to begin to report this. And to who?
Thanks again for any info.
I'm currently running jmeter against tomcat stand-alone. I have 75 threads running 100 request each. Halfway through without a problem.
When we get over 10 threads hitting jboss it core dumps.
Anyone have a solution?
I'd much rather go to production with JBoss-jetty than tomcat stand-alone.
I've finished a 75 thread 100 each run with tomcat with no problems. About to finish running 75 threads for 200 hits each. 15,000 hits with no problems.
I feel pretty confident in saying that the problem/bug/whatever is in, or has been found by, JBoss.
Forgot to mention that I tried JBoss3.0.3 with tomcat 4.1.12 and it core dumpted also.
Tomcat 4.1.12 by itself seems to work fine tho.
I know there have to be more people watching this thread than replying, so this is in the spirit of a broadcast. And yes, we will submit a bug report to Sun as soon as we have a moment to breath.
We're preparing a website application (MySQL/database access and no EJBs) and have experienced everything mentioned above. We used new 1.4 language features in the app and became quite stuck. We're close to or at a reasonable solution, having learned the following:
We tried all sorts of OS configurations and updates, and nothing seemed to help.
Crashing "under load" may be misleading. If I let the app run long enough, it crashes under light or almost non-existent load. (Although I agree with the first post indicating 35-40 threads being a fast path to ruination. We're using jmeter, too ;) ).
I get the behavior on my W2K laptop where Netbeans runs happily for days with the same JVM.
A quick predictor of unstable behavior is when memory use jumps up (in Task Manager or top under Linux) on every request, even though you're reasonably sure no new web server threads needed to be spawned for it. The memory is NOT reclaimed/GCed no matter how long you wait.
The abrupt halt in Linux/Solaris seems to be the interaction with the shell. It does essentially the same thing in Windows when run under Cygwin - you get a "java.exe error" dialog box. However, when running from a DOS shell, I get a happy out of memory error (albeit with a JBoss shutdown). Watching more closely after seeing this, I noticed memory use did actually spike way up over whatever limit you feed the JVM (spiking very quickly and briefly).
On Win2K (by far the most stable/forgiving JVM), the app is much harder to crash. For instance, using a JBoss/MySQL app, it was happy under load for W2K but crashed quickly under Linux/Solaris. However, after migrating MySQL tables from MyISAM to InnoDB, it crashed in Windows, too. In fact, in one of our crashes the error output indicated a database-related thread, so the problem may be around there. On that note, we still need to review our database code soon to make it as well-behaved as possible (even though it is routine stuff right now).
Now for the happy part - I think we finally managed a stable Windows configuration by compiling JBoss under 1.4.1. (I think the binary distro is a 1.3.1 product). Linux became much happier and stayed up longer, but crashed eventually. Haven't tried Solaris yet.
If you haven't done this yet, it's extremely easy if and only if you follow the "Source Code" page directions to grab it from CVS. I never could get the snapshot tarball to work. I got the 3.0.3 branch with "-r JBoss_3_0_3". Just cd to the build dir and run build.bat or build.sh; works every time!
If this is a jvm cross-version compatibility issue, my next casual question is: Does the JBoss build compile everything in the distro from source, or does it package certain libraries (e.g. Jakarta stuff) from their binary distibutions? I'll answer this for myself soon, but if someone knows I appreciate the shortcut. I still need to go back through all extra libraries and drivers our app using and compile them. Maybe Linux will settle down then, too.
Hope this helps someone!
I didn't even think about building jboss under 1.4.1.
Bad news. :(
Rebuilt JBoss with 1.4.1 . . . Still core dumps :(:(:(
All tests with tomcat stand-alone have been 100% successful.
Oh, one more thing I forgot. Especially if you're using jsp includes, esp. nested jsp includes, they must be precompiled (thank you Ant). That was an original source of crashes for us under Windows.
What can you say about the architecture/configuration of your application?
> If this is a jvm cross-version compatibility issue,
> my next casual question is: Does the JBoss build
> compile everything in the distro from source, or does
> it package certain libraries (e.g. Jakarta stuff)
> from their binary distibutions?
We don't compile third party libraries.