Could you describe a bit more detail?
- When the JBoss stop, during startup, under load or idle?
- How the CPU/process load at this time
- does debug/tract for log4j give any hint
- can JConsole view any information?
Don't rely on Windows Service Manager giving you any meaningful information. Instead, look in the server.log file. Most likely something caused a shutdown command to be sent to the app server and it shut itself down.
Also, did you set the -Xrs JVM option? Setting this option will prevent unwanted "shut down" signals. See this the "Myterious Shutdowns" section on this wiki page: http://community.jboss.org/wiki/RunJBossAsAServiceOnWindows
Thanks for your responses, I have already set -Xrs property. I want to add that there is nothing in server.log (or any other) and there is no JVM crash error log.
here is description of my server:
JVM: Java HotSpot(TM) 64-Bit Server VM (1.6.0_17)
The problem that I see in JConsole:
Operating System: Windows 2003 5.2
Number of processors: 4
Committed virtual memory: 5,335,164 kbytes
Total physical memory: 4,455,644 kbytes
Free physical memory: 159,752 kbytes
Total swap space: 10,373,196 kbytes
Free swap space: 4,720,480 kbytes
Committed virtual memory permanently grows and when no Free swap space is availible service is killed.
It looks like native memory leak. I don't use any native libraries by myself.
I have read about some issue with VFS leaking, beacause of Sun JDK bug.
But unfortunately there is no solution for this issue yet.
However I can't say exactly if this bug is growing my Committed virtual memory.
Are there any known issues? Or ways to profile native memory leaks on windows?
I'm a bit confused about 64-Bit Server VM (17.0-b16, mixed mode)
it looks like 1.7 JVM. If so could you retest with an actual 1.6 VM?
The heap usage looks not bad.
VisualVM displays Version that way, 6u17 is the version (1.6.0_17)
I've update my previouse post, to make it clearer.
For last 3.5 Hours
Committed virtual memory: 5,837,052 kbytes
memory has increased for 500mb, compared to what I wrote earlier.
https://issues.jboss.org/browse/JBVFS-159 was reported as Sun JDK Linux version specific issue.
Try to use some memory profiling tool on your application to try to find out what consumes all this memory. Often you can dramatically reduce memory consumption by optimizing your classpath - standardize on specific library versions and move them from your.war/WEB-INF/lib to jboss_home/server/default/lib .
Thank you Marko for your response.
I will take in mind your siggestion.
In fact I've profiled my application many times and and there are no memory leaks on application layer, as heap memory is normal, and there are no OOME in the application. I used Yourkit, visualVM, JConsole and all needed tools.
But is there any way to profile native memory consumed by whole JVM?
I have no experience with tracking native memory leaks, but I'm pretty sure that kind of profiling is beyond the capabilities of Java profilers like those you mention: JConsole, VisualVM, YourKit ...
A quick web search, and I found this: http://www.oracle.com/technetwork/java/javase/memleaks-137499.html#gbyvk
Not for the faint hearted ...
You have to limit -Xmx to well below max physical memory of the machine. Once you run out of physical memory, and your java process still demands more heap (or extra heap memory for that matter) you're in deep trouble, as your java process will become tremendously slow, and your system will most likely become non-responsive.
Still having the issue.
Can +UseParallelOldGC cousing the Virtual Memory growth?
Or 3GB Heap Size on 4GB total memory server?
Is the issue with VFS related only for Linux OS and not for Windows?
A hint for you, if jconsole is active it will consume (increasing) memory of the servers JVM!
No it is defenitely not because of JConsole, or any other tool, they are not attached all the time
Sorry, I meant this only at hint that jConsole might compound your problem, not that jConsole is the problem ;-)