-
1. Re: OutOfMemoryError in JBoss 5.01
alesj Sep 21, 2009 5:02 AM (in response to yongz)There is this memory leak:
- https://jira.jboss.org/jira/browse/JBCL-119
but if you're not re-deploying like mad +
have explicit CL dependencies on "static" deployment,
you're not hitting this. -
2. Re: OutOfMemoryError in JBoss 5.01
jaikiran Sep 21, 2009 5:12 AM (in response to yongz)In addition to what Ales said - it's better to move to AS 5.1.0 instead of 5.0.1 because 5.1.0 has a lot of bug fixes.
-
3. Re: OutOfMemoryError in JBoss 5.01
yongz Sep 21, 2009 5:14 AM (in response to yongz)Thanks a lot for the response.
but if you're not re-deploying like mad + have explicit CL dependencies on "static" deployment
Our EAR is about 37 MB, and we are using JBossws-metro.
Would you mind explaining a bit more about the re-deploying and explicit CL dependencies on "static" deployment?
Is there a way that we can avoid this? -
4. Re: OutOfMemoryError in JBoss 5.01
alesj Sep 21, 2009 5:18 AM (in response to yongz)"yongz" wrote:
Would you mind explaining a bit more about the re-deploying and explicit CL dependencies on "static" deployment?
Like I said, first, I doubt you're hitting this,
as explicit CL dependencies are not yet common in recent deployments.
But I might be wrong. :-)
The scenario goes like this.
You have deployments A and B, where B expresses explicit CL dependency on A
(via jboss-classloading.xml or some other CLMetaData info).
Then you re-deploy B a lot of times, meaning A's dependsOnMe info grows big ==> OOME. -
5. Re: OutOfMemoryError in JBoss 5.01
yongz Sep 21, 2009 5:34 AM (in response to yongz)Thanks Ales/JAIKIRAN, much appreciated.
via jboss-classloading.xml or some other CLMetaData info
The issue you described is close to what I saw, but we did not make any of the above CL changes to JBoss or in our application.
A few more questions:
1. Could you also confirm this CL issue would cause the server cannot handle load(more threads)?
2. Is there something(jars, etc) we can upgrade in AS 5.01?
3. As JAIKIRAN suggested, will this issue be fixed in 5.1.0?
Sorry about so many questions, but I have to sort this out on prod quickly. -
6. Re: OutOfMemoryError in JBoss 5.01
alesj Sep 21, 2009 6:39 AM (in response to yongz)"yongz" wrote:
1. Could you also confirm this CL issue would cause the server cannot handle load(more threads)?
This is a mem-leak, so it can cause a bunch of weird issues. :-("yongz" wrote:
2. Is there something(jars, etc) we can upgrade in AS 5.01?
You could, but it's risky as you don't exactly know how things/libs are related.
So, the best thing to do is to use the latest version of AS."yongz" wrote:
3. As JAIKIRAN suggested, will this issue be fixed in 5.1.0?
5.1.0 is already out, whereas this was just recently found and fixed.
It will be part of upcoming AS release, 5.2.x. -
7. Re: OutOfMemoryError in JBoss 5.01
yongz Sep 21, 2009 7:33 AM (in response to yongz)Hi Ales,
Unfortunately I was able to reproduce this issue on 5.1.0 with:
-Xms1300m -Xmx1300m -XX:MaxPermSize=256m2009-09-21 21:26:26,966 [main] ERROR kernel.plugins.dependency.AbstractKernelCon
troller : Error installing to Real: name=vfsfile:/D:/Temp/jboss-5.1.0.GA/server
/myasconfig/deploy/my-app.ear/ state=PreReal mode=Manual requiredState=Real
org.jboss.deployers.spi.DeploymentException: Error deploying prod-app.ear:
null
at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:196)
at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:99)
at org.jboss.deployers.vfs.spi.deployer.AbstractVFSRealDeployer.internal
Deploy(AbstractVFSRealDeployer.java:45)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(
AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(Deployer
Wrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(Deployer
sImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFi
rst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(Deployers
Impl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(Abstra
ctControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractContr
oller.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(Abstra
ctController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(Abstr
actController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(Abstr
actController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractContro
ller.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractContro
ller.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(Deployers
Impl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeploye
rImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter
.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction
.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileActi
on.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileServ
ice.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(Abstra
ctControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractContr
oller.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(Abstra
ctController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(Abstr
actController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(Abstr
actController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractContro
ller.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractContro
ller.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileServ
ice.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(
ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:
461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.OutOfMemoryError
at java.util.zip.Inflater.init(Native Method)
at java.util.zip.Inflater.<init>(Inflater.java:75)
at java.util.zip.ZipFile.getInflater(ZipFile.java:375)
at java.util.zip.ZipFile.getInputStream(ZipFile.java:320)
at java.util.zip.ZipFile.getInputStream(ZipFile.java:286)
at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFi
leWrapper.java:214)
at org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipE
ntryContext.java:1082)
at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipE
ntryHandler.java:153)
at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:230)
at org.jboss.ejb3.Ejb3Deployment.deployUrl(Ejb3Deployment.java:650)
at org.jboss.ejb3.Ejb3Deployment.deploy(Ejb3Deployment.java:614)
at org.jboss.ejb3.Ejb3Deployment.create(Ejb3Deployment.java:491)
at org.jboss.ejb3.deployers.Ejb3Deployer.deploy(Ejb3Deployer.java:182)
... 33 more
Exception in thread "CompilerThread0" java.lang.OutOfMemoryError: requested 1229
92 bytes for Chunk::new. Out of swap space?
Press any key to continue . . .
As you can see this happens when our ear(37M) is deployed, any idea that we can overcome this? Thanks. -
8. Re: OutOfMemoryError in JBoss 5.01
alesj Sep 21, 2009 7:55 AM (in response to yongz)Make sure you're running in VFS forceCopy mode:
13:52:32,843 INFO [ZipEntryContext] VFS force nested jars copy-mode is enabled.
Otherwise nested jar files are handled in-memory,
which is very memory consuming. -
9. Re: OutOfMemoryError in JBoss 5.01
yongz Sep 21, 2009 8:20 AM (in response to yongz)Thanks for the prompt response again Ales.
Just to confirm:13:52:32,843 INFO [ZipEntryContext] VFS force nested jars copy-mode is enabled.
Does this message indicate we are NOT running in VFS forceCopy mode?Make sure you're running in VFS forceCopy mode
We just use the default JBoss VFC settings, and from JBoss doc:JBoss AS sets boss.vfs.forceCopy=true by default.
Could you please advise how I can confirm that we are running in VFS forceCopy mode -
10. Re: OutOfMemoryError in JBoss 5.01
alesj Sep 21, 2009 8:24 AM (in response to yongz)If you didn't change anything, then you're running in copy mode
- as indicated in JBoss docs (and your comment).
In this case you should see that log info msg in server's log/console. -
11. Re: OutOfMemoryError in JBoss 5.01
jaikiran Sep 21, 2009 8:31 AM (in response to yongz)Exception in thread "CompilerThread0" java.lang.OutOfMemoryError: requested 1229 92 bytes for Chunk::new. Out of swap space?
The "CompilerThread0" reference is strange. I haven't usually seen OOM being reported against the thread name.
Just googled for this and it appears to be a bug in Sun JDK:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6487381
http://www.danielschneller.com/2007/04/exception-in-compilerthread0.html
Check if there is a new update of the JDK available and try against it. -
12. Re: OutOfMemoryError in JBoss 5.01
yongz Sep 21, 2009 8:56 AM (in response to yongz)Thanks Ales/JAIKIRAN
In this case you should see that log info msg in server's log/console.
The stack trace I posted are from the server console, and the server logs has the same message.Check if there is a new update of the JDK available and try against it.
We are currently using jdk1.5.0_19, will try the latest jdk1.5.0_21.
However, it was OK when we ran jboss 4.2.2 on jdk1.5.0_16. -
13. Re: OutOfMemoryError in JBoss 5.01
yongz Sep 21, 2009 9:29 AM (in response to yongz)We are currently using jdk1.5.0_19, will try the latest jdk1.5.0_21.
I just did a test and same issue happened on jdk1.5.0_21.
Any suggestion that I can move forward? Thanks. -
14. Re: OutOfMemoryError in JBoss 5.01
jaikiran Sep 21, 2009 9:41 AM (in response to yongz)Edit the run.conf (or run.bat.conf depending on the OS you use) to add the -client to the JAVA_OPTS section:
JAVA_OPTS="-client ....//the rest of it"
and see if that works.