-
15. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
That helps as far as knowing what (in a general sense) VFS is trying to accomplish.
These fixes, with updating VFS and changing the vfs.xml file, is that something I'll need to wait until 5.0.1 to get that or is there some workaround to use in the meantime.
Right now I'm looking into writing some sort of script to run with the Windows Script Host to delete the files from the vfs temp folder periodically. That seems like a bit of a hack way to do it (and I've never used the scripting host before either) but if it keeps the development servers from crashing every other build then ... so be it.
Thanks for your work on this though. I'm trying not to sound too impatient but since I took the initiative to upgrade to JBoss 5.0.0.GA, I've become the one responsible for making it work correctly and I don't currently know how to do that. -
16. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"bdean42" wrote:
These fixes, with updating VFS and changing the vfs.xml file, is that something I'll need to wait until 5.0.1 to get that or is there some workaround to use in the meantime.
No.
Once I commit the VFS change/fix,
you'll be able to use it - but it's gonna be a snapshot."bdean42" wrote:
Thanks for your work on this though. I'm trying not to sound too impatient but since I took the initiative to upgrade to JBoss 5.0.0.GA, I've become the one responsible for making it work correctly and I don't currently know how to do that.
NP.
Thank you for pointing this out,
and having the patience to explain what's going on.
I'll work on this tomorrow,
keeping you posted when I commit and do a few simple tests. -
17. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"alesj" wrote:
Once I commit the VFS change/fix,
you'll be able to use it - but it's gonna be a snapshot.
The initial version is in.
This is the new vfs.xml:<deployment xmlns="urn:jboss:bean-deployer:2.0"> <bean name="VFSCache"> <constructor factoryClass="org.jboss.virtual.spi.cache.VFSCacheFactory" factoryMethod="getInstance"> <!-- Use the CombinedVFSCache implementation --> <parameter>org.jboss.virtual.plugins.cache.CombinedVFSCache</parameter> </constructor> <start ignored="true"/> <property name="permanentRoots"> <map keyClass="java.net.URL" valueClass="org.jboss.virtual.spi.ExceptionHandler"> <entry> <key>${jboss.lib.url}</key> <value><null/></value> </entry> <entry> <key>${jboss.common.lib.url}</key> <value><inject bean="VfsNamesExceptionHandler"/></value> </entry> <entry> <key>${jboss.server.lib.url}</key> <value><inject bean="VfsNamesExceptionHandler"/></value> </entry> <entry> <key>${jboss.server.home.url}deploy</key> <value><inject bean="VfsNamesExceptionHandler"/></value> </entry> </map> </property> <property name="realCache"> <bean class="org.jboss.virtual.plugins.cache.IterableTimedVFSCache"/> </property> </bean> <bean name="VfsNamesExceptionHandler" class="org.jboss.virtual.plugins.context.helpers.NamesExceptionHandler"> <constructor> <parameter class="java.lang.String">sqljdbc.jar</parameter> </constructor> </bean> </deployment>
And you need to build the latest VFS trunk:
- http://anonsvn.jboss.org/repos/jbossas/projects/vfs/trunk/
If you use maven, I already deployed the snapshot to our snapshot mvn repo.
Try it and please let me know how it goes. :-) -
18. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"alesj" wrote:
And you need to build the latest VFS trunk:
I attached it to JIRA issue:
- https://jira.jboss.org/jira/browse/JBAS-6373 -
19. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
I've applied the changes and currently it still looks like when we redeploy the war and it extracts all those jar files, they stick around in the vfs-nested.tmp folder even after being undeployed or redeployed.
I've run the build that uses this server a few times and that vfs-nested.tmp folder has about 865 files in it (each of our wars contains around 100 jars). It's only about 150 mb so the disk isn't anywhere near filling up but I'm not convinced that it won't.
What is supposed to happen to these temporary jar files? Is there something that will trigger them being deleted? Some sort of cleanup thread that looks to see which files are open and deletes the others?
I'll keep a close eye on this today and see if I can tell what is going on but for now it looks like the files are still there. -
20. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"bdean42" wrote:
What is supposed to happen to these temporary jar files?
They are there to stay until you restart your app server.
- it's the usage of File.deleteOnExit()
They are referenced by VFS as they represent nested jars for VFS to handle."bdean42" wrote:
Is there something that will trigger them being deleted?
Some sort of cleanup thread that looks to see which files are open and deletes the others?
Currently no. It's a lot similar to what GC does.
We would have to keep a track of who's using what.
And even then it wouldn't perform well if you eventually soon after non-usage used it again - which leads to another temp/un-packaging.
But we had this temp usage before, with previous JBossAS, where every nested stuff was temped by default.
It was just that every url used directly referenced that temp resource.
e.g. classloaders, deployers, ...
Here with VFS, we hide the url usage/impl details behind VFS spi/api,
where it doesn't actually mean we'll resolve to use of the temp copy."bdean42" wrote:
I'll keep a close eye on this today and see if I can tell what is going on but for now it looks like the files are still there.
Let me know how it goes. -
21. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"bdean42" wrote:
so the disk isn't anywhere near filling up but I'm not convinced that it won't.
It shouldn't, as the most commont roots are permanently there.
But if you use some path outside those roots,
either add that root to permanent roots or configure some other (non-timed) type of real cache.
But if the disk starts to fill up again, let me know asap.
(although in that case I would be a bit out of ideas atm :-() -
22. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
It definitely is still a problem. After a few redeploys this morning that vfs-nested.tmp folder was up to 3000+ files and around 800 mb.
"alesj" wrote:
They are there to stay until you restart your app server.
- it's the usage of File.deleteOnExit()
This doesn't seem like a solution to me. While these are just development servers and not something in production, they get deployed to ... potentially hundreds of times a day, at least often enough that the vfs-nested.tmp folder *will* fill up the disk and bring the server to a grinding halt left unchecked.
But simply restarting the app server periodically isn't a solution. In fact one of the features of JBoss is that it supports hot redeploys. If these redeploys cause the hard drive to fill up with temp files, even within a single 8hr work day, I honestly don't think JBoss should claim to support redeploying war files. We shouldn't have to restart the app server.
In JBoss 4 I could set up a loop on a command prompt that just touched the deployed war file and slept for 40 seconds. I was using this to check that we could redeploy our war over and over with no problems. I found that there was really no limit to how many times we could redeploy (once I redeployed about 5000 times in a row with restarting JBoss). With this vfs-nested.tmp folder growing uncontrollably, there is no way I could run this same experiment in JBoss 5. -
23. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
I know this is a naive suggestion but instead of using deleteOnExit, would there be a way to have the thing that deploys/undeploys war files somehow register with vfs. Then maybe vfs could maintain some sort of map of deployments to files they use so that when they are undeployed, it actively deletes the files rather than passively waiting for the app server to exit, which could potentially never happen (at least we hope we never have to shutdown or restart the app server).
-
24. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
OK, I see your point.
I'll look into how we used to do this,
so you could re-run this 5000 times.
If it was possible before, I'm sure it must be even easier now,
as we have a single place where we keep the temp url/link.
I'll get back to you on this asap. -
25. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
I just ran a little experiment with redeploying repeatedly. After 31 redeployments of a simple war file, the deployment fails with the following exception:
09:07:32,551 WARN [HDScanner] Scan failed org.jboss.deployers.spi.DeploymentException: Exception determining structure: AbstractVFSDeployment(sample-project.war) at org.jboss.deployers.spi.DeploymentException.rethrowAsDeploymentException(DeploymentException.java:49) at org.jboss.deployers.structure.spi.helpers.AbstractStructuralDeployers.determineStructure(AbstractStructuralDeployers.java:85) at org.jboss.deployers.plugins.main.MainDeployerImpl.determineStructure(MainDeployerImpl.java:853) at org.jboss.deployers.plugins.main.MainDeployerImpl.addDeployment(MainDeployerImpl.java:303) at org.jboss.deployers.plugins.main.MainDeployerImpl.addDeployment(MainDeployerImpl.java:260) at org.jboss.system.server.profileservice.hotdeploy.HDScanner.scan(HDScanner.java:270) at org.jboss.system.server.profileservice.hotdeploy.HDScanner.run(HDScanner.java:221) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Arrays.java:2882) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390) at java.lang.StringBuilder.append(StringBuilder.java:119) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.initPath(AbstractVirtualFileHandler.java:310) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.initPath(AbstractVirtualFileHandler.java:303) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.initPath(AbstractVirtualFileHandler.java:303) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.initPath(AbstractVirtualFileHandler.java:303) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.getPathName(AbstractVirtualFileHandler.java:207) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.getLocalPathName(AbstractVirtualFileHandler.java:234) at org.jboss.virtual.plugins.context.zip.ZipEntryContext.initEntries(ZipEntryContext.java:533) at org.jboss.virtual.plugins.context.zip.ZipEntryContext.ensureEntries(ZipEntryContext.java:554) at org.jboss.virtual.plugins.context.zip.ZipEntryContext.checkIfModified(ZipEntryContext.java:693) at org.jboss.virtual.plugins.context.zip.ZipEntryContext.getChild(ZipEntryContext.java:737) at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.createChildHandler(ZipEntryHandler.java:175) at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.structuredFindChild(AbstractVirtualFileHandler.java:542) at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.getChild(ZipEntryHandler.java:155) at org.jboss.virtual.plugins.context.DelegatingHandler.getChild(DelegatingHandler.java:106) at org.jboss.virtual.VirtualFile.getChild(VirtualFile.java:451) at org.jboss.virtual.VFSUtils.getManifest(VFSUtils.java:266) at org.jboss.virtual.VFSUtils.addManifestLocations(VFSUtils.java:178) at org.jboss.deployers.vfs.spi.structure.helpers.AbstractStructureDeployer.addClassPath(AbstractStructureDeployer.java:239) at org.jboss.web.deployers.WARStructure.determineStructure(WARStructure.java:202) at org.jboss.deployers.vfs.plugins.structure.StructureDeployerWrapper.determineStructure(StructureDeployerWrapper.java:73) at org.jboss.deployers.vfs.plugins.structure.VFSStructuralDeployersImpl.doDetermineStructure(VFSStructuralDeployersImpl.java:196) at org.jboss.deployers.vfs.plugins.structure.VFSStructuralDeployersImpl.determineStructure(VFSStructuralDeployersImpl.java:221) at org.jboss.deployers.structure.spi.helpers.AbstractStructuralDeployers.determineStructure(AbstractStructuralDeployers.java:77) at org.jboss.deployers.plugins.main.MainDeployerImpl.determineStructure(MainDeployerImpl.java:853) at org.jboss.deployers.plugins.main.MainDeployerImpl.addDeployment(MainDeployerImpl.java:303) at org.jboss.deployers.plugins.main.MainDeployerImpl.addDeployment(MainDeployerImpl.java:260) at org.jboss.system.server.profileservice.hotdeploy.HDScanner.scan(HDScanner.java:270) at org.jboss.system.server.profileservice.hotdeploy.HDScanner.run(HDScanner.java:221)
I don't know if that's useful at all or even related. That may be a different issue. -
26. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
What is the status of the fix for this? I see there are several JIRA entries for JBoss AS 5.0.1.GA that are blockers on this issue. I agree 5.0.1 should not be released without a solution for this.
At the same time I don't see any activity in the VFS JIRA and there is not even any versions listed besides a 2.0.1 Beta.
I would really like to start suggesting that clients start moving to JBoss 5. This is critical enough of an issue that I am not going to do that right now because the only version available has this serious flaw.
It would be nice to get some more context for when 5.0.1 might be released and we can actually start using JBoss 5. -
27. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"stuart_wildcat" wrote:
At the same time I don't see any activity in the VFS JIRA and there is not even any versions listed besides a 2.0.1 Beta.
I would really like to start suggesting that clients start moving to JBoss 5. This is critical enough of an issue that I am not going to do that right now because the only version available has this serious flaw.
It would be nice to get some more context for when 5.0.1 might be released and we can actually start using JBoss 5.
Hold your horses. ;-)
We're on it:
- http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4206010#4206010
But as you can see, we've hit another hurdle - how to re-use temps.
Although I'm confident I'll have a proper solution till the end of this week,
releasing 5.0.1 should then be just a matter of QA's formality. -
28. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
Thanks for the link to the other forum topic. I figured there was plenty of work going on with this but looking at JIRA or this forum thread didn't show what was happening.
Will the JIRA list for VFS 2.0.1 beta be cleared before upgrading the VFS included in JBoss 5.0.1 GA? If so, watching that list will give an idea of progress. -
29. Re: -Djboss.vfs.forceNoCopy=true not working AS 5.0.0.GA
"stuart_wildcat" wrote:
Will the JIRA list for VFS 2.0.1 beta be cleared before upgrading the VFS included in JBoss 5.0.1 GA?
Sure.
I'll also properly input the ongoing features.
It would be ideal if I could do this in advance,
but there is too much re-factoring going on,
hence it almost impossible to put a stamp on what's gonna
stay in or what's gonna change.