JBoss/Tomcat deleting it's own config files

[timhague]

I'm having a problem related to very large amounts of deployment activity.

I'm running JBoss 3.0.2 Tomcat 4.0.4 on Windows 2000.

I'm testing an automated build process that deploys and undeploys 40 large ear files in quick succession in order to test each one. These contain a mixture of ejb jars and wars and range from a few kb to 12mb in size.

After some 38 of these have gone through fine I start getting messages in JBoss saying 'Too many files open' and deployment fails. If I then try to stop and restart JBoss I'm unable to do so because various xml files can't be found - standardjboss.xml, log4j.xml, web.xml. I then have to update my JBoss installation from the one we have in CVS to get it to work again!!

So two issues -
1. What is holding on to all the open files, so that eventually JBoss is running out? Is there something I can do increase the limit? Or is there anything I can do to force JBoss to release files on undepoyment?

2. Why are configuration files being deleted?

Any advise or assistance at all would be greatly appreciated!

[timhague]

An update on this - it appears to be a ZipException caused by having too many Zip files open - vis:

13:02:10,263 WARN [JARDeployer] operation failed; ignoring
java.util.zip.ZipException: Too many open files
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.(ZipFile.java:112)
at java.util.jar.JarFile.(JarFile.java:117)
at java.util.jar.JarFile.(JarFile.java:55)
at org.jboss.deployment.SubDeployerSupport.processNestedDeployments(SubDeployerSupport.java:200)
at org.jboss.deployment.SubDeployerSupport.init(SubDeployerSupport.java:127)
at org.jboss.deployment.MainDeployer.init(MainDeployer.java:668)
at org.jboss.deployment.MainDeployer.init(MainDeployer.java:686)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:604)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:580)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:564)
)

Does anyone know if this has been fixed in more recent versions of JBoss?

[jonlee]

Java imposes no real limitation on open files. However, the underlying operating system will impose restrictions. On *nix, you can use ulimit to increase this.

If your deployments are taking a long while, it may be possible that you have several deployers executing in parallel. You could increase the time between deployment scans to overcome this.

It is also possible that your OS is munching the files as the OS file interface and Java separate during the exception. I've only seen this in Windows with their file-locking work-around.

[timhague]

I've been keeping a track of the files in the tmp\deploy directory. I'm now attempting to manually delete all the files that I can between deployment - unfortunately I can't delete any archive files at all.

Attempting to manually delete these archives - even ones that have now been undeployed, results in 'The source or destination file may be in use' message - i.e. they are still being locked by JBoss.

I'm going to try upgrading to 3.2.1 to see if that actually releases archive file locks on undeployment!

Where can I look to find out what has changed/been fixed from one version of JBoss to the next?

[jonlee]

Go here for change notes:
http://sourceforge.net/docman/?group_id=22866

You'll probably find that the underlying problem is Windows and its file-sharing, file locking mechanism - I'm assuming that you are using Windows. Linux and the others usually don't care too much if you delete a file from under the program - even if it crashes the program. The problem probably is due to the file stream remaining open until the file stream object is reaped.

I haven't looked at the code around this area but I assume the objects linked to the temp files are no longer in scope but not garbage collected. You could try playing with GC and see if this gets things to release.