Probably the infamous JVM error, if files are not deleting then you are at least running 1.4.2. Heard a report that it may be fixed in 1.5, try it if you can and let me know.
Otherwise, check for file leaks in you app. For instance, we are tuning our app to use a static properties file instance variable... otherwise every time the class is instantiated a new file handle is opened and stays around for the life of the class (which in our instance, was for the entire time the user session was active).
FYI, since this is a JVM problem the same thing happens on all web servers... check out this link from BEA: