-
1. Re: Problem of memory with file upload
theute Aug 2, 2004 12:13 PM (in response to sysuser1)I do have a similar problem, i will try to debug that as i need it for the files attachment in the forum.
Thanks for letting us know and confirm that bug (it's in the sourceforge bugtracker as well) -
2. Re: Problem of memory with file upload
theute Aug 2, 2004 5:38 PM (in response to sysuser1)I found the problem, it's in the apache library commons-upload.jar
As we use the last release, the bug is not fixed, so i did a quick and (not so) dirty fix so we can upload files without totally smashing the machine.
You can get the new jar file here.
Just replace the one in your $NUKES_HOME/thirdparty/apache-commons/lib by the one i am providing and let me know if you get some improvement. -
3. Re: Problem of memory with file upload
theute Aug 2, 2004 5:45 PM (in response to sysuser1)commons-fileupload.jar from Apache of course (not commons-upload.jar as i wrote)
(I wish we were allowed to edit our posts...) -
4. Re: Problem of memory with file upload
sysuser1 Aug 3, 2004 4:30 AM (in response to sysuser1)Thanks for your help Thomas but the new library commons-fileupload.jar doesn't resolve my problem. No improvement for me.
And I don't understand because in the module html of Nukes, upload of file works fine...
You get some improvement with your commons-fileupload library? -
5. Re: Problem of memory with file upload
theute Aug 3, 2004 4:49 AM (in response to sysuser1)I do have a great increase, the module HTML is my test by the way, it allocates way to much memory for a form.
I used jprofiler to track the memory and realized that the Apache library was using many times big amount of memory, looking into the source code i saw that for each field of the form (any id or textform or file ...) it was allocating the size of the threshold !
(the threshold indicates from what size should the library store the fields in a file or in memory)
Our threshold is pretty high as we don't want the files to be written on the disk, so it was allocating way to much memory. For example when i was submitting a 50KB file into the HTML module, i got about 55MB increase in memory it's enough to make my computer swap a lot.
After the quick fix i still have a 20MB increase (much better, still a lot though).
If you have many fields in your form (and i repeat ANY field) the memory allocated will be way too much.
After the fixing i looked some more and found a guy complaining about the same thing and proposing a patch but Apache guys didn't want to use it...
I forgot to tell precisely the source change here it is:
In DeferredFileOutputStream there is this method:
public DeferredFileOutputStream(int threshold, File outputFile)
{
super(threshold);
this.outputFile = outputFile;
memoryOutputStream = new ByteArrayOutputStream(threshold);
currentOutputStream = memoryOutputStream;
}
That's where it allocates 'threshold' bytes whatever the size of the field, i just removed threshold so Java just gives 32Bytes and allocate more only if needed. -
6. Re: Problem of memory with file upload
sysuser1 Aug 3, 2004 5:53 AM (in response to sysuser1)Indeed, the module html have the same problem.
In my form I have 6 file fields and 6 text fields ! It's a lot !!
The unique solution for me it's to limit the number of my fields, isn't it?
Because when I use the library you have fixed, the problem is not solved for me.
I would to thank you for your help ! -
7. Re: Problem of memory with file upload
theute Aug 3, 2004 6:14 AM (in response to sysuser1)I am surprised you have no improvement, did you really recompile using the new library ? (do a build clean first)
Yes in that case 12 fields in a lot :)
You can also try to decrease the max size of a file, i think by default it's 4 MB. -
8. Re: Problem of memory with file upload
sysuser1 Aug 5, 2004 10:43 AM (in response to sysuser1)Ok, I had, yesterday, recompile the new library (indeed, I had forgotten to recompile it). ;)
It seemed work fine...
But I am surprised because today it doesn't work. :-( Indeed, no improvement in the use of memory. I don't understand.
Can you tell me how to decrease the max size of a file.
I do already in my form<input type=\"hidden\" name=\"MAX_FILE_SIZE\" value=\"100000\"/>
Thanks a lot in advanced ! :) -
9. Re: Problem of memory with file upload
sysuser1 Aug 5, 2004 11:50 AM (in response to sysuser1)It's ok, it work fine again...
Pfffff... It's a hard fight beetween me and my computer !! ;)
But I have always my problem when I want to update a file in my database.
See http://www.jboss.org/index.html?module=bb&op=viewtopic&t=52753
If someone can help me it will be very friendly...
Thanks !
Sandra