Jboss ESB - FS provider strange issue

[srufus]

Hi,
We are using the Jboss ESB's FS provider in our application.Everytime a file is put in the folder that
the FS provider is listening, it triggers the action pipleline.
This was working fine till we were using the JBoss ESB community edition(version 4.3)
We moved to the JBoss SOA pack which uses ESB version 4.4.

The issue we are facing is this.

1.FileA arrives in the folder and gets processed.FileA is moved into a "processed" directory with a suffix
FileA.esbDone
2.Another FileA arrives in the folder after some time. Ideally this new file should get processed and moved into
"processed" folder and will overwrite the FileA.esbDone.But the old FileA.esbDone doesnot get overwritten and the new FileA remains as FileA.esbInProcess in
the initial folder.

In ESB 4.3 the file was getting overwritten.

<fs-provider name="TestFileProvider">
 <fs-bus busid="testFileChannel">
 <fs-message-filter directory="D:\Test"
 input-suffix=".xml" work-suffix=".esbInProcess" post-delete="false"
 post-directory="D:\Test\processed" post-suffix=".esbDone"
 error-delete="false" error-directory="D:\Test\error"
 error-suffix=".esbERROR" />
</fs-bus>

Is this a problem with ESB 4.4 or am I missing something in the configuration ?


Thanks in advance
srufus

[beve]

Hi,

Is this a problem with ESB 4.4 or am I missing something in the configuration ?

Yes there seems to be a change in the code and I've been able to reproduce this issue. I'll look into this and hopefully post back with a solution.
Can you confirm that you are seeing the following in the server.log:

2009-02-13 02:52:28,986 DEBUG [org.jboss.soa.esb.util.FileUtil] Unable to rename file '/opt/jboss/esb/trunk/product/samples/quickstarts/helloworld_file_action/build/dirs/input/test.dat.esbInProcess' to '/opt/jboss/esb/trunk/product/samples/quickstarts/helloworld_file_action/build/dirs/output/test.dat.esbDone'. '/opt/jboss/esb/trunk/product/samples/quickstarts/helloworld_file_action/build/dirs/output/test.dat.esbDone' already exist.

Apologies for this.

Regards,

/Daniel

[srufus]

Hi Daniel,
This is the log statement I see on my server log :

2009-02-13 10:07:20,282 DEBUG [org.jboss.soa.esb.util.FileUtil] Unable to rename file 'C:\abc\FileA.xml.esbInProcess' to 'C:\abc\processed\FileA.xml.esbDone'. 'C:\abc\processed\FileA.xml.esbDone' already exist.
2009-02-13 10:07:20,282 DEBUG [org.jboss.soa.esb.util.FileUtil] Unable to rename file

Please let me know how to resolve this.

Thanks in advance,
srufus

[beve]

I've created a jira for this here: http://jira.jboss.org/jira/browse/JBESB-2401

About the options you have for this would be:
1. If you have a support contract you can request a patch which will be provided for you.
2. Update the jbossesb-rosetta.jar with the version of org.jboss.soa.esb.util.FileUtil.class from 4.3.

Regards,

/Daniel

[srufus]

Thanks Daniel !

srufus

[sem]

Is it possible instead of rewriting FileA.esbInProcess in process directory configure JBossESB to create file like FileA.esbInProcess.1 to keep track of all the files processed by ESB?
thanks

[srufus]

Hi Daniel,
I have replace the FileUtil.class in jbossesb-rosetta.jar but I still face the issue.
Are there any other files which need replacement too ?

Thanks in advance,
srufus

[beve]

Hi,

Are there any other files which need replacement too ?

no that is the only file that I updated when I tested this. If you send me your jbossesb-rosetta.jar I'll take a look.

Regards,

/Daniel

[jbossisgr8]

I think this is a bug which creeped in because of

https://jira.jboss.org/jira/browse/SOA-1024

https://jira.jboss.org/jira/browse/JBESB-2079

And these talk about changes to "FileGatewayListener.renameFile()" where this method no longer delete the file for renaming.

This holds good in case of multiple instances of listeners looking at the same location, but if only one listener is looking at a destination then if the files are not deleted then this issue arises.

Is it possible to give a naming convention like ".esbdone.1" and ".esbdone.2" etc... though i doubt that

[beve]

Hi Guys.

I've reopened https://jira.jboss.org/jira/browse/JBESB-2079 to make sure that the issues here are addressed.

jbossisgr8: Thanks for point out those jiras. I'll double check this issue with Kevin when he gets back.

srufus: Did you have a chance to send me your jbossesb-rosetta.jar and I'll take a look.

sem: With 4.x I'm not sure if this is possible. I'll need to look into this. You can always create a feature request for this.

Thanks for your feeback on this!

Regards,

/Daniel

[srufus]

Hi Daniel,
I think I had tested with the wrong version of the jar.
The good news is that the jar which has the FileUtil from 4.3 is working fine.

Thanks a lot for your help.

Thanks,
srufus

[kconner]

There was never any intention to allow a copy to overwrite an existing file. The fact that it allowed it was a bug and checking was added to tighten this up.

Why is this something you need to use?

[kconner]

BTW The reason this was a bug is that it was possible to have multiple threads attempt to process the same message. Having all but one fail prevents this, especially across different VMs.

[mccloud]

Hi Kevin,

I can understand that this behavior (overwriting files in the output dir) was a bug. However, the current behavior is rather strange as well. I tried the helloworld file action quickstart, and this is what happens when adding the same file, say 'testFile.dat' to the input dir.

1) On the first attempt, the file gets picked up by JBoss-ESB, enters the action pipeline, file is renamed to 'testFile.dat.esbWorking' and the actions are processed. After processing, the file is moved to the post-directory and renamed to 'testFile.dat.sentToEsb'.
2) On the second attempt, the 'testFile.dat' gets picked up by th ESB, enters the action pipeline, the file is renamed to 'testFile.data.esbWorking' and the actions are processed. After processing, the ESB tries to move the file to the output dir, but is unable to do so because this would overwrite the file created in our previous step. So the file remains in the input directory as 'testFile.dat.esbWorking'.
3) On the third attempt, 'testFile.dat' isn't picked up anymore because there is still a 'testFile.dat.esbWorking' present in the input dir from our second attempt.

What I don't understand is why:
a) I don't see an error in my error log on the second attempt.
b) On the second attempt, the actions get processed although it will be impossible to move the message to the output dir.

I do understand why it happens, but I don't think that this is desired behavior. I would expect that either the message doesn't get processed at all on the second attempt or that, after processing, it wouldn't block the next file. So, either support processing multiple files with the same name, or don't support it. Currently actions get processed on the first and second attempt.