Poll a directory, validate & FTP out. Loop this process over 10 directories

[donaldesb]

Hi, I am doing a test project using servicemix 4 to Poll a directory, validate the files inside, get the correct file & FTP out to the destination.

I need to loop this process over 10 directories. More directories will be added in future.

The FTP destination for the file in each directory is different.

The directory structure is like this example:

c:
dave\     contain dave.001 and dave.002 dave.003

c:
john\     contain john.001 and john.002 john.003

c:
kate\     contain kate.001 and kate.002 kate.003

The plan here is roughly as follow:

1.) Poll a directory

a. the directory name e.g. c:
dave\ is in a xml file where user can easily edit or add new.

b. validate to only get a file with extension 002 e.g. dave.002

c. FTP file to destination where the destination is also in the xml file e.g. dave's ip address

d. if FTP failed, send email warning to me.

2.) Poll the next directory whereby the directory name e.g. C:
john\ is retrieved in the same xml file and repeat step a, b, c & d

3.) Conitune to poll & FTP file in all 10 directories that are found in the xml files.

4.) If user add new directory & ip entry into xml, servicemix will be able to poll the newly added directory.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Is the above project feasible in servicemix 4?

How should I approach, design and create the project?

I have tried file poller and sender and found it difficult to loop through the 10 directories indicated in a xml file.

Another issue is that I am using windows platform for development but my servicemix is inside linux OS. What are the pitfalls to look out for?

[davsclaus]

Hi

I think Camel may be a good way to try to implement this.

For example you can implement a custom filter as a org.apache.camel.component.file.GenericFileFilter implementation, which reads that master XML file that contains the list of directories to use. Then you assign that filter to the ftp endpoint.

Then you know in the filter to return either true of false if the given file is within that master XML list of valid directories.

Then you can use recipient list EIP pattern to send the message to the FTP endpoint based on the IP address from the master XML list. You can use Java code to load the masterXML file and based on the file name from the message you can map it to the IP address.

And you can use error handling in Camel to catch exception and detour routing and send an email with some custom error message indicating the error.

http://camel.apache.org/file2

http://camel.apache.org/ftp2

http://camel.apache.org/enterprise-integration-patterns.html

http://camel.apache.org/error-handling-in-camel.html

Edited by: davsclaus on Sep 20, 2010 5:27 AM

[donaldesb]

Hi davsclaus,

Thanks for your guidance. I study & modify the hillert's example mentioned in camel webminar. I am left with exception handling and logging part.

Thanks a lot!

[jessicaleahy]

hi, I tried this one as well. I had a hard time with logging but I resolved it.

[donaldesb]

Hi jessica, i tried another approach for logging. i moved the camel project into servicemix. Took me some time to make the ftp and osgi deployment to work. Then i try splitting the existing servicemix log into various level. Finally creating the individual log file for each of the customers.

[jessicaleahy]

Hi donald,thank you for the information. When I run to this problem, which is I hope not to, I'll give your approach a try.

[jessicaleahy]

Do you have a code that you can recommend that i use?:)

Edited by: jessicaleahy on Oct 12, 2010 2:52 AM