How to recover data after an unexpected shutdown ?

[jc_cap]

Hello,

The camel route I developped reads files having a variable size in an input directory, then splits them into unitary element to aggregate them into fixed intermediate files before processing data.

My issu occurs when the server stops during splitting the input files because when I restart, I have lost data. To solve this issu, I added an agregation repository to my agregator but, at the end, I have more data than I am expected.

Is there a strategy, a best practice to avoid losing data when you restart your application ?

Is there a way to recover data after an unexpected shutdown during processing ?

Thanks for your help.

JC.

[davsclaus]

Its probably easer to just restart with the input file again if the aggregation was not fully completed.

So during aggregation store in a temporary file. And then when fully done, then rename/move the file to its intended location.

[jc_cap]

Hello,

thanks for your response.

I tried your solution, but sometimes I still have more data than required.

As you said, I store input files in a temporary directory just before they are splitted, and as soon as the aggregation is completed, I remove these files and put them into an history directory.

When an unexpected stop occurs, some files can be in this temporary directory because the aggregation is not complete. So, before the route starts again, I move these files into the input directory and sometimes I have more data at the end.

Best regards.

JC.

[davsclaus]

When you startup you should delete any temporary files, as they are not complete.