3 Replies Latest reply on Sep 6, 2012 2:29 AM by davsclaus

    How to recover data after an unexpected shutdown ?

    jc_cap
    jc_cap Sep 4, 2012 5:35 AM

      Hello,

      The camel route I developped reads files having a variable size in an input directory, then splits them into unitary element to aggregate them into fixed intermediate files before processing data.

      My issu occurs when the server stops during splitting the input files because when I restart, I have lost data. To solve this issu, I added an agregation repository to my agregator but, at the end, I have more data than I am expected.

      Is there a strategy, a best practice to avoid losing data when you restart your application ?

      Is there a way to recover data after an unexpected shutdown during processing ?

      Thanks for your help.

      JC.

      • 80 Views
      • Tags:
        • 1. Re: How to recover data after an unexpected shutdown ?
          davsclaus
          davsclaus Sep 4, 2012 7:28 AM (in response to jc_cap)

          Its probably easer to just restart with the input file again if the aggregation was not fully completed.

           

          So during aggregation store in a temporary file. And then when fully done, then rename/move the file to its intended location.

            • Actions
          • 2. Re: How to recover data after an unexpected shutdown ?
            jc_cap
            jc_cap Sep 5, 2012 10:43 AM (in response to davsclaus)

            Hello,

            thanks for your response.

            I tried your solution, but sometimes I still have more data than required.

            As you said, I store input files in a temporary directory just before they are splitted, and as soon as the aggregation is completed, I remove these files and put them into an history directory.

            When an unexpected stop occurs, some files can be in this temporary directory because the aggregation is not complete. So, before the route starts again, I move these files into the input directory and sometimes I have more data at the end.

             

            Best regards.

            JC.

              • Actions
            • 3. Re: How to recover data after an unexpected shutdown ?
              davsclaus
              davsclaus Sep 6, 2012 2:29 AM (in response to jc_cap)

              When you startup you should delete any temporary files, as they are not complete.

                • Actions