Pointers: AggregationRepository (HawtDB) and Recovery

[rich.schlegelmilch]

I have been attempting to use the HawtDB as my AggregationRepository. 

Here is my scenario:

1. An aggregation is defined to group messages over a dynamic time-period using completionTimeout(header(TIMEOUT_VALUE).  CompletionTimeout is required due to time sensitive nature of the exchange.

2. Everything works as expected during the happy-path --

3. If I stop the process, and restart I see that there is an exchange in the repository, and non-in the repository-completed. 

     - I also see the the INFO: On start-up there are 1 aggregate exchanges (not completed) in the repository -- in my logs.

4. The aggregation does not complete, once I restart the process.

5. As I peruse the logs (set to TRACE), I don't see anywhere the exchanges are re-started. The exchanges have not failed,  they just haven't completed yet.

I'm I missing something?  Do I need to re-inject these non-complete exchanges? 

I search through the documentation and haven't stumbled upon what I  looking for -- Any pointers would be appreciated.  My aggregation spans a fairly lengthy period,  I don't want to loose the exchanges during maintenance cycles, etc.

Thanks,

Edited by: rich.schlegelmilch on May 31, 2011 10:10 PM

Edited by: rich.schlegelmilch on May 31, 2011 10:11 PM

[davsclaus]

Hi

Thats because you only use completion timeout right? Because upon restarting the timeout map is not re-created.

It really doesn't know how long time elapsed since the server crashed.

We could possible add an improvement/fix for this so upon restart Camel detects this corner case (only complete by timeout). And then re-create the timeout map, meaning the timeout will start from 0 again. The tricky part is that the timeout value can only be re-created if the timeout value was a fixed value. And in your case thats a dynamic based on a header.

Alternative we can add an option which tells Camel to always publish all the exchanges from the repository upon startup. In your case that seems maybe okay as you only complete upon timeout.

Finally we could consider adding information in the repository about the completion conditions, so that completionTimeout(header(TIMEOUT_VALUE) would be evaluated and its value stored in the repository. So if the header was 30000, then that value is stored. Then we can re-create the timeout map based on that value. The problem is that this requires additional fields to be stored in the repository and we have other stores which are JDBC based and that would require to add more columns in the SQL schema.

[rich.schlegelmilch]

Greetings,

You are correct that I am only using timeout.  I was thinking it was my completion criteria.

What are my current options?  Is there a way to "touch" the Exchange thus re-initializing the timeout map? 

I'm going to think about my completion criteria to see if I can force a completion.  My gotcha is that I don't know a quantity of messages that I will receive.

Thanks for the response,

Rich -

[davsclaus]

I have created a ticket to see if we can resolve this issue

https://issues.apache.org/jira/browse/CAMEL-4037

[rich.schlegelmilch]

I just grabbed the latest SNAPSHOT for 2.8, I'll do some testing and,  provide feedback.

Thank You,  kind sir!