Is Fuse ESB able to re-send or re-submit message to an endpoint in an integration route?

[michael.chong]

Hi,

     I'm completely new to Fuse ESB. I would like to know if it's possible to re-send or re-submit a message to an endpoint in an integration route even though there's no transaction rollback?

[bibryam]

Hi,

you can use an error handler with RedeliveryPolicy configured to re-submit messages. It can be DefaultErrorHandler or DeadLetterChannel or TransactionErrorHandler.

[michael.chong]

I presume that's provided on condition there's a transaction rollback? I'm more keen to know if Fuse ESB has some form of "checkpoint" or "synchronization point" by default or by feature where one could always re-submit a message from any point in the integration route or flow without being triggered by an error or transaction rollback, e.g.

A --> B --> C --> D

A message would go through integration points or processing steps A, B, C and D, and we could always go back to retrieve the message in whatever form it was at A, B, C or D and re-submit it if necessary. Is this possible?

[bibryam]

It doesn't require transaction rollback, but some kind of error to trigger the reply.

As per your case, you will have to do something custom, as each endpoint in the pipeline modifies the message and there is no mechanism OOTB to preserve the state of the message at different steps.

[davsclaus]

You can use JMS at A B C D to be able to persist the message. And then do Camel routes with from A ... to B and have these routes transactional. Then in case of errors then the TX rollback the message.

from A -> do something -> B

from B -> do something -> C

from C -> ...

[michael.chong]

Thanks, that's an interesting insight! However, as mentioned earlier, I'm completely new to Fuse ESB, so when you say "persisting the message using JMS at A, B, C, D", do you mean that the message be sent explicitly to a JMS queue at A, B, C and D? In my limited knowledge, I'm assuming that by default the underlying reliable mechanism/messaging being used by Fuse to route a message from A to B, B to C, and finally C to D in the integration route is Active MQ. If so, then I presume we could configure or turn on some kind of message persistence setting at each of the points A, B, C and D, thereby allowing the possibility of re-send or re-submitting from those points. Or am I wrong?

[davsclaus]

Yes you would use AMQ as the JMS to persist the message at the save points you want.

Though you would need to break out your route into individual routes, and not a single route. Then each route is separated and can safely do their own transaction.

Also mind Apache Camel which you are using for routing in JBoss Fuse is not a stateful workflow engine with a canonical message format, and therefore does not have a "persistence" out of the box. If that helps understand "the picture" about the A -> B -> C -> D in JBoss Fuse.

[davsclaus]

And if you do not need persistence, then Camel's built-in error handler can do redelivery at the point of error. Then you can do a single route that does A -> B -> C -> D. And if so for example there is an error with B -> C, then the error handler can resubmit (we call that redeliver) the message from that point.

There is some details here:

http://camel.apache.org/error-handling-in-camel.html

[michael.chong]

From your latest replies, it seems that Fuse:

1. Cannot re-deliver a message without error handling. I asked this question because I thought Fuse would have some message management tool where messages could be re-deliver or retrieve/recover from any point in a Camel route even though they're not erroneous?

2. A Camel route can only handle a single transaction? Can it handle more than 1 transaction?

3. A Camel route does not persist a message (i.e. it does not use AMQ?). In this case, how reliable is Fuse in ensuring there're no loss of messages when there's a system shutdown or failure?

[michael.chong]

BTW, is the dead letter queue a real queue where the failed or erroneous messages could be recover manually or in some way?

[davsclaus]

The DLQ in ActiveMQ is a real dead letter queue, and you can from the web console, select the message(s) from this DLQ and click a "resubmit" button.

The Camel DLQ is just any Camel endpoint. How to resubmit or handle those messages is a manual process. But if you tell Camel to use a JMS queue as DLQ, then you can also from the web console select message(s) from this queue and move them to another queue, which you can use as the manual process to resubmit the messages.

But with Camel DQL the endpoint could also be a file endpoint so you store the message to disk etc.

[davsclaus]

Ad 1)

Camel is flexible and you choose what you need and want to use.

If you go for HA and whatnot then you can use AMQ as the messaging infrastructure for that.

So if you want to persist at every step in the route, you would need to break it down into routes using from AMQ -> do something -> to AMQ.

Ad 2)

Yes a Camel route can be configured to be transactional or not. But you can process messages concurrently in a Camel route, and therefore have multiple in-flight transactions. You can link Camel routes together using direct endpoint and have them participate in the same transactions. Or start a new transaction using the propagation behaviors (requires_new) etc. Though these propagation behaviors is often just required that people use. Camel uses Spring TransactionManager and its API. There is a transaction guide in the user documentation for jboss fuse. And also covered in the two bigger Camel books (http://camel.apache.org/books)

Ad 3)

Use AMQ it has HA and reliable messaging.

[michael.chong]

Thanks for all the info but I'm a bit confused on how HA works with Fuse ESB.....are you saying that Fuse ESB is not HA without AMQ? And if we use AMQ as the underlying messaging infrastructure for Camel routes, do we have to define the endpoints of camel routes explicitly as JMS queues? Or we don't have to if message persistence is not required? In this case, will HA still works without message persistence?

[davsclaus]

AMQ is part of Fuse. So Fuse does HA using AMQ. In fact AMQ has always been the HA and reliable messaging of Fuse. And so is Apache ActiveMQ for Apache ServiceMix etc.

[michael.chong]

If Fuse does HA using AMQ, that tells me messages are persisted by default for HA, and the Camel routes should be using AMQ as the underlying messaging infrastructure to route messages from one point to another point (or endpoint) with rollback or recovery features, correct? Or am I still wrong?