LRA coordinator can be scaled?

[silva.andreiacamila]

I'm going to use LRA coordinator in pair with apache camel in a Saga implementation.

I would like to know more about the scalability and failover of LRA.

[ochaloup]

Hi silva.andreiacamila,

thank you for asking. Unfortunately the LRA is still under development and we have not implemented the failover and scalability capabilities yet. Currently the Narayana LRA is a single node implementation where failover needs to be done manually.

Nevertheless we are aware of the importance of this functionality, we have this on our roadmap and we expect to address it soon. Just I can't tell you exact time frame for it.

Ondra

[zhfeng]

Hi,

you could take a look at camel/lra.adoc at master · apache/camel · GitHub and I think you could use the camel route exception handler and the redeliver strategy when doing the failover.

Amos

[rafaelcba]

ochaloup Can you please confirm or clarify if the current implementation is stateful (sagas log) in the way that even if the coordinator is restarted (when running in a POD on Kubernetes/Openshift) it can recover from the last persisted state?

What kind of storage mechanism does it use? Is there any plan to use any distributable persistent mechanism to store the sagas log (eg: distributed cache, data store, kafka)?

[mmusgrov]

Whenever the state of an LRA changes the new state is made persistent using the Narayana transaction object store logging mechanism. The store location is configurable via properties (Narayana Project Documentation). We have various storage options such as the file system, artemis journal, or a database.

Using distributed caches poses significant issues due to the problem of network partitioning.

[ochaloup]

rafaelcba hi Rafael. Thanks for keep asking just as Mike posted there is still no distributed persistent mechanism. We would like to have and I personally consider it as a bit an issue of the current implementation. It's on roadmap but I can't provide you any ETA. The focus is still on the LRA spec changes.

Regarding the Kubernetes/Openshift: if you switch the coordinator as the single-node app then you are right and you'll be safe. The Narayana provides 2 places where data can be stored. It's either at the file system - then you need to use the StatefulSet in the Kubernetes as it's the only (from my knowledge) setup which guarantees the persistent volume being mounted back in case of a crash/rescheduling/node changes plus it guarantees the hostname stability which is desired as well.

The second option is to configure the storage being placed in the database. You should be fine with any JDBC SQL storage. Then you don't need to have guarantees of the persistent volume to be mounted back to the same place. But you are dependent on the availability of the database storage.

Correct, if the coordinator is restarted and the prior storage is mounted then it just safely continue processing.

[rafaelcba]

Thank you, guys!

Great explanation! Good to know all these details.