HornetQ Cluster : TTL configuration for static connectors

[suzith]

Wildfly Version : 8.2.0 Final

HornetQ Version : 2.3.0 (Embedded with Wildfly)

Issue description :

- I have setup HornetQ cluster (Active-Backup topology) over TCP/IP using static connectors. Cluster and failover functionality works correctly when the active node goes down(shutdown completely).

- The issue is with the Split Brain scenario during network disconnect. The problem is with two nodes active at the same time. I do not have option to shutdown/restart nodes manually.

Question :

- Is there a way to recover from split brain scenario ?

- Is there a way to delay failover mechanism and have backup node wait for "X" amount of time before it becomes active ? The "X" amount of time can overcome network disconnect, hence there is no need to failover.

- Is "X" interval configurable somewhere ?

- If yes, does it work well with TCP/IP using static connectors ?

Any help is appreciated.

[jbertram]

Is there a way to recover from split brain scenario ?

There's no automated way to recover from split brain.

Is there a way to delay failover mechanism and have backup node wait for "X" amount of time before it becomes active ? The "X" amount of time can overcome network disconnect, hence there is no need to failover.

I'm not aware of any way to delay failover.

There is some built-in protection against split-brain as noted in the documentation, but that protection doesn't apply to situations with just 2 nodes.  It's really up to you to ensure that the network connection between the live and backup is reliable.  If it's not reliable then you should increase the size of your cluster to mitigate the possibility of split brain.  If you can't do either of those then your best option is to use shared storage rather than replication.

[suzith]

Thanks Justin for your response.

As per my understanding, Split Brain situation is when both(in case of 2 node cluster) nodes are active.

- What is the behavior of the JMS clients(for e.g. remote MDBs) in case of two active nodes ? - which active node do they connect to ?

- would shared storage guarantee message delivery for subscribers connected to both active nodes ?

- any known limitations when used with TCP/IP static connectors?

[jbertram]

What is the behavior of the JMS clients(for e.g. remote MDBs) in case of two active nodes ? - which active node do they connect to ?

It depends on how the client is configured.

would shared storage guarantee message delivery for subscribers connected to both active nodes ?

Shared storage doesn't suffer from the same split-brain weakness.

any known limitations when used with TCP/IP static connectors?

I don't understand your question.  Any known limitations in respect to what and how do you define "limitation"?  Every implementation decision has pros and cons.  What might be a limitation in one scenario could be a feature in another.