HornetQ (2.3.20 Final) Message Queue running on Jboss-AS in AWS clustered and multiple regions

[steve.jamieson]

Hi,

We are currently performing a setup using hornetq as a remote message queue hosted in AWS. There is only one queue end point used and the consumers are local to the region on another host.

To achieve HA and DR we are looking to run the message queues in a cluster, and curious if anyone else has performed this type of setup. Reading around I have noticed some setups that include a HA , however most of these appear to reference what is similar to a single region setup

Region A will be the primary site (With 2 queue servers, one in each availability zone)

Region B will be the secondary dr site (With 2 queue servers, one in each availability zone)

From the documentation I have been reading all 4 servers would need to be in the same cluster, some servers are configured to be live servers and other servers being configured to be backup servers.

Some of the challenges in the setup are

• Region B is designed to be a DR site and not process requests unless Region A is unable to serve requests
• Messages should be replicated to Region B from Region A in the event that Region A is lost, minimal message loss will occur
• The consumers at Region A should not connect to Region B and vice versa, The consumers at Region B should not connect to Region A

So far the proposed solution would be both message queue servers in Region A in the cluster in a live role, the message queue servers in Region B configured to be a backup server in the cluster

The question would be has anyone done a setup similar to this or have advice on how to approach the desired outcome ?

Thanks,

Steve

[clebert.suconic]

I - Why you don't use Artemis 1.1... it's much better already.

II - some high profile users I know were not using failover at all as they were just trusting the storage and monitoring to restart the instance as soon as something died, and have the clients to reconnect.

[jbertram]

DR functionality isn't specifically supported by HornetQ.  However, you if Region A and Region B have a reliable network connection between them then I think you could do what you want with a 4 node cluster (2 live nodes in Region A, 2 backup nodes in Region B just as you've described).  There are a couple of key issues that I see at this point:

1. Ensure the consumers in each region do not use a connection factory with HA = true so that when fail-over occurs the consumers don't connect to the servers in the other region.
2. If only one of the live nodes in Region A fail then you would end up with a node in Region A up and a node in Region B up. I'm not sure how to mitigate that one.

At the end of the day I think you should look at Apache ActiveMQ Artemis. The HornetQ code-base was donated to Apache ActiveMQ a little over a year ago now and is continuing it's life as ActiveMQ Artemis. No further work will be done on HornetQ.

[steve.jamieson]

Hi Clebert,

We will be looking at changing our messaging architecture in the future, however this needs to be phased in due to timelines,

Thanks for the suggestion, will have a review.

[steve.jamieson]

Hi Justin,

Thanks for the feedback and suggestions, after some further testing have found the application does not like connect to a backup server.

As for the message queue solution overall we will be assessing other alternatives, however needs to be worked into the current project and roadmap.

Cheers,

Steve

[jbertram]

...after some further testing have found the application does not like connect to a backup server.

What do you mean, exactly?