4 Replies Latest reply on Sep 8, 2010 8:43 AM by belaban

    HornetQ Cluster in AWS

    schwanitzb
    schwanitzb Feb 22, 2010 3:42 PM

      Apparently, AWS does not support UDP multicast between machine instances (even those in the same security group). So this, I believe, rules out the discovery group approach to HornetQ clustering. AWS instances get IP addresses assigned from a very large pool and cannot be made static. So this, I believe, rules out specifying members of a cluster explicitly.

       

      Has anyone done any research or documentation on implementing a HornetQ cluster in the Cloud (AWS to be specific)?

       

      Thanks,

      Bill

        • 1. Re: HornetQ Cluster in AWS
          apwalker Feb 23, 2010 9:38 PM (in response to schwanitzb)
          I would suggest taking a look at the stormgrind project http://www.jboss.org/stormgrind as they have sovled a lot of the problems with running clusters and clustering in AWS environment.
            • Actions
          • 2. Re: HornetQ Cluster in AWS
            timfox
            timfox Feb 24, 2010 4:42 AM (in response to apwalker)

            IIRC The way JGroups/Infinispan solved this problem was by having a filesystem based "discovery" mechanism.


            Assuming each node has access to a shared file system - each node writes some information on it's address to a shared file(s) in a well known location when it starts up, other nodes then read that file(s) to "discover" what other nodes are in the cluster.

             

            We could do something similar in HQ. You could add a feature request if you like.

              • Actions
            • 3. Re: HornetQ Cluster in AWS
              schwanitzb
              schwanitzb Feb 26, 2010 12:44 PM (in response to timfox)

              Tim & Aaron,

               

              Thanks for your responses. I have created a feature request in JIRA: https://jira.jboss.org/jira/browse/HORNETQ-316

               

              Cheers,

              Bill

                • Actions
              • 4. Re: HornetQ Cluster in AWS
                belaban
                belaban Sep 8, 2010 8:43 AM (in response to timfox)

                JGroups has a few options regarding discovery on AWS:

                1. GossipRouter is an external lookup process (you can have multiple of those to prevent SPOFs), which is used for discovery only. After initial discovery, nodes talk to each other directly
                2. FILE_PING uses the file system to store and read information about cluster nodes. Note that a shared file system, like a Samba /CIFS or NFS mount is needed, unless you run all of your instances on the same local box
                3. S3: you can use an S3 bucket to do discovery (S3_PING)

                 

                 

                This code is pretty simple, and can be copied from JGroups

                  • Actions