4 Replies Latest reply on Apr 5, 2016 12:50 PM by anupshandilya

    Does Hornetq provide sychronous data replication mechanism from live server onto backup server in HA?

    anupshandilya
    anupshandilya Apr 3, 2016 1:13 PM

      Hi,

       

      We have Hornetq servers set up in HA mode with data replication mechanism in place.

      In this, we have a live server and a corresponding backup server on two different nodes(physical servers).

       

      On the producer and consumer side, we have transaction enabled for each message sent and consumed.

       

      When a producer writes a message to live server with transaction enabled, it appears the transaction is in place till the live hornetq server writes to its memory and makes it durable(onto disk).Some parallel thread copies this message to backup server

      The questions are as below.

      a. Live is trying to copy this message to backup but goes down. This would mean loss of message. Is this understanding correct ?

      b. If 'a' is true, is there an option in hornetq to make the make the call synchronous till the message gets onto backup server

       

      Please take a look and kindly answer.

      • 194 Views
      • Tags:
        • 1. Re: Does Hornetq provide sychronous data replication mechanism from live server onto backup server in HA?
          jbertram
          jbertram Apr 4, 2016 4:18 PM (in response to anupshandilya)

          Live is trying to copy this message to backup but goes down. This would mean loss of message. Is this understanding correct ?

          That understanding is not quite correct.  The broker will confirm (via callbacks) that the data has been written both to the replica and to local storage before it returns a confirmation back to the client that the transaction was committed successfully.

            • Actions
          • 2. Re: Does Hornetq provide sychronous data replication mechanism from live server onto backup server in HA?
            anupshandilya
            anupshandilya Apr 5, 2016 1:53 AM (in response to jbertram)

            Thanks a lot Justin for answering.

            The speed of data replication is very good. We are liking it.

            Based on the above answer, I have few more follow up questions.

            We want to augment our understanding on the behaviour of clustering mechanism with respect to data replication.

             

            We have got transaction enabled on producer and consumer side that write messages to queue.

            The acknowledgement on producer and consumer side is AUTO_ACKNOWLEDGE. For us, information in messages is vital. Hence, we are doing acknowledgement on each message.

             

            Here is the scenario and the follow up question :

            1. Transaction is in place on both producer and consumer side

            2. Data inserted on local live server queue is guaranteed by broker to be copied to backup server. Otherwise, transaction would not succeed.

            3. On client side, we have static configuration of live and backup server.

            4. Network connection is good(reliable) between live and backup

            5. Messages are being continuously being inserted into queue and consumed over time

            6. Fail Over is performed ( live to backup  by a clean shut down of the live server).

             

            Do you see any stray circumstances where hornetq might loose few messages( to copy onto backup and make it available to consumer ) or it should be good without any loss( we assume all above settings 1-4 are good and working fine ).

             

            We are trying to test and benchmark the solution. Hence, your answers would be valuable for our decision making.

              • Actions
            • 3. Re: Does Hornetq provide sychronous data replication mechanism from live server onto backup server in HA?
              jbertram
              jbertram Apr 5, 2016 11:56 AM (in response to anupshandilya)

              Do you see any stray circumstances where hornetq might loose few messages( to copy onto backup and make it available to consumer ) or it should be good without any loss( we assume all above settings 1-4 are good and working fine ).

              The whole aim of replication is to maintain data integrity.  Furthermore, transactions are the best way a client can ensure data integrity.  Combining these two (data replication with client transactions) is the most sure way to maintain data integrity.  Bugs are always possible, but if everything works the way it was designed to work then I don't see any circumstances where message loss could occur.

                • Actions
              • 4. Re: Does Hornetq provide sychronous data replication mechanism from live server onto backup server in HA?
                anupshandilya
                anupshandilya Apr 5, 2016 12:50 PM (in response to jbertram)

                Thank you Justin.

                We will continue the tests and let you know how it went.

                 

                Have a good day

                  • Actions