11 Replies Latest reply on Mar 18, 2010 8:14 AM by srinivaas venkat

    Cluster message redistribution / lost messages

    srinivaas venkat Novice

      HI,

      I have an issue with Hornetq2.0.0.GA and JBoss 5.1.0 GA where the messages are not picked sometimes by the consumer even though the redistribution is enabled. Sometimes the messages are lost when there are no consumers. This issue happens only in Linux but not in Windows.

       

      I have a queue cluster of two nodes nodeq1 and nodeq2. I have another cluster of two nodes nodeb1 and nodeb2 where I have my MDBs deployed.

       

      Steps to replicate the issue

      Started both the queue nodes nodeq1 and nodeq2
      Started one consumer node nodeb2 for MDB2 to consume from nodeq2
      Sent 4 messages and all got consumed by MDB2 from nodeb2

       

      Started nodeb1 also for MDB1 to consume from nodeq1 (and anyway MDB2 should consume from nodeq2)
      Sent 4 messages and two got consumed by MDB1 and two by MDB2

       

      Stopped nodeb1
      Sent 4 messages and all 4 got consumed by MDB2

       

      Stopped nodeb2 as well
      Sent 4 messages, and I encountered 3 different scenarios

       

      Scenario1: Sometimes all 4 messages were sent to nodeq2  and all remained unconsumed
      Scenario2: Sometimes 2 messages were sent to nodeq1 and two messages to nodeq2 and all remained unconsumed
      Scenario3: Sometimes 4 messages were sent to nodeq1 and ALSO 4 messages to nodeq2 and all of them were shown as consumed in jmx console even though there were no consumers to pick the message in both the queue nodes. Also the below exception was reported in both the queue logs for Scenario3 alone

       

      2010-02-18 06:12:51,113 WARN  [org.hornetq.core.client.impl.FailoverManagerImpl] (Thread-8 (group:HornetQ-client-global-threads-717964854)) Failed to connect to server.
      2010-02-18 06:12:59,251 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (New I/O server worker #1-2) Duplicate message detected - message will not be routed
      2010-02-18 06:12:59,859 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (New I/O server worker #1-2) Duplicate message detected - message will not be routed
      2010-02-18 06:13:00,663 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (New I/O server worker #1-2) Duplicate message detected - message will not be routed
      2010-02-18 06:13:01,488 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (New I/O server worker #1-2) Duplicate message detected - message will not be routed

       

      Again Started nodeb2 for MDB2 to consume from nodeq2
      Sent 4 messages and two were sent to nodeq1 and two to nodeq2. Only two messages from nodeq2 were consumed by MDB2 and remaning 4 messages in nodeq1 remanined unconsumed even though the redistubution is enabled in both the nodeq1 and nodeq2. (We have set forward-when-no-consumers to false and redistribution-delay to 0 in both the nodes)

       

      I've also attached the xmls for all 4 nodes. pls help