0 Replies Latest reply on Dec 6, 2012 9:00 AM by crytek

    BAD HornetQ behavior when setting the "management-notification-address"

    crytek

      Hello,

       

      I've tried to run some tests after changing the default "management-notification-address" => "hornetq.notifications"  to other address and it seems that errors appear inside the logs and also HQ behaves abnormally when sending messages to topics.

       

      The scenario is the following:

       

      1. 2 nodes inside a cluster (node1 and node2)

      2. On each cluster node, inside the hq configuration file, create a jms topic destination: "jms.topic.notificationsTopic". 

      3. On each cluster node , change "management-notification-address" => "hornetq.notifications"  to "management-notification-address" => "jms.topic.notificationsTopic"

      4. Only on node1, create a jms topic destination named for e.g: "TestTopic1",  used for sending/consuming messages.

      5. Create a client example, which sends 100 messages to TestTopic1 on node1, and after that, consumes the messages using two subscribers, each with a different subscription:

       

      E.g.

       

                  Topic topic = (Topic) initialContext.lookup("jms/topic/TestTopic1");

       

                  connection = connFactory.createConnection(APPLICATIONREALM_USER,

                          APPLICATIONREALM_USER_PWD);

                  connection.setClientID("durable-client");

       

                  TopicSession session = (TopicSession) connection.createSession(false,

                          Session.AUTO_ACKNOWLEDGE);

       

                  TopicPublisher producer = session.createPublisher(topic);

                  TopicSubscriber subscriber1 = session.createDurableSubscriber(topic, "subscription-1");

                  TopicSubscriber subscriber2 = session.createDurableSubscriber(topic, "subscription-2");

       

                  TextMessage txtMsg = null;

                  int sentMsgsNo = 100;

       

                  for (int i = 0; i < sentMsgsNo; i++) {

                      txtMsg = session.createTextMessage("Test topic durable subscription.Message " + i);

                      producer.send(txtMsg);

                  }

                  System.out.println("Publisher sent messages.");

       

                  connection.start();

       

                  while ((txtMsgResponse = (TextMessage) subscriber1.receive(5000)) != null) {

                      System.out.println("Subscriber1 received message: " + txtMsgResponse.getText());

                      subscriber1MsgsNo++;

                  }

       

                  while ((txtMsgResponse = (TextMessage) subscriber2.receive(5000)) != null) {

                      System.out.println("Subscriber2 received message: " + txtMsgResponse.getText());

                      subscriber2MsgsNo++;

                  }

       

      Actual:

       

      After this example runs, each subscriber consumes 50 messages, instead of 100 each. 

      Also inside the logs, some errors/warnings appear:

       

      Node2 logs:

       

      2012-12-06 14:07:46,677 WARN  [org.hornetq.core.postoffice.impl.BindingsImpl] (Old I/O server worker (parentId: 1285635179, [id: 0x4ca13c6b, /10.30.7.83:5545])) Couldn't find binding with id=29798 on routeFromCluster for message=ServerMessage[messageID=7598,priority=4, bodySize=293,expiration=0, durable=true, address=jms.topic.testTopic3,properties=TypedProperties[{}]]@1652225357 binding = BindingsImpl [name=jms.topic.TestTopic1]

       

      Node1 logs:

       

      2012-12-06 13:54:11,577 ERROR [org.hornetq.core.server.management.impl.ManagementServiceImpl] (Thread-59 (HornetQ-client-global-threads-1265244981)) Failed to call listener: java.lang.IllegalStateException: Cannot find queue info for queue fad653b3-cb4d-406d-a58f-f83ad6a60136106f945d-3f8e-11e2-8aaf-47782ca2c34c

          at org.hornetq.core.postoffice.impl.PostOfficeImpl.onNotification(PostOfficeImpl.java:287) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.server.management.impl.ManagementServiceImpl.sendNotification(ManagementServiceImpl.java:693) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.postoffice.impl.PostOfficeImpl.removeBinding(PostOfficeImpl.java:525) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.removeBinding(ClusterConnectionImpl.java:1427) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.doBindingRemoved(ClusterConnectionImpl.java:1415) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl$MessageFlowRecordImpl.onMessage(ClusterConnectionImpl.java:1219) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:994) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.client.impl.ClientConsumerImpl.access$400(ClientConsumerImpl.java:48) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:1120) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [hornetq-core-2.2.16.Final.jar:2.2.16.Final (HQ_2_2_16_FINAL, 122)]

          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_07]

          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_07]

          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_07]

       

       

      If node2 is stopped, this doesn't happen anymore and everything seems to work ok.

       

      This scenario was reproduced on JbossAS 7.1.2 with HQ 2.2.16 and Jboss 7.2.0 Alpha1 nightly build with HQ 2.3.0Beta.

       

      I've also attached my HQ config files.