8 Replies Latest reply on Jan 18, 2010 6:26 AM by Tim Fox

    JBossESB and HornetQ

    Gabor Auth Newbie

      Hi,

       

        I've installed the HornetQ 2.0.0.GA into the JBossAS 5.1.0.GA and I've installed the JBossESB 4.7 into the AS. It works perfectly... but only single node.

        I've configured a JBoss cluster with two nodes, and the story has begun...

       

        I've started the second node, and the JBoss cluster view has been modified:

      2010-01-16 16:33:40,313 INFO  [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.KRWSFE] (Incoming-4,10.8.23.10:44963) New cluster view for partition KRWSFE (id: 1, delta: 1) : [10.8.23.10:1099, 10.8.23.19:1099]
      2010-01-16 16:33:40,353 INFO  [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.KRWSFE] (AsynchViewChangeHandler Thread) I am (10.8.23.10:1099) received membershipChanged event:
      2010-01-16 16:33:40,353 INFO  [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.KRWSFE] (AsynchViewChangeHandler Thread) Dead members: 0 ([])
      2010-01-16 16:33:40,354 INFO  [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.KRWSFE] (AsynchViewChangeHandler Thread) New Members : 1 ([10.8.23.19:1099])
      2010-01-16 16:33:40,354 INFO  [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.KRWSFE] (AsynchViewChangeHandler Thread) All Members : 2 ([10.8.23.10:1099, 10.8.23.19:1099])
      2010-01-16 16:33:40,355 INFO  [org.jboss.cache.RPCManagerImpl] (Incoming-9,10.8.23.10:44963) Received new cluster view: [10.8.23.10:44963|1] [10.8.23.10:44963, 10.8.23.19:40251]

       

        The two HornetQ node has been connected too:

      2010-01-16 16:33:48,082 INFO  [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-4 (group:HornetQ-server-threads3446221-2086061525)) Connecting bridge sf.my-cluster.8756de69-02b4-11df-b035-00163621d9d7 to its destination
      2010-01-16 16:33:48,427 INFO  [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-4 (group:HornetQ-server-threads3446221-2086061525)) Bridge sf.my-cluster.8756de69-02b4-11df-b035-00163621d9d7 is connected to its destination

       

        But... I've sent some request into the ESB via ServiceInvoker, and I've seen this:

      2010-01-16 20:43:18,993 TRACE [hu.javaforum.services.bf.DocumentsUploadBFBean] (pool-34-thread-1) hu.javaforum.services.bf.DocumentsUploadBFBean.process[98]: Exit (3.33ms)

      [--- the ESB put message into the reply queue ---]
      2010-01-16 20:43:19,005 ERROR [org.hornetq.core.server.impl.ServerSessionImpl] (New I/O server worker #1-4) Failed to send message
      java.lang.NullPointerException
              at java.nio.ByteBuffer.wrap(ByteBuffer.java:373)
              at org.hornetq.core.postoffice.impl.BindingsImpl.routeFromCluster(BindingsImpl.java:464)
              at org.hornetq.core.postoffice.impl.BindingsImpl.route(BindingsImpl.java:243)
              at org.hornetq.core.postoffice.impl.PostOfficeImpl.route(PostOfficeImpl.java:631)
              at org.hornetq.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:1995)
              at org.hornetq.core.server.impl.ServerSessionImpl.handleSend(ServerSessionImpl.java:1426)

       

        I've installed and configured the HornetQ from the documentation, but I cannot decide: I've found a HornetQ or a JBossESB related bug or I'm thoughtless...

       

      Best regards,

      Gábor Auth

        • 1. Re: JBossESB and HornetQ
          Tim Fox Master

          I know little about configuring JBoss ESB to use HornetQ - that would be a question for the JBoss ESB team.

           

          Regarding clustering in HornetQ - did you successfully run the clustering examples?

           

          Also the logs you posted show JBoss Application server clustering - this is nothing to do with HornetQ clustering.

          • 2. Re: JBossESB and HornetQ
            Tim Fox Master

            Having said that, I think I can see a bug.

             

            Looks like a message was sent to a node in the cluster, then distributed internally to other node in the cluster, then forwarded to a client, which then resent the same message back to a server, prompting the error.

             

            Looks like one of the internal clustering errors isn't being removed when a message is delivered to a client.

             

            Can you file a JIRA, and someone will take a look?

            • 3. Re: JBossESB and HornetQ
              Gabor Auth Newbie

              Hi,

               

              Thank you for your quick reply.

               

              I know, the JBossAS clustering is not HornetQ clustering, but I wrote above HornetQ bridge log messages too.

               

              I've run the clustered-queue standalone HornetQ test, it was successful, after I've run the test with alive JBossAS, and I've seen these lines in the server.log file:

              11:10:24,373 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
              11:10:24,477 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
              11:10:24,649 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
              11:10:24,653 INFO  [BridgeImpl] Connecting bridge sf.my-cluster.86ee1122-0350-11df-be4d-0016363ffb71 to its destination
              11:10:24,685 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
              11:10:24,789 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
              11:10:24,893 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id

               

              But the testcase was successful too:

                   [java] Sent message: This is text message 0
                   [java] Sent message: This is text message 1
                   [java] Sent message: This is text message 2
                   [java] Sent message: This is text message 3
                   [java] Sent message: This is text message 4
                   [java] Sent message: This is text message 5
                   [java] Sent message: This is text message 6
                   [java] Sent message: This is text message 7
                   [java] Sent message: This is text message 8
                   [java] Sent message: This is text message 9
                   [java] Got message: This is text message 0 from node 0
                   [java] Got message: This is text message 1 from node 1
                   [java] Got message: This is text message 2 from node 0
                   [java] Got message: This is text message 3 from node 1
                   [java] Got message: This is text message 4 from node 0
                   [java] Got message: This is text message 5 from node 1
                   [java] Got message: This is text message 6 from node 0
                   [java] Got message: This is text message 7 from node 1
                   [java] Got message: This is text message 8 from node 0
                   [java] Got message: This is text message 9 from node 1

               

              I've created a small test application, it pushes messages into the selected queue in the first JBossAS(+ESB) node, and an another application pops messages from the same queue name in the second JBossAS(+ESB) node, and I've seen: the deployed queue is not clustered... so I'm confused... 

               

              So, I've installed again JBossAS 5.1.0.GA and I've installed HortnetQ 2.0.0.GA into the JBossAS (as I've read in the documentation), and I've run these test applications... and I've seen same results, the consumer on the second node has not seen the message, but the producer on the first node has put it into the queue. I've seen messages in the queue on the first JBossAS node and I've not seen messages in the same queue on the second JBossAS node... Whats wrong?

               

              Gábor Auth

              • 4. Re: JBossESB and HornetQ
                Tim Fox Master

                Can you replicate this without JBossESB? The HornetQ team won't debug ESB code.

                 

                Also please post your test application and configuration, and explain what you expected to see, and what you actually aw.

                • 5. Re: JBossESB and HornetQ
                  Gabor Auth Newbie

                  No, I can't replicate this without ESB, because I don't know the JBossESB what did do exactly. The ESB service creates at least one pair of queue: one for a request messages and another one for a reply messages. The ESB service receives a request message from a (for example) "SimpleServiceRequest" queue, process it, and puts a response message to the "SimpleServiceRequest_reply" queue. The ESB client puts it own message into the "SimpleServiceRequest" queue and wait for reply on the "SimpleServiceRequest_reply" queue.

                   

                  But...

                   

                  I have installed a HornetQ 2.0.0.GA into a JBossAS 5.1.0.GA (without ESB) on two RHEL 5.4 Linux machines as I've read in the documentation, I've started the 'all-with-hornetq' server on these two machines and I've run these small applicaton:

                        InitialContext ic = new InitialContext();
                        Queue queue = (Queue) ic.lookup("/queue/TestQueue");
                        ConnectionFactory qcf = (ConnectionFactory) ic.lookup("java:/XAConnectionFactory");
                        Connection connection = qcf.createConnection();
                        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

                         String hostname = InetAddress.getLocalHost().getHostName();
                        Boolean isProducer = "node1".equals(hostname);
                        if (isProducer)
                        {
                          connection.start();
                          MessageProducer producer = session.createProducer(queue);
                          TextMessage message = session.createTextMessage("This is text message" + new Date().getTime());
                          producer.send(message);
                          connection.close();
                        } else
                        {
                          connection.start();
                          MessageConsumer consumer = session.createConsumer(queue);
                          TextMessage message = (TextMessage) consumer.receiveNoWait();
                          if (message != null)
                          {
                            perfLogger.doLogInfo("Got message: '%1$s'", message.getText());
                          } else
                          {
                            perfLogger.doLogInfo("No message in the queue");
                          }
                          connection.close();
                        }

                   

                  The consumer on the node2 don't receive messages what the producer creates in the node1. The JBossAS and HornetQ configuration is default install, I've created only a TestQueue queue on two node in the hornetq-jms.xml, I've started a JBossAS cluster with `/opt/jboss/bin/run.sh -b 0.0.0.0 -c all-with-hornetq` command line on each node.

                   

                  Gábor Auth

                  • 6. Re: JBossESB and HornetQ
                    Tim Fox Master

                    auth.gabor wrote:

                     

                    No, I can't replicate this without ESB, because I don't know the JBossESB what did do exactly.

                     

                     

                    The ESB service creates at least one pair of queue: one for a request messages and another one for a reply messages. The ESB service receives a request message from a (for example) "SimpleServiceRequest" queue, process it, and puts a response message to the "SimpleServiceRequest_reply" queue. The ESB client puts it own message into the "SimpleServiceRequest" queue and wait for reply on the "SimpleServiceRequest_reply" queue.

                     

                    But...

                     

                    I have installed a HornetQ 2.0.0.GA into a JBossAS 5.1.0.GA (without ESB) on two RHEL 5.4 Linux machines as I've read in the documentation, I've started the 'all-with-hornetq' server on these two machines and I've run these small applicaton:

                          InitialContext ic = new InitialContext();
                          Queue queue = (Queue) ic.lookup("/queue/TestQueue");
                          ConnectionFactory qcf = (ConnectionFactory) ic.lookup("java:/XAConnectionFactory");
                          Connection connection = qcf.createConnection();
                          Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);

                           String hostname = InetAddress.getLocalHost().getHostName();
                          Boolean isProducer = "node1".equals(hostname);
                          if (isProducer)
                          {
                            connection.start();
                            MessageProducer producer = session.createProducer(queue);
                            TextMessage message = session.createTextMessage("This is text message" + new Date().getTime());
                            producer.send(message);
                            connection.close();
                          } else
                          {
                            connection.start();
                            MessageConsumer consumer = session.createConsumer(queue);
                            TextMessage message = (TextMessage) consumer.receiveNoWait();
                            if (message != null)
                            {
                              perfLogger.doLogInfo("Got message: '%1$s'", message.getText());
                            } else
                            {
                              perfLogger.doLogInfo("No message in the queue");
                            }
                            connection.close();
                          }

                     

                    The consumer on the node2 don't receive messages what the producer creates in the node1. The JBossAS and HornetQ configuration is default install, I've created only a TestQueue queue on two node in the hornetq-jms.xml, I've started a JBossAS cluster with `/opt/jboss/bin/run.sh -b 0.0.0.0 -c all-with-hornetq` command line on each node.

                     

                    Gábor Auth

                    Firstly, the HornetQ team won't know what ESB does either. You need to contact the ESB team for that.

                     

                    A few other observations:

                     

                    Why are you using XAConnectionFactory? You aren't using XA so you should just use /ConnectionFactory

                     

                    Secondly receiveNoWait will only return a message if one is *immediately available in the queue*. There is no guarantee that the message has reached the second node yet.

                     

                    Thirdly, see what you have set routeWhenNoConsumers to in your cluster connection config.

                    • 7. Re: JBossESB and HornetQ
                      Tim Fox Master

                      The reason your application is not working as you expect, is because when you send the message there are no consumers on any nodes, so your message just remains on the first node.

                       

                      You then create the consumer on the second node, but the message is not on the second node, so it won't be consumed.

                       

                      If you created the consumer *before* you sent the message, you would receive it.

                       

                      If you want messages to be automatically distributed to other nodes when you nedd to enable "message redistribution", by default this is not enabled- this is discussed in the user manual, clustering chapter.

                       

                      I suggest you read the user manual, especially the parts about "cluster connections", "load balancing" and "message redistribution".