1 2 Previous Next 15 Replies Latest reply on Apr 26, 2010 9:15 AM by timfox

    Questions about HornetQ upgrade from JBoss messaging

    yongz

      Hi,

       

      * HornetQ 2.0.0.GA

      * Java 6

      * JBoss 5.1.0.GA

       

      We are currently upgrading to HornetQ 2.0.0.GA from JBoss messaging, and I have two questions:

       

      1. In JBoss messaging, a queue can be declared as clustered or local when the server(connection factory) is clustered. Because we do not want all queues to be clustered. Is this still achievable in HornetQ?
      2. We are using client side load balancing on a cluster queue:
        • There are one message producer and 3 consumers.
        • One consumer on the same node with the producer, and the rest 2 consumers on other 2 nodes.
        • The message producer creates a temporary queue on each connection(3 connections in total).
        • Every message has JMSReplyTo set to the temporary queue.
        • When message consumer receives the message, the JMSReplyTo is extracted and a reply is sent to the temporary queue by JMSReplyTo.

       

      This used to work very well in JBoss Messaging. However, with HornetQ, only the consumer on the same node with the producer can send messages, and the consumers on other nodes got the following error message:

       

      Caused by: javax.jms.InvalidDestinationException: Queue 51b13e55-5615-4283-8e4e-66ab5f2092b2 does not exist

      at org.hornetq.jms.client.HornetQSession.createProducer(HornetQSession.java:333)

      at org.hornetq.jms.client.HornetQSession.createSender(HornetQSession.java:822)

       

      It appears that the temporary queue only works on the same node with message producer in HornetQ? is this true? or it would be an issue for us.

       

      Thanks in advance.

        • 1. Re: Questions about HornetQ upgrade from JBoss messaging
          yongz
          It’ll be much appreciated if anyone in HornetQ team can spare few mins of your valuable time. I have read HornetQ User Manual about clustering and temporary queue, but have not been able to confirm the above information.
          While reading this JIRA:
          Tim mentioned that temporary queue information is propagated asynchronously across the cluster so other nodes know about it. Can I take this as that temporary queue is still accessible across the cluster? if so, how can we ensure the propagation has completed before we can send message to the temporary queue from another node?
          Thanks a lot.
          • 2. Re: Questions about HornetQ upgrade from JBoss messaging
            timfox
            Please post your config and a code example which demonstrates the issue.
            • 3. Re: Questions about HornetQ upgrade from JBoss messaging
              yongz

              Hi Tim,

               

              The config and code example which demonstrates the issue are attached as below:

               

              • The code example is to be run inside JBoss server: JmsTestClient and TempQueueTest.
              • At least two clustered JBoss servers are required.

               

              Here are some logging messages:

               

              • JmsTestClient runs on a jboss server and sends messages to two MDBs: one MDB is on the same node, another MDB is on a remote node. The messages are balanced on the two nodes via client side load balancing as expected:

               

              Start to run example.................

              Sent message: A:This is text message 0   <--- send to the same node

              Sent message: B:This is text message 0  <--- send to the remote node

              Sent message: A:This is text message 1

              Sent message: B:This is text message 1

              Sent message: A:This is text message 2

              Sent message: B:This is text message 2

              Sent message: A:This is text message 3

              Sent message: B:This is text message 3

              Sent message: A:This is text message 4

              Received message: A:This is text message 0   <--- MDB receives on the same node

              Received message: A:This is text message 3

              Received message: A:This is text message 1

              Sent message: B:This is text message 4

              Received message: A:This is text message 2

              Sent message: A:This is text message 5

              Received message: A:This is text message 4

              Sent message: B:This is text message 5

              Received message: A:This is text message 5

              Sent message: A:This is text message 6

              Received message: A:This is text message 6

              ................................

               

              • JmsTestClient successfully receives reply message from temporary queue on the same node:

               

              Prepare to receive messages....

              Try to receive messages from node: A

              Got message: reply message here. from node A <--- reply message received from temporary queue on the same node

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              Got message: reply message here. from node A

              receive other message from node A: null

               

              Try to receive messages from node: B <--- no reply message received from temporary queue on the remote node

               

              • The remote MDB received the message, but fails to access temporary queue created on remote node:

               

              Received message: B:This is text message 2

              Received message: B:This is text message 1

              Received message: B:This is text message 0

              Received message: B:This is text message 4

              Received message: B:This is text message 3

              javax.jms.InvalidDestinationException: Queue e22cb849-5ffd-41e7-add4-6a727eed6f56 does not exist

              at org.hornetq.jms.client.HornetQSession.createProducer(HornetQSession.java:333)

              at org.hornetq.ra.HornetQRASession.createProducer(HornetQRASession.java:1161)

              at com.test.temp.TempQueueTest.onMessage(TempQueueTest.java:75)

              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

              at java.lang.reflect.Method.invoke(Method.java:597)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeTarget(MethodInvocation.java:122)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)

              at org.jboss.ejb3.EJBContainerInvocationWrapper.invokeNext(EJBContainerInvocationWrapper.java:69)

              at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.invoke(InterceptorSequencer.java:73)

              at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.aroundInvoke(InterceptorSequencer.java:59)

              at sun.reflect.GeneratedMethodAccessor372.invoke(Unknown Source)

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

              at java.lang.reflect.Method.invoke(Method.java:597)

              at org.jboss.aop.advice.PerJoinpointAdvice.invoke(PerJoinpointAdvice.java:174)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.fillMethod(InvocationContextInterceptor.java:72)

              at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_fillMethod_30614027.invoke(InvocationContextInterceptor_z_fillMethod_30614027.java)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.setup(InvocationContextInterceptor.java:88)

              at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_setup_30614027.invoke(InvocationContextInterceptor_z_setup_30614027.java)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:62)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:56)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.tx.StatelessBMTInterceptor.handleInvocation(StatelessBMTInterceptor.java:106)

              at org.jboss.ejb3.tx.BMTInterceptor.invoke(BMTInterceptor.java:55)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:68)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:80)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:67)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67)

              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

              at org.jboss.ejb3.mdb.MessagingContainer.localInvoke(MessagingContainer.java:282)

              at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.delivery(MessageInflowLocalProxy.java:270)

              at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:140)

              at $Proxy338.onMessage(Unknown Source)

              at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:219)

              at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:767)

              at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:45)

              at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:885)

              at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)

              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

              at java.lang.Thread.run(Thread.java:619)

              javax.jms.InvalidDestinationException: Queue e22cb849-5ffd-41e7-add4-6a727eed6f56 does not exist

               

               

              Also, could you please advise how to configure a local permanent queue on a clustered server?

               

              Thanks.

              • 4. Re: Questions about HornetQ upgrade from JBoss messaging
                timfox

                Hi Yong,

                 

                I think this issue could be related to this JIRA: https://jira.jboss.org/jira/browse/HORNETQ-281 which is already fixed in TRUNK.

                 

                Can you try again from TRUNK and see if it fixes your issue?

                • 5. Re: Questions about HornetQ upgrade from JBoss messaging
                  yongz

                  Thanks Tim,

                   

                  I ran the test again using Trunk build, unfortunately still got the queue exception on the remote node:

                   

                  javax.jms.InvalidDestinationException: Queue bed3cd34-48f7-4791-acb4-c54704c27136 does not exist

                  at org.hornetq.jms.client.HornetQSession.createProducer(HornetQSession.java:331)

                  at org.hornetq.ra.HornetQRASession.createProducer(HornetQRASession.java:1161)

                  at com.test.temp.TempQueueTest.onMessage(TempQueueTest.java:75)

                  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                  .........................

                  • 6. Re: Questions about HornetQ upgrade from JBoss messaging
                    timfox
                    Yong - can you open a JIRA for this? - then someone will have a proper look at it to see whether it's config or a bug. Thx
                    • 7. Re: Questions about HornetQ upgrade from JBoss messaging
                      yongz

                      Hi Tim,

                       

                      A JIRA has been opened as requested:

                       

                      https://jira.jboss.org/jira/browse/HORNETQ-286

                       

                      Thanks.

                      • 8. Re: Questions about HornetQ upgrade from JBoss messaging
                        jmesnil

                        I finally found out the cause of your problem and it is not a timing issue.

                         

                        First, the binding from the JMS temporary queue is indeed received by server #2.

                        The issue is that JMS specifies that a destination must exist before a producer sends a message to it.

                         

                        In HornetQ, the JMS MessageProducer queries the server to know if a queue corresponding to the  destination exists.

                        In your case, the producer will query the server #2 to know if the temp destination exists. However the check is done only for queues *local to the server* and not for remote queues (see ServerSessionImpl.executeQueueQuery).

                         

                        The issue to query remote queues is that they're bound to the post office using their clusterName (i.e. name + nodeID) while local queues are bound using their routingName (i.e. name). At first glance, I do not understand why we do not use the routing name for remote queues too to add it to the address manager. A check on the binding type can then be used to distinguish between local and remote bindings.

                        • 9. Re: Questions about HornetQ upgrade from JBoss messaging
                          timfox

                          Can't you just execute a binding query rather than a queue query from HornetQSession when the MessageProducer is created?

                          • 10. Re: Questions about HornetQ upgrade from JBoss messaging
                            jmesnil

                            Tim Fox wrote:

                             

                            Can't you just execute a binding query rather than a queue query from HornetQSession when the MessageProducer is created?

                            Yes, it'd work (and I'll have to do the same for the MessageConsumer creation).

                             

                            However, shouldn't ClientSession.queueQuery() also returns true if there is a remote queue on the server bound to the given name?

                            • 11. Re: Questions about HornetQ upgrade from JBoss messaging
                              timfox

                              Queues aren't bound using names, they are bound using the address. The name is just an attribute of the binding.

                               

                              A remote queue and local queue that refer to the same *distributed* queue have different names anyway. This is necessary since the queue name must be unique in the postoffice.

                              • 12. Re: Questions about HornetQ upgrade from JBoss messaging
                                jmesnil

                                Tim Fox wrote:

                                 

                                Queues aren't bound using names, they are bound using the address. The name is just an attribute of the binding.

                                Binding does not have a name attribute. Do you mean the uniqueName, the routingName or the clusterName?

                                 

                                Tim Fox wrote:

                                 

                                A remote queue and local queue that refer to the same *distributed* queue have different names anyway. This is necessary since the queue name must be unique in the postoffice.

                                Right, for local queue, uniqueName = routingName while for remoting queue, uniqueName = clusterName = routingName + nodeID.

                                 

                                I'll use bindingQuery to fix the bug but I find counterintuitive that queueQuery returns false if there is a remote queue for the given queue name on the server.

                                • 13. Re: Questions about HornetQ upgrade from JBoss messaging
                                  timfox

                                  I don't see why it is counter-intuitive.

                                   

                                  QueueQuery returns the binding with the specified unique name. There can clearly only be a maximum of one of those in the post office.

                                  • 14. Re: Questions about HornetQ upgrade from JBoss messaging
                                    jmesnil

                                    Tim Fox wrote:

                                     

                                    I don't see why it is counter-intuitive.

                                     

                                    QueueQuery returns the binding with the specified unique name. There can clearly only be a maximum of one of those in the post office.

                                    queueQuery() will tell me that there is no such queue but a message will be routed to the remote queue with the given (routing) name...

                                     

                                    anyway, let's use bindingQuery.

                                    1 2 Previous Next