1 2 Previous Next 15 Replies Latest reply on Jul 27, 2015 6:03 PM by clebert.suconic

    Inconsistant results from QueueBrowser.getEnumeration()

      I am porting a project from JBossMQ to HornetQ and have come across a stumbling block.  One of the requirements is to be able to display all currently queued messages, both scheduled and in progress.  I can do so with JBossMQ with code like:

       

      List messages;
      messages = (List)server.invoke( new ObjectName( queueName ), "listInProcessMessages", new Object[] {}, new String[] {} );
      process(messages);
      messages = (List)server.invoke( new ObjectName( queueName ), "listScheduledMessages", new Object[] {}, new String[] {} );
      process(messages);
      messages = (List)server.invoke( new ObjectName( queueName ), "listMessages", new Object[] {}, new String[] {} );
      process(messages);
      

       

      With HornetQ I found that the listInProcessMessages and listMessages methods are not available, and the listMessages method requires a filter parameter, returns an array of maps, and for quite a while the array I got back was always empty.  Once I did get data it turned out not to contain the object inside the message.  After looking around it seemed to me the best solution would be to use QueueBrowser.getEnumeration(), but I found this too was always empty.

       

      My testing involved an application running in JBoss AS4 with a single queue, and single listener, and queuing 19 ObjectMessages at a time (fairly small in size), each of which the listener takes between 5 and 60 seconds to process.

       

      I was able to run the hornetQ browser example, so I slowly changed things in my code to exactly reproduce the example, and I finally received data from both the listMessages method and QueueBrowser.getEnumeration() when I removed the listener from my queue.  Trying to understand why this allowed me to finally browse the data I guessed that all my messages were being buffered onto the listener as soon as they are sent and hence not available to be browsed.  I added <consumer-window-size>0</consumer-window-size> to hornetq-jms.xml, but then QueueBrowser.getEnumeration() would only ever return 1 message, always the next scheduled message.  When the current in progress message was completed and the currently browsable message was picked up by the listener I could then browse the next message until all were processed.  I have since tried 1 and 1024 as for my consumer-window-size with very surprising results.  I never get the same results from QueueBrowser.getEnumeration().  Always, I still get the next scheduled message, but sometime I get more messages, between 1 and 15 consecutive scheduled messages.  While a single message is being processed I can run QueueBrowser.getEnumeration() several times.  One time I get the next 3 scheduled messages.  The next time I get the next 12 scheduled messages.  The next time I get the next 1 scheduled message.

       

      Does anyone know why I am getting these results?

       

      Is there a configuration option I need to set or a different method I need to use to get this data?  I will settle for only scheduled messages if in progress messages are not retrievable.

       

      Any help you can give will be greatly appreciated.

       

      Thanx

        • 1. Re: Inconsistant results from QueueBrowser.getEnumeration()
          ataylor

          From a management API point of view it doesnt make sense to retun the body of a message. Typically these are viewed via JMX or management consoles and users are only interested in the info about the message, i.e. message id etc.

           

           

          When a message has a scheduled delivery time it isnt actually delivered to the queue until that time is reached making it available for delivery. This is why you see no messages, when u do see messages its probably because the scheduled delivery time has been hit and the messages put in the queue.

          1 of 1 people found this helpful
          • 2. Re: Inconsistant results from QueueBrowser.getEnumeration()
            timfox

            If you think there is a problem with HornetQ here, please file a JIRA, with a test program that replicates the issue and exact instructions on how to replicate it.

            • 3. Re: Inconsistant results from QueueBrowser.getEnumeration()

              Andy and Tim, Thank you for taking the time to help...

              Andy Taylor wrote:

               

              From a management API point of view it doesnt make sense to retun the body of a message. Typically these are viewed via JMX or management consoles and users are only interested in the info about the message, i.e. message id etc.

               

               

              When a message has a scheduled delivery time it isnt actually delivered to the queue until that time is reached making it available for delivery. This is why you see no messages, when u do see messages its probably because the scheduled delivery time has been hit and the messages put in the queue.

              For my use case it would be tremendously helpful for me to have the body of the message which I could get with JBossMQ through listInProgressMessages, listScheduledMessages, and listMessages but I cannot get with HornetQ through listMessages.  My testing does show that listMessages reliably shows me all messages that are not yet buffered, but without hte body of the message this is only of minorly helpful to my use case.  This is why I chose the QueueBrowser.

               

              Concerning scheduled delivery times, I do not use scheduled delivery times so I would not think that could have an impact.  Also, this would not explain why point a message is returned by the crowser, seconds later it is not returned by the browser, and a few further seconds later it is once more returned by the browser.

              Tim Fox wrote:

               

              If you think there is a problem with HornetQ here, please file a JIRA,  with a test program that replicates the issue and exact instructions on  how to replicate it.

              Given the number of configuration options and classes available, I am not convinced I have found a bug.  All I know for sure is that I have not figured out how to do what I want to do.  I am hoping some one can point out to me a setting or another class I can use to accomplish this.  Failing that I may have no other choice than to file a JIRA, but I want to exaust all other options first.

              • 4. Re: Inconsistant results from QueueBrowser.getEnumeration()
                timfox

                I don't see why you can't use a QueueBrowser

                • 5. Re: Inconsistant results from QueueBrowser.getEnumeration()

                  Tim Fox wrote:

                   

                  I don't see why you can't use a QueueBrowser

                  I am using a QueueBrowser, it just is not doing what I expected.  I have created an example which can be dropped into hornetq-2.0.0.GA/examples/jms and run like any of the other examples.  As stated in the comments of the example, I have found:

                   

                  Further testing shows each of these messages will fit in a buffer of 300 bytes so using multiples of 300 for consumer-window-size is informative.  It seems that the number of messages browsed is always AT LEAST as many messages as fit in the buffer.

                   

                  consumer-window-size = 300: 1 message buffered, AT LEAST 1 message browsed (until all are buffered)
                  consumer-window-size = 600: 2 messages buffered, AT LEAST 2 messages browsed
                  consumer-window-size = 900: 3 messages buffered, AT LEAST 3 messages browsed
                  consumer-window-size = 1200: 4 messages buffered, AT LEAST 4 messages browsed
                  consumer-window-size = 1500: 5 messages buffered, 5 messages browsed (there are only 10)

                  consumer-window-size = 0: 0 messages buffered, AT LEAST 1 message  browsed but rarely more than 1

                   

                  This is starting to look like a bug to me, but I want to run it by this forum first to see if there is something else I am overlooking.

                   

                  By the way, my use case will require consumer-window-size = 0 because I have 10 (currently) slow consumers in my actual software with 150 - 200 messages being queued every 7 - 8 minutes.  What I want is 0 messages buffered and all messaged browsed

                  • 6. Re: Inconsistant results from QueueBrowser.getEnumeration()
                    timfox

                    I can't replicate your issue. Here is a very simple test program that sets consumerr window size to 0, creates a consumer which does nothing, and creates a browser.

                     

                    All messages are seen by the browser as expected:

                     

                    public void testBrowseWithZeroConsumerWindowSize() throws Exception
                       {
                          ClientSessionFactory sf = createInVMFactory();
                         
                          sf.setConsumerWindowSize(0);

                     

                          ClientSession session = sf.createSession(false, true, true);

                     

                          session.createQueue(QUEUE, QUEUE, null, false);

                     

                          ClientProducer producer = session.createProducer(QUEUE);

                     

                          final int numMessages = 100;

                     

                          byte[] bytes = new byte[240];
                         
                          for (int i = 0; i < numMessages; i++)
                          {
                             ClientMessage message = session.createMessage(false);
                            
                             message.getBodyBuffer().writeBytes(bytes);
                            
                             message.putIntProperty("foo", i);
                            
                             producer.send(message);

                          }

                     

                          //Create a normal non browsing consumer
                          ClientConsumer consumer = session.createConsumer(QUEUE);
                         
                          session.start();
                         
                          ClientConsumer browser = session.createConsumer(QUEUE, true);

                     

                          for (int i = 0; i < numMessages; i++)
                          {
                             ClientMessage message2 = browser.receive(1000);

                     

                             assertEquals(i, message2.getIntProperty("foo").intValue());

                          }

                     

                          session.close();

                     

                          sf.close();
                       }

                    1 of 1 people found this helpful
                    • 7. Re: Inconsistant results from QueueBrowser.getEnumeration()
                      timfox

                      Your test program is confusing - why are you calling browse 101 times?

                       

                      I'm not reallly sure what you're trying to achieve here.

                      • 8. Re: Inconsistant results from QueueBrowser.getEnumeration()

                        Tim,

                         

                        Thank you for your sample program.  Using createConsumer(QUEUE, true) looks like it will do what I want, but now I will have to figure out how incorporate the core classes into my application.

                         

                        Is there a way to use both the JMS interface with JNDI and the core classes?  My application already uses JMS as it currently runs in JBoss 4 using JBossMQ.  Most of the functionality worked automatically when I replaced JBossMQ with HornetQ and I really do not want to replace all that code if I do not have to.  The one thing that did not work is a report that shows all messages currently queued.

                         

                        My hornetq-configuration.xml is the default one created for JBoss 4:

                         

                        <configuration xmlns="urn:hornetq"
                                       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                       xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
                        
                           <log-delegate-factory-class-name>org.hornetq.integration.logging.Log4jLogDelegateFactory</log-delegate-factory-class-name>
                           
                           <bindings-directory>${jboss.server.data.dir}/hornetq/bindings</bindings-directory>
                        
                           <journal-directory>${jboss.server.data.dir}/hornetq/journal</journal-directory>
                        
                           <large-messages-directory>${jboss.server.data.dir}/hornetq/largemessages</large-messages-directory>
                        
                           <paging-directory>${jboss.server.data.dir}/hornetq/paging</paging-directory>
                        
                           <connectors>
                              <connector name="netty">
                                 <factory-class>org.hornetq.integration.transports.netty.NettyConnectorFactory</factory-class>
                                 <param key="host"  value="${hornetq.remoting.netty.host:localhost}"/>
                                 <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
                              </connector>
                        
                              <connector name="in-vm">
                                 <factory-class>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</factory-class>
                              </connector>
                        
                           </connectors>
                        
                           <acceptors>   
                              <acceptor name="netty">
                                 <factory-class>org.hornetq.integration.transports.netty.NettyAcceptorFactory</factory-class>
                                 <param key="host"  value="${hornetq.remoting.netty.host:localhost}"/>
                                 <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
                              </acceptor>
                        
                              <acceptor name="in-vm">
                                <factory-class>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</factory-class>
                                <param key="server-id" value="0"/>
                              </acceptor>
                        
                           </acceptors>
                        
                           <security-settings>
                              <security-setting match="#">
                                 <permission type="createTempQueue" roles="guest"/>
                                 <permission type="deleteTempQueue" roles="guest"/>
                                 <permission type="consume" roles="guest"/>
                                 <permission type="send" roles="guest"/>
                              </security-setting>
                           </security-settings>
                        
                           <address-settings>
                              <!--default for catch all-->
                              <address-setting match="#">
                                 <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                                 <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                                 <redelivery-delay>0</redelivery-delay>
                                 <max-size-bytes>-1</max-size-bytes>
                                 <page-size-bytes>10485760</page-size-bytes>         
                                 <message-counter-history-day-limit>10</message-counter-history-day-limit>
                              </address-setting>
                           </address-settings>
                        
                        </configuration>
                        

                         

                        My hornetq-jms.xml is only slightly modified:

                         

                        <configuration xmlns="urn:hornetq"
                                    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                    xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
                        
                           <connection-factory name="NettyConnectionFactory">
                              <connectors>
                                 <connector-ref connector-name="netty"/>
                              </connectors>
                              <entries>
                                 <entry name="/ConnectionFactory"/>
                                 <entry name="/XAConnectionFactory"/>
                              </entries>
                        
                              <consumer-window-size>0</consumer-window-size>
                           </connection-factory>
                           
                           <connection-factory name="InVMConnectionFactory">
                              <connectors>
                                 <connector-ref connector-name="in-vm"/>
                              </connectors>
                              <entries>
                                 <entry name="java:/ConnectionFactory"/>
                                 <entry name="java:/XAConnectionFactory"/>
                              </entries>
                           </connection-factory>
                           
                           <queue name="DLQ">
                              <entry name="/queue/DLQ"/>
                           </queue>
                           
                           <queue name="ExpiryQueue">
                              <entry name="/queue/ExpiryQueue"/>
                           </queue>
                           
                           <queue name="Process">
                              <entry name="queue/Process"/>
                           </queue>
                           
                           <queue name="Notification">
                              <entry name="queue/Notification"/>
                           </queue>
                        
                        </configuration>
                        

                         

                        In my code I tried the following:

                         

                        // I have tried several different arguments to createClientSessionFactory, these are the latest
                        ClientSessionFactory factory = HornetQClient.createClientSessionFactory("localhost", 5445);
                        ClientSession session = factory.createSession(false, true, true);     // This line fails
                        ClientConsumer browser = session.createConsumer("jms.queue.Process", true);
                        

                         

                        The exception I get from this is:

                         

                        HornetQException[errorCode=0 message=Failed to initialise session factory]
                            at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:1043)
                            at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:828)
                            at my.code.getQueueBrowser(MyClass.java:415)

                        ...

                        Caused by: java.net.SocketException: Not a multicast address
                            at java.net.MulticastSocket.joinGroup(MulticastSocket.java:273)
                            at org.hornetq.core.cluster.impl.DiscoveryGroupImpl.start(DiscoveryGroupImpl.java:113)
                            at org.hornetq.core.client.impl.ClientSessionFactoryImpl.initialise(ClientSessionFactoryImpl.java:231)
                            at org.hornetq.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:1039)
                            ... 8 more

                         

                        Tim Fox wrote:

                         

                        Your test program is confusing - why are you calling browse 101 times?

                         

                        I'm not reallly sure what you're trying to achieve here.

                        Browse is simulating running the report to display all messages currently queued which can occur at any time during processing.  I called it 101 times to make sure all messages have time to complete processing.

                         

                        Thank you

                        • 9. Re: Inconsistant results from QueueBrowser.getEnumeration()
                          timfox

                          Like I mentioned, if you can create a simple self-contained test program someone will investigate.

                           

                          So far I haven't seen any evidence of a bug.

                           

                          The only reason I posted a core program is that it was slightly simpler. You could easily write the same program using JMS.

                          • 10. Re: Inconsistant results from QueueBrowser.getEnumeration()
                            timfox

                            I'm not really sure where this thread is going....

                            • 11. Re: Inconsistant results from QueueBrowser.getEnumeration()

                              In adding to the directionless threads. I've had a similar issue(But may be completely different), But, this time I am using MDB's to consume messages from the queue. If the MDB's are running all messages that are not paged out are in the Delivering State, and so the QueueBrowser is unable to view them. I have tried reducing the <consumer-window-size> to 0 on all of my JMS connectors, but it doesn't seem to affect anything because it appears that MDB's use a core connector(from my breif persusing of the source) and not a JMS connector to connect the queue. If I stop the MDB before sending the messages. I am able to browse the messages in the queue. Anyone have any ideas on a work around Perhaps a way to force the MDB's to use a JMS connector, or stop them from pre-fetching all of the messages ?? (Doesn't seem to make much sense with MDB's that exist on the same server anyways, and are probably using an same-VM connector).

                               

                              Queue where this is being observed as an issue: PerseusExecutorDestination

                               

                              Server:   Jboss 5.1.0.GA, all profile

                              HornetQ: hornetq-2.1.0.Final

                              OS:      Windows XP amd64.

                               

                               

                              MDB confing Annotations:

                               

                              @MessageDriven(mappedName = PerseusExecutorMDBBean.NAME, activationConfig =
                              {
                                  @ActivationConfigProperty(propertyName = "acknowledgeMode", propertyValue = "Auto-acknowledge"),
                                  @ActivationConfigProperty(propertyName = "destinationType", propertyValue = "javax.jms.Queue"),
                                  @ActivationConfigProperty(propertyName = "destination", propertyValue = "queue/PerseusExecutorDestination")   
                              }, messageListenerInterface = MessageListener.class)
                              @TransactionManagement(TransactionManagementType.BEAN)
                              @Depends(KeyLockerServiceBean.OBJECT_NAME)
                              @Pool(value="StrictMaxPool",   
                                  timeout=1000*60*60) //I set the timeout to be really large(60 minutes in this case), so that messages dont have isssues waiting for to be consumed
                              @ResourceAdapter("hornetq-ra.rar")

                              • 12. Re: Inconsistant results from QueueBrowser.getEnumeration()
                                clebert.suconic

                                For MDBs, you need to pass consumerWindowSize as one of the activation properties. That will make the Resource Adapter to configure the factory properly on HornetQActivation.

                                 

                                 

                                ...

                                @ActivationConfigProperty(propertyName = "consumerWindowSize, propertyValue = "0"),

                                ..

                                 

                                You can alternately also change this config on ra.xml, but that would make all the MDBs to have that same windowSize.

                                • 13. Re: Inconsistant results from QueueBrowser.getEnumeration()
                                  clebert.suconic

                                  Also...

                                   

                                  jms-ds is used on outbound connections. i.e. when you lookup the jmsXA and use it to produce messages, or consume messages outside of the MDB context. (such as creating a consumer).

                                   

                                  For inbound connections (MDB activation), you need to specify properties either at the ra.xml, or at the MDB (through annotation or some XML file on the deployment).

                                   

                                   

                                  and...   hornetq-jms only defines the pure ConnectionFactories. The Resource Adapter won't use those definitions.. you have to make the proper definitions at the jms-ds or at the inbound MDB connections.

                                  • 14. Re: Inconsistant results from QueueBrowser.getEnumeration()

                                    That fixed it thanks.

                                    1 2 Previous Next