8 Replies Latest reply on Oct 15, 2012 12:28 PM by raysen_jia

    cannot enqueue more than 20,000 message

    raysen_jia Newbie

      Hi All,

       

      I am a newer about hornetq, I use hornetq core api to send message. Trying to send 100,000 messages, but after I send 19100 messages, it seems blocked by something. code as below:

       

       

      session = sf.createSession();
      ClientProducer producer = session.createProducer(queueName);
      final String propName = "myprop";
      for(int i = 0; i <= 100000; i++){
              ClientMessage message = session.createMessage(false);
                      
              message.putStringProperty(propName, "Hello sent at " + new Date()+ " " + i);
          
              System.out.println("Sending the message. " + i);
              producer.send(message);
      }
      

       

      Then I got the following result, only send to 19099, and blocked. why it doesn't send the other messages?

       

      Sending the message. 0

      .

      .

      .

      Sending the message. 19095

      Sending the message. 19096

      Sending the message. 19097

      Sending the message. 19098

      Sending the message. 19099

        • 1. Re: cannot enqueue more than 20,000 message
          Justin Bertram Master

          What <address-full-policy> are you using?

          • 2. Re: cannot enqueue more than 20,000 message
            raysen_jia Newbie

            Thank you, I use the default configuration:

            <address-full-policy>BLOCK</address-full-policy>

             

            While I have sloved this problem by changing the create queue method.

             

            The problem is that i use java code to create the queue, if i configure the queue in the hornetq-configuration.xml, it's ok.

             

            I am not sure if it is a bug?

             

            I tried again, it seems not optimistic, when i try to read the queue, it's empty.

             

            with the following code to read the queue:

             

                            HashMap<String, Object> server1Config = new HashMap<String, Object>();
                            server1Config.put(TransportConstants.HOST_PROP_NAME, "127.0.0.1");                server1Config.put(TransportConstants.PORT_PROP_NAME, "5445");
                            TransportConfiguration server1 = new TransportConfiguration(NettyConnectorFactory.class.getName(), server1Config);
            
                            ServerLocator serverLocator = HornetQClient.createServerLocatorWithoutHA(server1);
                         
            
                         ClientSessionFactory sf = serverLocator.createSessionFactory();
                        
                         ClientSession coreSession = sf.createSession(false, false, false);
            
                         final String queueName = "queue.exampleQueue";
            
                         coreSession.close();
            
                         ClientSession session = null;
            
                         try
                         {
                            session = sf.createSession();
            
                            final String propName = "myprop";
                                  ClientConsumer messageConsumer = session.createConsumer(queueName, true);
                            session.start();
                            
                            for (int i = 0; i < 1000000 ; i++) {
                                ClientMessage messageReceived = messageConsumer.receive();
                                System.out.println("Received TextMessage:" + messageReceived.getStringProperty(propName));
                            }
            

             

            Message was edited by: Raysen Jia

            • 3. Re: cannot enqueue more than 20,000 message
              Justin Bertram Master

              <address-full-policy>BLOCK</address-full-policy>

              This is why your producer is blocking.  You are using the BLOCK <address-full-policy>.  I recommend you read this section of the HornetQ documentation.

               

               

              While I have sloved this problem by changing the create queue method.

               

              The problem is that i use java code to create the queue, if i configure the queue in the hornetq-configuration.xml, it's ok.

              Whether you create the queue administratively or programmatically shouldn't make a difference.

               

               

              I tried again, it seems not optimistic, when i try to read the queue, it's empty.

              Can you paste your producer code?

              • 4. Re: cannot enqueue more than 20,000 message
                raysen_jia Newbie

                Dear Justin,

                 

                Thanks for your help.

                 

                I'll try to read the <address-full-policy>BLOCK</address-full-policy> section later, thanks for your help.

                 

                The reason why I can send success might because the queue configuration in the configuation file is not correct, i declare the following codes in hornetq-configuration.xml.

                 


                <queues>

                     <queue name="queue.exampleQueue">

                            <address>/queue/exampleQueue</address>

                            <durable>false</durable>

                      </queue>

                </queues>

                 

                When I try to send message to it, it displays success, while after that when I try to read from it, it throw nullpoint exception of the messageReceived is null.

                 

                Does the queue configuration is right in the hornetq-configuration.xml

                 

                the producer code as below:

                 

                 

                 

                                session = sf.createSession();
                
                                ClientProducer producer = session.createProducer(queueName);
                
                                ClientMessage message = session.createMessage(false);
                
                                final String propName = "myprop";
                
                                message.putStringProperty(propName, "Hello sent at "
                                        + new Date());
                
                                System.out.println("Sending the message.");
                
                                producer.send(message);
                
                                ClientConsumer messageConsumer = session
                                        .createConsumer(queueName);
                                session.start();
                
                                ClientMessage messageReceived = messageConsumer.receive(1000);
                                System.out.println("Received TextMessage:"
                                        + messageReceived.getStringProperty(propName));
                

                 

                While I had read the PAGING section, it's very useful, Thanks very much.

                 

                I think the problem which I am facing currently is how to configre the right queue.

                 

                In java code, i use the following code to create the queue:

                 

                            ClientSession coreSession = sf.createSession(false, false, false);
                
                            final String queueName = "queue.exampleQueue";
                
                            coreSession.createQueue(queueName, queueName, true);
                
                            coreSession.close();
                

                 

                Does this kind of create queue has any difference with which I create with queue configure file?

                 

                Thanks again!

                 

                Message was edited by: Raysen Jia

                • 5. Re: cannot enqueue more than 20,000 message
                  Justin Bertram Master

                  I don't see the point of  <address>/queue/exampleQueue</address> so you might want to drop that.

                   

                  Also, you probably want to call org.hornetq.api.core.client.ClientConsumer#receiveImmediate() rather than org.hornetq.api.core.client.ClientConsumer#receive(long).

                  • 6. Re: cannot enqueue more than 20,000 message
                    raysen_jia Newbie

                    Dear Justin,

                     

                    In hornetq-configuration.xml, when configure a new queue, the <address> section is mandatory, i have to declare it in the configure file.

                     

                    I run the same code, when the queue is created from code, I can received the messaged, but if the queue is predifined in the configure file, when i try to received the message, the message is null when expire.

                     

                    The reason might be two:

                    1. send the message fail

                    2. receive the message fail

                     

                    for point1, the reason might be the configuration file problem, could you please help me check if there is any error in the queue configure file

                    for point2, is there any tool to monitor if the queue is send success to queue?

                     

                    Thanks very much.

                     

                    The example code which i ues ie in the example core, whole code as below, I have tried again, the result is that, the queue configure is Ok.

                     

                    The reason might be the producer

                     

                    ClientProducer producer = session.createProducer(queueName);

                     

                    if the queue is created by coreSession.createQueue(queueName, queueName, true), we can sent message to the queue, but if the queue is already existed, i can send any number of message even with BLOCK, in another word, the message is not send success.

                     

                    try {
                                HashMap<String, Object> server1Config = new HashMap<String, Object>();
                                server1Config.put(TransportConstants.HOST_PROP_NAME, "127.0.0.1");
                                server1Config.put(TransportConstants.PORT_PROP_NAME, "5445");
                                TransportConfiguration server1 = new TransportConfiguration(NettyConnectorFactory.class.getName(), server1Config);
                    
                                ServerLocator serverLocator = HornetQClient.createServerLocatorWithoutHA(server1);
                    
                                ClientSessionFactory sf = serverLocator.createSessionFactory();
                    
                                // Step 4. Create a core queue
                                ClientSession coreSession = sf.createSession(false, false, false);
                    
                                final String queueName = "queue.exampleQueue";
                    
                    //            coreSession.createQueue(queueName, queueName, true);
                    
                                coreSession.close();
                    
                                ClientSession session = null;
                    
                                try {
                    
                                    // Step 5. Create the session, and producer
                                    session = sf.createSession();
                    
                                    ClientProducer producer = session.createProducer(queueName);
                    
                                    // Step 6. Create and send a message
                                    ClientMessage message = session.createMessage(false);
                    
                                    final String propName = "myprop";
                    
                                    message.putStringProperty(propName, "Hello sent at " + new Date());
                    
                                    System.out.println("Sending the message.");
                    
                                    producer.send(message);
                    
                                    // Step 7. Create the message consumer and start the connection
                                    ClientConsumer messageConsumer = session.createConsumer(queueName);
                                    session.start();
                    
                                    // Step 8. Receive the message.
                                    ClientMessage messageReceived = messageConsumer.receiveImmediate();
                                    System.out.println("Received TextMessage:"
                                            + messageReceived.getStringProperty(propName));
                                } finally {
                                    // Step 9. Be sure to close our resources!
                                    if (sf != null) {
                                        sf.close();
                                    }
                                }
                            } catch (Exception e) {
                                e.printStackTrace();
                                System.exit(-1);
                            }
                    

                     

                    Message was edited by: Raysen Jia

                    • 7. Re: cannot enqueue more than 20,000 message
                      Clebert Suconic Master

                      On the core-api,  you have to explicit ack all the messages.

                       

                       

                      if you want to have similar Auto-ACK semantics, you have to create your session with this:

                       

                       

                      ClientSession coreSession = sf.createSession(false, false, false, 0);

                       

                      ^^ this is for the ACK flush config

                       

                       

                      And you have to call:

                       

                      messageReceived.acknowledge()

                       

                       

                      after you received the message. Otherwise the message will just build up on the server queues.

                      1 of 1 people found this helpful
                      • 8. Re: cannot enqueue more than 20,000 message
                        raysen_jia Newbie

                        Dear Justin & Clebert,

                         

                        Thanks for your help, the reson is as Justin said, I set the <address-full-policy>BLOCK</address-full-policy>.

                         

                        After I set the BLOCK --> PAGE,  and set corrsponding element, it works correct.

                         

                        Thank you all.

                         

                        Best Regards,

                        Raysen Jia