3 Replies Latest reply on Feb 11, 2011 4:44 AM by Rick Xu

    46.6. Avoiding Anti-Patterns

    Rick Dong Newbie

      Hi All,

       

      I noticed there are some changes to the section on "46.6. Avoiding Anti-Patterns" between 2.1.2.Final and 2.0.0.BETA5 in that the following part is no longer existed in the 2.1.2.final version:

       

      • Avoid many selectors on a queue. Another common anti-pattern is a single queue                     which has many consumers, each one with a distinct message selector. As messages                     come into the queue they will typically only match one of the consumers. This                     does not normally give good performance since as new consumers are added the                     entire queue has to be scanned for matching messages.

        This anti-pattern can normally be avoided by instead using a topic with many                     durable subscriptions, each subscription defines a message selector. With topic                     subscriptions the selector expression is evaluated by HornetQ before the message                     goes into the subscription, so no scanning is involved.

       

      Can someone please confirm whether this is no longer an anti-pattern? Also, which is the proper way to support say 100k consumers using core API? The reason I ask this is because I noticed that the RAM required to handle 100k users is a lot lesser if using a single address + single queue with 100k filtered consumers than creating 100k queues and 100k consumers for an address. What would be the differences in performance for the two approaches?

       

       

      Thanks,

       

      rd

        • 1. Re: 46.6. Avoiding Anti-Patterns
          Clebert Suconic Master

          If you have lots of consumers on a queue: you will be doing lots of scans on a single queue.

           

           

          If you have lots of queues on an address: you will be distributing the message towards multiple queues. But you will be creating several addresses per queue.

           

           

           

          Since you have very unique requirements, I would recommend you test careful to validate the best option for your architecture. Maybe you should test the new paging on trunk (which is based on files).

          • 2. Re: 46.6. Avoiding Anti-Patterns
            Rick Dong Newbie

            Thanks for the reply clebert. I found that neither the approach works well in supporting lots of consumers using the filter. My goal is to have a many consumers each is only interested in the designated message (e.g. a message with a receiver ID). Having created many queues each with a filter on the receiver ID means that a message won't be routed until the matching queue is found. Same for creating one queue with many filtered consumers as the one and only consumer would have to be found. I've written some test case for both and the performance is rather poor with either of the approache. However, there is a third approach that is quite fast which is by creating non-filter queue that binds to an address which represents the the unique consumer. But this can lead to OOM when the message is to be sent to many addresses (e.g.: happened to me with 1 million). Any suggestions?

             

            Stack traces with OOM

             

            Exception in thread "pool-6-thread-1" java.lang.OutOfMemoryError: GC overhead limit exceeded

                at org.jboss.netty.buffer.HeapChannelBuffer.<init>(HeapChannelBuffer.java:47)

                at org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(BigEndianHeapChannelBuffer.java:39)

                at org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:139)

                at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:73)

                at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:64)

                at org.jboss.netty.buffer.DynamicChannelBuffer.copy(DynamicChannelBuffer.java:273)

                at org.hornetq.core.buffers.impl.ChannelBufferWrapper.copy(ChannelBufferWrapper.java:222)

                at org.hornetq.core.message.impl.MessageImpl.forceCopy(MessageImpl.java:939)

                at org.hornetq.core.message.impl.MessageImpl.encodeToBuffer(MessageImpl.java:872)

                at org.hornetq.core.message.impl.MessageImpl.getEncodedBuffer(MessageImpl.java:470)

                at org.hornetq.core.protocol.core.impl.wireformat.SessionSendMessage.encode(SessionSendMessage.java:70)

                at org.hornetq.core.protocol.core.impl.ChannelImpl.send(ChannelImpl.java:163)

                at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBatched(ChannelImpl.java:147)

                at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:289)

                at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:139)

                at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:144)

             

             

            ----- or the follow trace ----------------

             

             

            SEVERE: Caught unexpected exception

            java.lang.OutOfMemoryError: GC overhead limit exceeded

                at java.util.ArrayList.<init>(ArrayList.java:112)

                at org.hornetq.core.postoffice.impl.DuplicateIDCacheImpl.<init>(DuplicateIDCacheImpl.java:73)

                at org.hornetq.core.postoffice.impl.PostOfficeImpl.getDuplicateIDCache(PostOfficeImpl.java:743)

                at org.hornetq.core.paging.impl.PagingStoreImpl.<init>(PagingStoreImpl.java:196)

                at org.hornetq.core.paging.impl.PagingStoreFactoryNIO.newStore(PagingStoreFactoryNIO.java:93)

                at org.hornetq.core.paging.impl.PagingManagerImpl.newStore(PagingManagerImpl.java:230)

                at org.hornetq.core.paging.impl.PagingManagerImpl.createPageStore(PagingManagerImpl.java:104)

                at org.hornetq.core.paging.impl.PagingManagerImpl.getPageStore(PagingManagerImpl.java:130)

                at org.hornetq.core.server.impl.ServerSessionImpl.requestProducerCredits(ServerSessionImpl.java:1033)

                at org.hornetq.core.protocol.core.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:494)

                at org.hornetq.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:471)

                at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:451)

                at org.hornetq.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:412)

                at org.hornetq.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:459)

                at org.hornetq.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:135)

                at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

                at java.lang.Thread.run(Thread.java:619)

            • 3. 46.6. Avoiding Anti-Patterns
              Rick Xu Newbie

              I meet the same problom today. I just simply recieve messages with a listener:

               

              Server side:

              ServerLocator serverLocator = HornetQClient.createServerLocatorWithoutHA(new TransportConfiguration(InVMConnectorFactory.class.getName()));

              ClientSessionFactory CSF = serverLocator.createSessionFactory();

               

               

              ClientSession coreSession = CSF.createSession(true, true);

              ClientSession listenerSession = CSF.createSession();

              ClientConsumer messageConsumer = listenerSession.createConsumer(QUEUE_NAME);

              messageConsumer.setMessageHandler(ToyCarRMIServer.this);

              listenerSession.start();

               

              public void onMessage(final ClientMessage message) {

                   System.out.println("recieve a messge")

              }

               

              Client side:

              ServerLocator serverLocator = HornetQClient.createServerLocatorWithoutHA(new TransportConfiguration(NettyConnectorFactory.class.getName()));

              ClientSessionFactory CSF = serverLocator.createSessionFactory();

              ...

              ClientProducer producer = session.createProducer(QUEUE_NAME);

              ...

              producer.send(message_request);

               

              and quickly OOM in server process:

              heap dump.png

               

               

              Exception in thread "pool-1-thread-2" java.lang.OutOfMemoryError: Java heap space

                  at org.jboss.netty.buffer.HeapChannelBuffer.<init>(HeapChannelBuffer.java:47)

                  at org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(BigEndianHeapChannelBuffer.java:39)

                  at org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:139)

                  at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:73)

                  at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:64)

                  at org.jboss.netty.buffer.DynamicChannelBuffer.copy(DynamicChannelBuffer.java:273)

                  at org.hornetq.core.buffers.impl.ChannelBufferWrapper.copy(ChannelBufferWrapper.java:222)

                  at org.hornetq.core.remoting.impl.invm.InVMConnection.write(InVMConnection.java:121)

                  at org.hornetq.core.protocol.core.impl.ChannelImpl.send(ChannelImpl.java:199)

                  at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBatched(ChannelImpl.java:147)

                  at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:291)

                  at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:135)

                  at com.toycar.rmi.server.ToyCarRMIServer$5.run(ToyCarRMIServer.java:236)

                  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                  at java.lang.Thread.run(Unknown Source)

              Exception in thread "Thread-12 (group:HornetQ-server-threads9690924-3109534)" java.lang.OutOfMemoryError: Java heap space

                  at java.util.Arrays.copyOf(Unknown Source)

                  at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)

                  at java.lang.AbstractStringBuilder.append(Unknown Source)

                  at java.lang.StringBuffer.append(Unknown Source)

                  at java.io.StringWriter.write(Unknown Source)

                  at java.io.PrintWriter.write(Unknown Source)

                  at java.io.PrintWriter.write(Unknown Source)

                  at java.io.PrintWriter.print(Unknown Source)

                  at java.io.PrintWriter.println(Unknown Source)

                  at java.lang.Throwable.printStackTrace(Throwable.java:512)

                  at java.util.logging.SimpleFormatter.format(Unknown Source)

                  at java.util.logging.StreamHandler.publish(Unknown Source)

                  at java.util.logging.ConsoleHandler.publish(Unknown Source)

                  at java.util.logging.Logger.log(Unknown Source)

                  at java.util.logging.Logger.doLog(Unknown Source)

                  at java.util.logging.Logger.log(Unknown Source)

                  at org.hornetq.core.logging.impl.JULLogDelegate.error(JULLogDelegate.java:68)

                  at org.hornetq.core.logging.Logger.error(Logger.java:169)

                  at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:104)

                  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                  at java.lang.Thread.run(Unknown Source)