1 2 Previous Next 16 Replies Latest reply on Jan 16, 2012 4:35 AM by gaohoward

    Large Messages are not handled properly in Hornetq Clustering

    hushen.savani
    hushen.savani Dec 27, 2011 9:46 AM

      Hi,

          

           I have my application running on JBossAS-6.1.1 with Hornetq-2.2.5. I have clustered my application (which is of cource written with Honetq apis (JMS)).  I have tested my application with clustering (2 nodes) with normal sized messages. It works fine. It load balances the messages between two nodes in RR Fashion properly. But when transmitting the large messages, following exception is thrown in one of the nodes:

       

      19:28:33,251 WARN  [org.hornetq.core.server.impl.QueueImpl] removing consumer which did not handle a message, consumer=org.hornetq.core.server.cluster.impl.ClusterConnectionBridge@66b2cd4f, message=Reference[3679]:RELIABLE:ServerMessage[messageID=3679,priority=4,expiration=[Tue Dec 27 19:28:43 IST 2011], durable=true, address=jms.queue.mocmMasterQueue,properties=TypedProperties[{_HQ_LARGE_SIZE=271064, _HQ_ROUTE_TOsf.mocm-cluster.afdcd7f8-3092-11e1-8e63-002128bc0745=[B@2abc60d6}]]: java.lang.NullPointerException
          at org.hornetq.core.buffers.impl.ResetLimitWrappedHornetQBuffer.<init>(ResetLimitWrappedHornetQBuffer.java:39) [:]
          at org.hornetq.core.message.impl.MessageImpl.getBodyBuffer(MessageImpl.java:267) [:]
          at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:223) [:]
          at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142) [:]
          at org.hornetq.core.server.cluster.impl.BridgeImpl.handle(BridgeImpl.java:407) [:]
          at org.hornetq.core.server.impl.QueueImpl.handle(QueueImpl.java:2017) [:]
          at org.hornetq.core.server.impl.QueueImpl.deliver(QueueImpl.java:1587) [:]
          at org.hornetq.core.server.impl.QueueImpl.doPoll(QueueImpl.java:1472) [:]
          at org.hornetq.core.server.impl.QueueImpl.access$1100(QueueImpl.java:72) [:]
          at org.hornetq.core.server.impl.QueueImpl$ConcurrentPoller.run(QueueImpl.java:2299) [:]
          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [:]
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_12]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_12]
          at java.lang.Thread.run(Thread.java:619) [:1.6.0_12]


19:28:33,259 ERROR [org.hornetq.utils.OrderedExecutorFactory] Caught unexpected Throwable: java.util.NoSuchElementException
          at org.hornetq.utils.PriorityLinkedListImpl$PriorityLinkedListIterator.repeat(PriorityLinkedListImpl.java:189) [:]
          at org.hornetq.core.server.impl.QueueImpl.deliver(QueueImpl.java:1604) [:]
          at org.hornetq.core.server.impl.QueueImpl.doPoll(QueueImpl.java:1472) [:]
          at org.hornetq.core.server.impl.QueueImpl.access$1100(QueueImpl.java:72) [:]
          at org.hornetq.core.server.impl.QueueImpl$ConcurrentPoller.run(QueueImpl.java:2299) [:]
          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [:]
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_12]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_12]
          at java.lang.Thread.run(Thread.java:619) [:1.6.0_12]


19:29:12,898 ERROR [org.hornetq.core.postoffice.impl.PostOfficeImpl] failed to expire messages for queue sf.mocm-cluster.afdcd7f8-3092-11e1-8e63-002128bc0745: java.util.ConcurrentModificationException
          at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) [:1.6.0_12]
          at java.util.HashMap$KeyIterator.next(HashMap.java:828) [:1.6.0_12]
          at org.hornetq.core.postoffice.impl.PostOfficeImpl.cleanupInternalPropertiesBeforeRouting(PostOfficeImpl.java:815) [:]
          at org.hornetq.core.postoffice.impl.PostOfficeImpl.route(PostOfficeImpl.java:591) [:]
          at org.hornetq.core.postoffice.impl.PostOfficeImpl.route(PostOfficeImpl.java:556) [:]
          at org.hornetq.core.server.impl.QueueImpl.move(QueueImpl.java:1907) [:]
          at org.hornetq.core.server.impl.QueueImpl.expire(QueueImpl.java:875) [:]
          at org.hornetq.core.server.impl.QueueImpl.expireReferences(QueueImpl.java:1098) [:]
          at org.hornetq.core.postoffice.impl.PostOfficeImpl$Reaper.run(PostOfficeImpl.java:1240) [:]
          at java.lang.Thread.run(Thread.java:619) [:1.6.0_12]

       

      I have explored that similer bug is reolved in JBoss-5.1.1-EAP. Can anyone please suggest pointers on how to resolve this issue with JBossAS-6.1.1?

       

      Thank you.

        1 2 Previous Next