1 2 Previous Next 17 Replies Latest reply on Aug 17, 2011 10:55 AM by mookythemook

    Message Producer flow control seems to have stopped all producers to a destination - but no obvious reason why...

    mookythemook

      Just ran into a situation where my system that is using hornetq "hung" (stopped doing any work).

      Looking at a thread-dump, I can see my 10 processing threads all blocked trying to send a message - seemingly by the flow control. However, Im not sure why.

       

      All 10 threads have the identical stack trace :

      Agent-thread-8@15088, prio=5, in group 'main', status: 'waiting'

        java.lang.Thread.State: WAITING

                  at sun.misc.Unsafe.park(Unsafe.java:-1)

                  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)

                  at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)

                  at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:905)

                  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1217)

                  at java.util.concurrent.Semaphore.acquire(Semaphore.java:441)

                  at org.hornetq.core.client.impl.ClientProducerCreditsImpl.acquireCredits(ClientProducerCreditsImpl.java:74)

                  at org.hornetq.core.client.impl.ClientProducerImpl.doSend(ClientProducerImpl.java:305)

                  at org.hornetq.core.client.impl.ClientProducerImpl.send(ClientProducerImpl.java:142)

                  at org.hornetq.jms.client.HornetQMessageProducer.doSend(HornetQMessageProducer.java:451)

                  at org.hornetq.jms.client.HornetQMessageProducer.send(HornetQMessageProducer.java:199)

                  at moo.jms.JmsSessionProvider$2.send(JmsSessionProvider.java:175)

      <snip>

       

      Ive not set any limit on the destination (all destinations are created programmatically using the hornet admin (jmx) api).

      Ive not set any ProducerMaxRate (its -1).

      The ProducerWindowSize is the default (64 * 1024).

      The messages are not very big - ~10k?

      The broker is completely idle.

      There are no messages in the queue the producers are trying to send to.

      There are about 23 messages in 1 other queue.

      Im really not sure why the flow-control has kicked in.

       

      Anything else I should look at?

        1 2 Previous Next