1 Reply Latest reply on Mar 31, 2008 6:08 AM by Adrian Brock

    How to know when all messages have been processed?

    Tim Shaw Newbie

      Hi,

      We need to implement a parallelising system using messaging - work is handled by message beans.

      The upstream and message processing works fine - we've run cluster performance tests on 200000 messages and scalability is fine (and linear across 3 systems in a cluster).

      The problem is to determine the best way of knowing when all the messages have been processed.

      Initial attempts resulted in the thread http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4138673#4138673 which wasn't much use.

      I am loathe to use a semaphore approach - we know the number of messages before we start, so each message could decrement this - as this would introduce contention on the semaphore (however implemented I think).

      There doesn't appear to be a way of querying the queue.

      We can't stick a 'final' message onto the queue as this could be processed before all the others have been processed (each message can take a while).

      I have some other ideas, but they're all a bit twiddly : entry in DB for each completion ... count entries)

      Just wondering if anyone has had the same problem - and what suggestions there are for solving it?

      Many Thanks

      tim

        • 1. Re: How to know when all messages have been processed?
          Adrian Brock Master

          Any solution that uses non-volatile storage to hold the "count" of messages
          processed is going to fail when the server with that storage crashes unless you
          are clever about how you design the protocol.

          e.g. you could collate the responses by sending jms responses to a client
          that is designed for the purpose. e.g. uses CLIENT_ACK to wait for all
          reponses in a group (defined in the message properties) before considering the work done
          and then doing whatever is required.