3 Replies Latest reply on Jun 13, 2006 2:45 AM by lupson

    MessageListener not processing messages until publisher has

    lupson

      Hi all,

      sorry if this has been up previously. I've searched the forums without finding anything about my issue, so here goes:

      I have a JMS message provider (aka "Topic publisher") that delivers ObjectMessage(s) to a EJB 3.0 MessageDrivenBean (aka "Topic subscriber").

      Let's say I publish five messages in a Stateless Session EJB 3.0 (called from a JSP page) using a while()-loop and then returns flow control to the caller, for example a simple JSP-page. In that case, everything works out just fine, the MessageListeners "onMessage( ..)" is invoked and the underlying business logic is called and so on.

      My problem: If I wish to send a large number of messages in sequence, the MessageListener won't start processing the messages until the sender has finished sending all its messages - the while()-loop has ended and the method invoked in the Stateless Session EJB has finished.

      Tried a variety of solutions:
      * Inserting Thread.sleep every 5 messages for 10 seconds, to let the MessageListener "catch up". Does not work even if I let the thread sleep for several minutes.
      * Creating new instances for the TopicFactory, TopicConnection, TopicSession and TopicPublisher for each individual message and closing them after each call to "publish". No difference.
      * I've set max priority (9) on the messages, no luck either.

      Until the message-creating SLSEJB method (having the while-loop) finishes, the subscriber won't start processing. Of course, I could be a bit patient and wait until all messages have been created. That works for up to a 10000-20000 messages. After that, the JVM reports OutOfMemoryException. (Running in JBoss-IDE). I've successfully sent about 15000 messages which then have been processed.

      The thing I havn't tried is transaction management for the publisher, maybe creating small transactions for perhaps 10 messages at a time. Thinking of it, maybe the SLSEJB transacts the whole method invocation before actually sending the messages. Should have a look into that, Im not that familiar with that part of Session Beans.

      Im running jboss-4.0.4.GA in EJB 3.0 configuration.

      Anyone have the slightest idea how I can get the MessageListener to start processing messages concurrently with the publisher sending messages? Note that the Stateless Session EJB and MessageListener runs in the same JBoss instance.

      Of course I could have the SL EJB call the underlying business logic of the MessageListener directly using RMI or whatever, same JVM and all, but this is part of a "proof of concept" I'm creating for use in our business.

      Kind regards and thanks in advance for any answers.

        • 1. Re: MessageListener not processing messages until publisher

          Hey

          Reading the explanation of your problem, I would suspect that it is a design issue and hardware limitations. Here is what I think. You probably have a single processor machine and while publishing the messages in the loop you taking most of the resources of the machine. Also, since message processing is asynchronous by nature, I doubt there is a way to reliably send a message and expect it's retrival before the next message arrives.
          I also don't think that your out of memory problem has to do with the amount of messages, since JBossMQ should start softening them once it reaches certain limit. Check how you are handling a connection. The only reason why I am saying that is I just got back from the customer call and observed the out of memory problem due to the fact that client would open up a new connection to send a message every time without disposing the old one. There was more open sockets on the server then it could handle and thus it choked.
          Hope it helps, but then again, these are just thoughts.

          Regards

          Oleg Zhurakousky

          • 2. Re: MessageListener not processing messages until publisher
            jaikiran

             

            If I wish to send a large number of messages in sequence, the MessageListener won't start processing the messages until the sender has finished sending all its messages - the while()-loop has ended and the method invoked in the Stateless Session EJB has finished.


            This is what is happening in your case:

            - I guess you have Container managed transaction on your bean
            - When the method(meant to send messages) on the bean is called, the container begins a transaction.
            - The transaction is committed only when the method completes.
            - So even if you are sending 'n' number of messages through the loop, the messages wont be actually delivered until the transaction is committed.

            You will have to decide about whether to use Container managed transaction or Bean Managed transaction(which allows you to start and commit the transaction wherever you want in the bean). Or may be even try to implement the requirement using a different logic instead of using a while loop.





            • 3. Re: MessageListener not processing messages until publisher
              lupson

              Thank you for our answers. jaikiran were absolutely right - the bean used Container managed transaction. I added a:

              @TransactionManagement( value=TransactionManagementType.BEAN)

              annotation and now it works as a charm, concurrently processing and sending messages.

              The "normal" use case of the JMS-based system wouldn't be handling of massive amounts of messages, I'm just using the framework to import and process large amounts of legacy data, but I'll make sure to optimize the connection management. The while-loop is actually just a JDBC resultset iteration. As I wrote, using JMS for this particular purpose isn't strictly necessary - but a convenient way to stress-test the JMS and validate the business logic with large amounts of data.

              Well, now I'm starting to rant away. Thanks again for your answers!