6 Replies Latest reply on Aug 20, 2008 10:07 AM by timfox

    Best practice for scheduled message processing

    pitpalme
    pitpalme Aug 19, 2008 10:23 AM

      Hello,

      I have a whole bunch of data to be processed. This data is generated by a Stateless Session Bean which "evaluates" a lot of records from a database.
      I now need to process the results asynchronously and parallel; This is kind of batch job.

      So I though "Just send the results, one by one, via JMS to a MDB. Done."

      But: As this is similar to a classical batch there now arised the necessity to only process messages at night for not having to much performance impact on the server during working hours.

      One solution would be to just "prepare" as much data as the server is capable to deal with during "off hours". But this does not scale nor is flexible enough.

      So what I'd like to do it to "schedule" delivery of messages from the JMS-queue to it's subscriber (MDB) so I can pump all preprocessed data to the Queue and just "halt" delivery during working time. The queue nevertheless should ideally be open to still receive more messages and queue them as well.

      So my question is: is there any chance to get this done within JEE/JBoss?

      Thanks a lot for any help, links to previous topics appreciated, if I'm not the first with this desire (and just was to stupid to search correctly) :-)
      --
      Regards,

      P.

      • 1678 Views
      • Tags:
        • 1. Re: Best practice for scheduled message processing
          ataylor
          ataylor Aug 19, 2008 11:20 AM (in response to pitpalme)

          With JBM 1.4 it is possible to specify the earliest time that a message will be delivered. The user manual explains how to do this, but here you are anyway.

           long now = System.currentTimeMillis();

 Message msg = sess.createMessage();

 msg.setLongProperty(JBossMessage.JMS_JBOSS_SCHEDULED_DELIVERY_PROP_NAME,
 now + 1000 * 60 * 60 * 2);

 prod.send(msg);


            • Actions
          • 2. Re: Best practice for scheduled message processing
            pitpalme
            pitpalme Aug 19, 2008 1:24 PM (in response to pitpalme)

            Thanks a lot for your reply.

            "ataylor" wrote:
            With JBM 1.4 it is possible to specify the earliest time that a message will be delivered. The user manual explains how to do this, but here you are anyway.


            If I understand this correctly I'd this way "schedule" by "calculating" when I do expect the just about to be send message being processed.

            This would mean:

            - Queue shall be allowed to deliver messages between, e.g., 9pm and 5am
            - "Pre-Processing" starts at 9:30pm
            - "Pre-Processing" now has to count and calculate, which messages can be send without any "earliest time" (or current time), because they're expected to be processed before 5am and send the rest with a timestamp of next, or even later, night.

            If I understood this right this is not exactly what I'm looking for. I'm more interested in something that only "halts" the queues delivery attempts. If not automatically, but only by calling some MBean-operation I'd be fine.

            Background is: pre-processing will be done in a much shorter amount of time than real message processing. Therefore one pre-processing run will generate a few million of messages which should be started to process. This will be finished within the time frame of one night. But than, at a defined time in the morning all message *deliveries* should be stopped and the messages kept "on hold".
            Next evening, e.g. 9pm, I want to "unrelease" the queue and continue processing messages.

            This should continue until the queue is empty.

            I need the servers resources during work-time for business logic execution and want to process the lower priority "mega batch messages" during "night-shift" asynchronously.

            Any help appreciated, thanks a lot :-)
            --
            Regards,

            P.

              • Actions
            • 3. Re: Best practice for scheduled message processing
              ataylor
              ataylor Aug 20, 2008 3:59 AM (in response to pitpalme)

              There may be a way of stopping the MDB from consuming messages i don't know, speak to the App server guys they could help you. Alternatively you couls write you're own mbean that consumes from the queue and stop and start the connection your self.

                • Actions
              • 4. Re: Best practice for scheduled message processing
                timfox
                timfox Aug 20, 2008 4:41 AM (in response to pitpalme)

                 

                "ataylor" wrote:
                There may be a way of stopping the MDB from consuming messages i don't know, speak to the App server guys they could help you. Alternatively you couls write you're own mbean that consumes from the queue and stop and start the connection your self.


                Or you could write a simple cron like program that deploys and undeploys the MDBs at the required times.

                  • Actions
                • 5. Re: Best practice for scheduled message processing
                  pitpalme
                  pitpalme Aug 20, 2008 8:11 AM (in response to pitpalme)

                  Hello,

                  "ataylor" wrote:
                  There may be a way of stopping the MDB from consuming messages i don't know, speak to the App server guys they could help you.


                  Thanks a lot for your reply. I read this as "a JBM queue can't be controlled the way you want it", right? So I'll go and figure how to play with the MDBs :-)

                  "timfox" wrote:
                  Or you could write a simple cron like program that deploys and undeploys the MDBs at the required times.


                  Thanks for this hint, albeit it sounds a little bit drastics it seems to be an, at least, quiet simple solution :-)
                  --
                  Regards,

                  P.

                    • Actions
                  • 6. Re: Best practice for scheduled message processing
                    timfox
                    timfox Aug 20, 2008 10:07 AM (in response to pitpalme)

                    If you like you can add a JIRA feature request for the ability to pause resume delivery on a queue via the management interface.

                      • Actions