Starting threads from Scheduler

[kidahl]

Hi, I am currently initiating batch processing from a scheduler implementing org.jboss.varia.scheduler.Schedulable.

This scheduler does a sequence of calls to an EJB (proxied by Spring).

This works great, but to achieve better performance, I would like to do several of the EJB calls in parallell. As far as I can see it, this could be solved by a) Using 5-10 parallell threads in my scheduler, or b) Doing asynchronous MDB calls insted of calls to my SLSB.

Do anyone have any experience with one vs. the other? Is it advisable to start threads from a scheduler at all?

Are there other techniques that I can use to enable parallell batch processing?

(As a side note if you wonder why the scheduler must do all these calls: I need the scheduler to break up the batch to allow me to have one transaction per call. I have so far been unable to make a new transaction when one EJB calls another EJB in combination with Spring and Hibernate.)

Thanks,
Karl Ivar Dahl

[genman]

You can easily create 5-10 schedulers programatically or just by putting them each into jboss-service.xml. If they all fire off at once, one of them executes in its own thread.

[kidahl]

Thanks for the tip!
In my case, using separate schedulers introduces the problem of coordinating them; the threads must work on different "chunks" of data to avoid conflicts.

I'm sure the coordination can be done, but it would be easier to use just one "master" scheduler if possible.

[kidahl]

Wait a minute... are you saying that a scheduler can start other schedulers programmatically? And they will all run in parallell?

I'm a bit unsure what the benefits will be over using threads, but i'll look into it.

Thanks again,
Karl Ivar

[genman]

I'm not sure I know what you want. But if things must execute in order, it seems like you can't use separate threads, right?

[kidahl]

Almost ;)

Transactions from the same origin must be processed in sequence. Transactions from different origins can be processed i parallell.

So the threads must be coordinated to avoid working on the same origin at the same time.

Anyway, I have solved this now by using one scheduler that populates a java.util.concurrent.BlockingQueue. My worker threads feed of the blockingqueue, and the swchedulerwaits for all threads to die before finishing.

Unless starting threads from my scheduler should be avoided, I believe this is a simple solution.

-Karl Ivar