Why asynchronous jobs are queued in jBPM 6 job executor?

[rsmagdum]

Hi,

I have a process definition where a parallel gateway has two branches. Each branch has a single asynchronous node. Each async node logs 1 to 100000 numbers in server.log file.

Please see Process_Definition.png for this.

When I run this process defintion, in process model, I can see that both async task are running parallel.

Please see Process_Model.jpg file for this.

But actually in jobs, I found only one job was running and second was queued. (After 1 min also, this queued job was not picked by job executor)

Please see Queued_Job.jpg for this.

1) Though my thread pool = 5, interval = 1s and retry = 3 was set, Why that job was queued for long time?

salaboy21

Thanks,
Rahul

[rsmagdum]

On what basis job executor decides to start a new thread for picking up next command?
Our observation is, though we start multiple instances of async jobs, only few threads from pool are used.
Sometime only single thread executes all async jobs if execution time/async job is less.

[salaboy21]

Hi Rahul,

it seems that you are heavy testing the executor service which is great.

Can you please share the project so I can take a look and answer your questions? I need to take a look at how are you using it and what is going on under the covers.

[rsmagdum]

Hi Mauricio,
Thanks for reply.

I am trying to understand how jBPM 6 job executor works.
For that I created two BPMNs -

1) Parallel Task BPMN -
    In this process defintion, a parallel gateway has two branches and each branch has a single async node to execute.

    Each async node logs 1 to 98000 numbers in server.log file (means it took significant time to complete entire execution).
    I used org.jbpm.examples.cmd.UserCommand as CommandClass.

    My executor setting was : Thread Pool Size = 10, Interval = 1s and Retry Count = 3

    When I run it, both async jobs were triggered but only 1 job was running and second was queued.

    (After 1 min also, this queued job was not picked by job executor)
    After completion of first job, second job was picked up and process completed.
    But why second job was not picked up by job executor?

2) Multi instance BPMN

    In this process definition, a multi-instance sub process has a single async node, which do the same thing as I explained above.

    This multi instance has a CollectionItem which contains 5 items.

    Means we are triggering 5 async jobs.

    In this case also, though my executor setting was set correctly only 1 job was running and all other jobs were queued.

    They executes sequentially and process completed.

    Why others jobs not picked up by job executor?

I also tried running this definitions with all 3 runtime strategies.

For us, Asynchronous job execution is one important criteria for workflow engine selection. But its not working as expected.

I am attaching zip of my files.

Please find attachment.

Thanks,
Rahul

[salaboy21]

Ok give me some time to take a look to see if I can reproduce the behaviour that you described here and get back to you.

Thanks a lot for sharing the project, it really helps a lot

[rsmagdum]

Hi Mauricio,

Its not heavy testing at all. I am just creating couple of jobs for asynchronous execution.

am I doing it correctly? Are you able to reproduce this issue?

Thanks,
Rahul

[rjintur]

Hi salaboy21,

In ideal conditions, does JobExecutor pick a thread from thread pool for each async task, based on thread availability? If yes, why we are not seeing those threads being picked up in our environment, be it multi-instance subprocess with Async tasks only or a parallel gateway with async tasks? Is there any setting that needs to be done that we might be missing?

We saw 3-4 threads being picked up from a pool of 10 threads, when number of iterations in multi-instance was 1000 or more, but not for 100 or 10. Our command in async task in resource intensive (its printing things in a log file in a loop, sleeping for some time etc). Ideally we should see different threads picking up these async tasks.

Multi-threading failure in jbpm is a major setback for jBPM comparison w.r.t. Activiti, since we need this  feature extensively.

Thanks for your help, patience and guidance,

Ritu

[rsmagdum]

Can anyone help me on this?

Regards,
Rahul

[swiderski.maciej]

see my comment on the blog, do you mind creating jira for it?

[rjintur]

Hi swiderski.maciej

I have opened the ticket [JBPM-4275] JobExecutor is not executing Async jobs/tasks in parallel, running them sequentially - JBoss Issue Tracker for this issue.

Please let me know if you need any other information.

Thanks,

Ritu

[rsmagdum]

This problem is resolved now.
[JBPM-4275] JobExecutor is not executing Async jobs/tasks in parallel, running them sequentially - JBoss Issue Tracker
Thanks swiderski.maciej for this quick enhancement.

Thanks,
Rahul