mulit-threading and commit transaction

[dharraj]

Hello,

I have a method to persist process instance as follows

 public static synchronized void persistPI(Token token)
 throws Exception
 {
 ProcessInstance pi=token.getProcessInstance();
 JbpmSessionFactory factory = JbpmSessionFactory.getInstance();
 JbpmSession jbpmSession = factory.openJbpmSession();
 jbpmSession.beginTransaction();
 try
 {
 jbpmSession.getGraphSession().saveProcessInstance(pi);
 jbpmSession.commitTransaction();
 }
 catch (Exception e)
 {
 e.printStackTrace();
 jbpmSession.rollbackTransaction();
 throw e;
 }
 finally
 {
 jbpmSession.close();
 }

 }

I have multiple threads (one per child token) calling this method. I am getting the following exception

java.lang.RuntimeException: couldn't save process instance 'org.jbpm.graph.exe.ProcessInstance@1088a1b'
 at org.jbpm.db.GraphSession.saveProcessInstance(GraphSession.java:209)
 at jpl.mipl.pgs.jbpm.util.Helper.persistPI(Helper.java:35)
 at jpl.mipl.pgs.jbpm.action.ForkHandler.run(ForkHandler.java:195)
 at jpl.mipl.pgs.jbpm.action.ForkHandler.run(ForkHandler.java:223)
 at java.lang.Thread.run(Thread.java:534)
Caused by: java.util.ConcurrentModificationException
 at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:448)
 at java.util.AbstractList$Itr.next(AbstractList.java:419)
 at org.jbpm.db.LoggingSession.saveLogs(LoggingSession.java:86)
 at org.jbpm.db.GraphSession.saveProcessInstance(GraphSession.java:204)
 ... 4 more

Any ideas.

Thanks for your help and time
Raj

[ralfoeldi]

Raj,

as I just stated in the other thread you will have to lock on processInstance. What seems to be happening here is that some other thread is doing something that creates logs. If you take a look at the code you'll see:

void saveLogs(ProcessInstance processInstance) {
 LoggingInstance loggingInstance = processInstance.getLoggingInstance();
if (loggingInstance!=null) {
 Iterator iter = loggingInstance.getLogs().iterator();
 while (iter.hasNext()) {
 ProcessLog processLog = (ProcessLog) iter.next();
 saveProcessLog(processLog);
 }
}
...

A ConcurrentModificationException will only occurr if the Logs List is concurrently changed.

Greetings

[dharraj]

Hello Rainer

Do you mean locking on process instance in this method or everywhere the process instance is used? Because my original code was doing just that, as follows:

public static void persistPI(ProcessInstance pi)
 throws Exception
 {
 synchronized(pi) {
 JbpmSessionFactory factory = JbpmSessionFactory.getInstance();
 if (factory == null)
 {
 System.err.println("Factory is null....");
 return;
 }
 JbpmSession jbpmSession = factory.openJbpmSession();
 jbpmSession.beginTransaction();
 try
 {
 jbpmSession.getGraphSession().saveProcessInstance(pi);
 jbpmSession.commitTransaction();
 }
 catch (Exception e)
 {
 e.printStackTrace();
 jbpmSession.rollbackTransaction();
 throw e;
 }
 finally
 {
 jbpmSession.close();
 }
 }
 }

and I was still getting the same exception. The only other use of process instance in the code that is running concurrently is

ContextInstance ci = token.getProcessInstance().getContextInstance();
 synchronized(ci)
 {
 ci.createVariable(_node_,joinNode,parentToken);
 ci.notify();
 }

Does 'token.signal()' change process log list?

Thanks
Raj

[dharraj]

Hello Rainer,

I think I have the answer to my following question.

Does 'token.signal()' change process log list?

I have the fix and it works. Thanks

Raj

[dharraj]

Hello Rainer,

Here is how I understand the problem, see if I am correct.

I have one thread that is saving the process instance, which results in the call to the saveLogs. The saveLogs traverses pi's log list using the iterator. Meanwhile another thread calls token.signal(), which causes a new SignalLog being added to the process instance's log list.

So the solution is

synchronized(pi)
{
 token.signal()
}

Am I right?

Thanks
Raj

[ralfoeldi]

Right on.

Basically every non-read access to any element of a processInstance should be inside of a lock on the processInstance.

Logs are written, variables set, etc. etc. There are many reasons why this is required.

You cannot however use synchronized(). This locks on the Java Object, not on the processInstance as entity. If you were retrieing the processInstance in another Thread Hibernate would probably give you a different Java Object instance representing the same processInstance and synchronize would not work.

GraphSession has a lockProcessInstance() method that will lock by way oh database.

Greetings

[dharraj]

Thanks Rainer.

If the process instance persistence and token signal are both synchronized over process instance, I will not achieve a truely parallel fork, right?

Raj

[ralfoeldi]

Hi Raj,

correct, you will not achieve true Java concurrency with a fork. This is discussed in some detail in various Threads here. There are some very good reasons for jBPM not to start threads.

jBPM implements concurrency in a business process sense, not in a Java multithreading sense. This shouldn't be a real problem. For long running actions you can always spawn real threads by way of async execution (JMS, etc.).

Greetings

[dharraj]

Hello Rainer

correct, you will not achieve true Java concurrency with a fork

Are their any plans to enable this? If so, do you roughly know when?

Thanks
Raj

[ralfoeldi]

Hi Raj,

check out the various threads on this subject. This has been discussed in extreme detail.

As I mentioned there are good reasons not to spawn threads. One would be that you are simply not allowed to do so in a J2EE environment. The others lie in the concept and implementation details of the processInstance.

While missing concurrency sounds like a problem, I'm not quite sure it realy is, at least I haven't seen a use case that requires concurrency.

a) if the execution time of the concurrent paths is short (quickly executing Actions) you will not realy notice the missing concurrency.
b) if they are long running you can execute the long running parts asychronously (roll your own JMS stuff or use the new feature in 3.1)
c) if they require user interaction you will need telepathic siamese twins to get concurrent user interaction
d) if your concern is CPU utilisation (as is mine with two 16GB, 4CPU Suns in a cluster :-) you will need to have a workload backlog lined up to fully use those anyway, concurrency or no concurrency.

So my answer to your question is another question: what is the use case for Java concurrency in jBPM?

This does not give you concurrency, but maybe you don't even need it...

(I'm not aware of what the jBPM Team is planning on this, though.)

Greetings

[edgarpoce]

hi Rainer

So my answer to your question is another question: what is the use case for Java concurrency in jBPM?

Another use case for the record ;). I'm using jbpm to manage a billing system. The billing process involves not only human interaction but also time consuming queries that I'll run against an oracle 10g cluster.

I decided to use JBPM because I thought I could model the concurrent execution of the calculations, unfortunately I was wrong and we are evaluating other products because of this limitation.

Other than that I'm very happy with JBPM.

br,
edgar

[ralfoeldi]

Hi Edgar,

I would like to understand this one.

Doesn't szenario b) do the trick? Executing whatever long running task you have asynchronously by way of JMS or jBPM 3.1?

Then you would only have to lock on the processInstance for the short time you need to update variables and trigger a transition.

Am I missing something?

Greetings

[edgarpoce]

hi

Doesn't szenario b) do the trick? Executing whatever long running task you have asynchronously by way of JMS or jBPM 3.1?

I didn't know about the new feature in 3.1, I'll take a look at it asap. thanks for the tip.

Even when I could manage to roll my own jms stuff I don't like the idea of doing it, I'd like the workflow to do it for me if possible.

br,
edgar

[dharraj]

Hello All,

Let me mention this again and as clearly as I can, I am trying to model an existing process (application) pipeline using the JBPM. The input(s) to the next stage (application) in pipeline is output from previous stage. There are stages that requires multiple input(s) which are generated by mutually exclusive applications.

Now, in my world, what do you mean by the following statement.

Doesn't szenario b) do the trick? Executing whatever long running task you have asynchronously by way of JMS or jBPM 3.1?

Then you would only have to lock on the processInstance for the short time you need to update variables and trigger a transition.

Do you mean to say that instead of actual split (no child tokens, etc.), I send a JMS message that triggers running the mutually exclusive applications in parallel?

Raj

[ralfoeldi]

Hi Raj,

you do the split

 ForkA

 StateA1 StateA2
 (sends JMS (sends JMS
 and waits) and waits)

 MDB finds the
 => => meaning of life
 and signals waiting
 <= <= Token

 JoinA

The Fork Node creates two Tokens. These arrive in states that have ActionHandlers. These ActionHandlers do NOT 'find the meaning of life', they only send a JMS Message to a MDB requesting to 'find the meaning of life'. (This is the only legal way to spawn a Thread in J2EE). When (if?) the MDBs 'find the meaning of life' they signal the waiting Tokens that continue on their path.

Maybe it's easier to understand if you think of the MDBs as humans (better suited for this specific task anyway) where the workflow waits for user feedback. Only here this is implemented without Tasks.

Does this make it any more clear (if you ignore the philosophical distractions)?