[kconner]

"burrsutter" wrote:
Do you also find the need for synchronous processing within jBPM? Where the calling thread should be blocked as each node (assumes no wait-states) is executed and control is returned to the calling thread when the process instance is completed?

Doing this in the current manner is the only way that is guaranteed to work 100% of the time and scales very well. It also integrates well with transactions, something else that synchronous invocations of the process is not guaranteed to do.

In general you do not know how jBPM will choose to control the process flow. Any action within the process instance can choose to defer processing, whether it is through async, timers, human intervention, invoking ESB services or other possibilities. It just takes one of these to break the integration you are suggesting.

Kev

[kconner]

"camunda" wrote:
Hence, the interessting question is, with the async flag in place, why should it be changed? What would be the advantage of it?

There are other reasons why a process instance is not guaranteed to work in a synchronous manner on the calling thread, such as timers, human intervention, ESB invocations and possibly others. In general this is not something that the invoker of the process instance can control.

There are also issues when using transactions, especially if a transport such as JMS is used, and resource usage/contention/scaling.

These are all issues that we take into consideration with the current approach.

Kev

[kconner]

Doing this in the current manner is the only way that is guaranteed to work 100% of the time and scales very well.

I should also add that if you believe this is not the case then we need to find out why and fix any issues.

Kev

[apestel]

There are other reasons why a process instance is not guaranteed to work in a synchronous manner on the calling thread, such as timers, human intervention, ESB invocations and possibly others. In general this is not something that the invoker of the process instance can control.

For clarity to make sure I understand, let's say that we remove ESB from the picture and we have a customer just using jBPM. They create a web application that accepts a bunch of form data for a loan application and on submission, the web app creates a jBPM process instance with the form data and signals it which does credit check, loan rate calculation, maybe other steps, and then finishes by producing a loan result in a jBPM process variable. When the process finishes (signal returns after couple seconds), the web app extracts the loan result from the finished jBPM process and displays it in a web page to the user.

Would we say that is improper use of jBPM and the web app should instead signal the jBPM process in a separate thread, block the web app thread, implement the jBPM process with some sort of callback at process completion, and wait for the jBPM process to call back to the web application when it has completed so that the web application can get the jBPM process variable with the loan result and render that in the page returned to the user?

[kconner]

"apestel" wrote:
When the process finishes (signal returns after couple seconds), the web app extracts the loan result from the finished jBPM process and displays it in a web page to the user.

As I said previously, return from signal guarantees nothing as far as process completion is concerned (even without ESB in the mix).

Your example above is very likely to include a remote invocation to a system run by Equifax (or one of the other agencies) and this is likely to be done over an asynchronous messaging transport if they want it to scale. (yes, I have worked for Equifax and an online bank and have written both sides of this). This may take an indeterminate time and the jBPM processing is likely to include timeout behaviour or even human interaction depending on the circumstances.

"apestel" wrote:
Would we say that is improper use of jBPM and the web app should instead signal the jBPM process in a separate thread, block the web app thread, implement the jBPM process with some sort of callback at process completion, and wait for the jBPM process to call back to the web application when it has completed so that the web application can get the jBPM process variable with the loan result and render that in the page returned to the user?

I would definitely say that taking the simple approach will not be guaranteed to work in all circumstances and that I hope they can handle this ;)

Kev

[camunda]

Kevin wrote:

Having the process instance inform a service would be a cleaner way of handling this scenario.

But how to implement this if you think of the use case that an external client asks for the status of the process instance?

Kevin wrote:

It is certainly true that something needs to manage any correlation with the external services.
...
The older signal functionality is broken and token id is not sufficient. There are very good reasons why we rewrote that functionality.

Bernd wrote:

You can set an "expected state" on the jbpm SignalCommand. And I think if you get some really external signal, you need he SignalCommand, or how do you want to get around this?

You skipped the second question in that sentence ;-) Let's say you handed over the token id as correlation identifier to a third party application. It calls back, e.g. via JMS, containing only the token id. Now you have to signal the process instance with that token id, right?

Okay, I see your problems with the pure Signal. But otherwise correlation has to be solved correctly inside of the ESB, in this example the ESB needs to restore all these EPR data (token-id, node-id, counter, ...) from the token-id as correlation identifier. And currently this functionality is missing, so this is why I see the demand for the "old" SignalCommand.

By the way, an often requested use case is to update the version of the ProcessDefinition of a special ProcessInstance (e.g. to apply a fix). This is possible with a Command in jbpm, even if it is marked as experimental, since it can cause some problems. One problem with the currrent implementation is, that node-id has changed, so the EPR isn't working. I added a note in the ChangeProcessInstanceVersionCommand but think we can ignore that for know. If people use that command, they should know what they are doing...

Kevin wrote:

Why token id and not process instance id?

I always work with the token id, because it is unambigous if multiple tokens exist for the ProcessInstance.

Kevin wrote:

These are all issues that we take into consideration with the current approach.

Basically I agree with it. I am not complaining about the solution as such, the problem is more the too complicated code and missing examples and the lack of an excellent tutorial for it. And some "details" are missing, like the correlation stuff. All of it "just" a matter of resources and time ;-)

apestel wrote:

When the process finishes (signal returns after couple seconds), the web app extracts the loan result from the finished jBPM process and displays it in a web page to the user.

This is a common discussion. Basically you should not tightly couple the process with the user interface. As Kevin wrote, other things can happen in between that interrupt normal process flow. Another argument is, that you want to gain flexibility in changing your business processes. For example you change the loan application to include a human task in some cases. Now you have to change your web application too! But you shouldn't! Better is some kind of generic piece of software in the web application that somehow can handle the "result" of the signal (wait-state x, ended, ...) in a generic way...

Cheers
Bernd

[marklittle]

"camunda" wrote:

apestel wrote:

When the process finishes (signal returns after couple seconds), the web app extracts the loan result from the finished jBPM process and displays it in a web page to the user.

This is a common discussion. Basically you should not tightly couple the process with the user interface. As Kevin wrote, other things can happen in between that interrupt normal process flow. Another argument is, that you want to gain flexibility in changing your business processes. For example you change the loan application to include a human task in some cases. Now you have to change your web application too! But you shouldn't! Better is some kind of generic piece of software in the web application that somehow can handle the "result" of the signal (wait-state x, ended, ...) in a generic way...

It's actually more fundamental than jBPM: within the ESB we've been trying to encourage a more loosely coupled, message-driven approach for years. Within the industry we've been doing it for much longer. :-)

[kconner]

"camunda" wrote:
But how to implement this if you think of the use case that an external client asks for the status of the process instance?

Sorry, I thought I had previously agreed that was a valid use case.

"camunda" wrote:
You skipped the second question in that sentence ;-) Let's say you handed over the token id as correlation identifier to a third party application. It calls back, e.g. via JMS, containing only the token id. Now you have to signal the process instance with that token id, right

Sorry, I thought I had answered this. You cannot do this with just the token id as it is not sufficient for signalling. You need to have a different correlation mechanism with the external service and something has to manage that correlation.

"camunda" wrote:
And currently this functionality is missing, so this is why I see the demand for the "old" SignalCommand.

It is the correlation functionality that is missing and not the signal command.

"camunda" wrote:
By the way, an often requested use case is to update the version of the ProcessDefinition of a special ProcessInstance (e.g. to apply a fix). This is possible with a Command in jbpm, even if it is marked as experimental, since it can cause some problems. One problem with the currrent implementation is, that node-id has changed, so the EPR isn't working. I added a note in the ChangeProcessInstanceVersionCommand but think we can ignore that for know. If people use that command, they should know what they are doing...

This is not something we support with the current integration as there are many ways in which it can go wrong. As you say, it is experimental.

"camunda" wrote:
I always work with the token id, because it is unambigous if multiple tokens exist for the ProcessInstance.

Except tokens come and go, loop etc.

"camunda" wrote:
Basically I agree with it. I am not complaining about the solution as such, the problem is more the too complicated code and missing examples and the lack of an excellent tutorial for it. And some "details" are missing, like the correlation stuff. All of it "just" a matter of resources and time ;-)

I think we agree on all of these :)

Thanks very much for the feedback, keep it coming :)

Kev

[camunda]

Kevin wrote:

It is the correlation functionality that is missing and not the signal command.

Agreed. But in reality in your project, what do think the people do? Build a (complicated) correlation mechanism or hack an action which enables the SIgnalCommand again? ;-)

BTW: In pre-ESB projects we often used the token-id only for correlation, in some circumstances this doesn't cause problems. This is a project based decision. But to have the expected state is definitely best practice, I agree.

Kevin wrote:

camunda wrote:

I always work with the token id, because it is unambigous if multiple tokens exist for the ProcessInstance.

Except tokens come and go, loop etc.

But you could have to process flows in parallel, both communicating with the ESB or maybe both flows even communicate with an external partner. Only the token id is unambiguous in this case. In the projects I know I always was fine with the token id.

BTW: Where is the problem with a loop? That doesn't change anything with the token id. Only Fork/Join does...

Kevin wrote:

Thanks very much for the feedback, keep it coming :)

Don't worry ;-)

[kconner]

"camunda" wrote:
Agreed. But in reality in your project, what do think the people do? Build a (complicated) correlation mechanism or hack an action which enables the SIgnalCommand again? ;-)

It is our intention to provide some support for this but, as always, there are other issues to be addressed.

"camunda" wrote:
But you could have to process flows in parallel, both communicating with the ESB or maybe both flows even communicate with an external partner. Only the token id is unambiguous in this case. In the projects I know I always was fine with the token id.

But in this case the communication would initially be driven by the jBPM process and, as such, the correlation mechanism would again be relevant. You would still need to handle instances where the token had moved forward and/or looped and the token id is not sufficient for this.

"camunda" wrote:
BTW: Where is the problem with a loop? That doesn't change anything with the token id. Only Fork/Join does...

It doesn't change the token id but it can change the request being made. As such any corresponding responses should be matched up.

"camunda" wrote:
Don't worry ;-)

Glad to hear it :)

Kev