1 2 Previous Next 24 Replies Latest reply on Jan 7, 2009 11:28 AM by kconner

    StartProcessInstanceCommand: Return token id/process instanc

    burrsutter

      From https://jira.jboss.org/jira/browse/jbesb-2239

      A very common request is to have the BpmProcessor action return a token/process instance id. Perhaps in the

       <action name="start_a_new_process_instance"
       class="org.jboss.soa.esb.services.jbpm.actions.BpmProcessor">
       <property name="command" value="StartProcessInstanceCommand" />
       <property name="process-definition-name" value="bpm_orchestration3Process"/>
       <property name="key" value="businessKey"/>
       <property name="esbToBpmVars">
       <mapping esb="BODY_CONTENT" bpm="theBody" />
       <mapping esb="contentsAsString" bpm="theData" />
       </property>
       </action>
      


      Perhaps we could do that as variable mapping.

      It has also be requested that we clean up the callback EPR XML. It is a rather large chunk of information currently


        • 1. Re: StartProcessInstanceCommand: Return token id/process ins
          apestel

          Would seem that the BpmProcessor action itself shows the need for returning a token/process id. It seems the other BpmProcessor commands (SignalCommand, CancelProcessInstanceCommand, and setProcessInstanceVariables) need to specify a token ID or process instance ID (http://www.redhat.com/docs/en-US/JBoss_SOA_Platform/4.3.GA/html-single/Programmers_Guide/index.html#d0e5151).

          If someone uses the ESB to kick off a process instance and they also want to use the ESB to be able to cancel that same process. If the ESB client that kicks off the process doesn't get the process ID returned, it will be difficult for that client to cancel the process that he just created. The non-create/start commands on BpmProcessor seem pretty useless if the create/start commands can't return a process instance id or token id.

          In addition to this is the even more common case I run into where the customer (NAVTEQ most recently) want to use BPM for service orchestration. So, client invokes a service on the ESB and that ESB service needs to synchronously kick off the orchestration of other services, aggregate a result, and return it to the original client.

          I've been telling folks to just not even use BpmProcessor and to implement their own custom action instead that invokes/controls the BPM process. I see many more real life use cases for a synchronous BpmProcessor than an asynchronous BpmProcessor.

          If this action were updated to support synchronous use, it would be nice to generically support "bpmToEsbVars" in addition to "esbToBpmVars" to be consistent with the way things are done when jBPM invokes an ESB service.

          • 2. Re: StartProcessInstanceCommand: Return token id/process ins
            kconner

            The callback EPR is necessary in order to correctly identify the state of the process and verify that we are doing the correct thing when the response arrives.

            If there is extraneous information in there then we should remove it. What is it that you feel needs 'cleaning up'?

            As for the process instance, I agree that it is useful if you want to cancel the process. Anything else and it should be done through jBPM.

            • 3. Re: StartProcessInstanceCommand: Return token id/process ins
              kconner

               

              "apestel" wrote:
              SignalCommand, CancelProcessInstanceCommand, and setProcessInstanceVariables


              Signal and setProcess should have been removed in the rewrite but weren't. Signal is severely broken and you should not be modifying *any* process variables other than through the interactions with jBPM. It is up to jBPM to drive that process.

              "apestel" wrote:
              If someone uses the ESB to kick off a process instance and they also want to use the ESB to be able to cancel that same process.

              Agreed, and that is the only useful use case so far.

              "apestel" wrote:
              In addition to this is the even more common case I run into where the customer (NAVTEQ most recently) want to use BPM for service orchestration. So, client invokes a service on the ESB and that ESB service needs to synchronously kick off the orchestration of other services, aggregate a result, and return it to the original client.

              Having things done 'synchronously' in an ESB does not imply synchronous invocations.

              "apestel" wrote:
              I've been telling folks to just not even use BpmProcessor and to implement their own custom action instead that invokes/controls the BPM process. I see many more real life use cases for a synchronous BpmProcessor than an asynchronous BpmProcessor.

              Synchronous BpmProcessors mean nothing in this environment. If you create a business process then you are ceding control to that process but that does not mean that you cannot receive notifications/replies from it.

              "apestel" wrote:
              If this action were updated to support synchronous use, it would be nice to generically support "bpmToEsbVars" in addition to "esbToBpmVars" to be consistent with the way things are done when jBPM invokes an ESB service.

              Synchronous invocation of the jBPM APIs is not necessary for integration with jBPM and is inadvisable for a number of reasons, including its effect on transactions and thread usage.

              If you can send me an example where you think this is necessary then I will look at it and see how it can be reworked. If we are missing something then we can add it.


              • 4. Re: StartProcessInstanceCommand: Return token id/process ins
                apestel

                Here are two examples / use cases:

                ---------------------------------------------------------------
                First, we have an organization that wants to accept loan applications via web services from various sources. In this case, the logical loan application process is a long running process containing human tasks.

                So, they want to expose one Web Service on the ESB that accepts the loan application and begins processing it. An email will be sent when the application is approved or denied, but they also want to provide a mechanism for the requester to check status while the process is running. So, they want to expose a second web service on the ESB that allows the requester to get back status on the loan application in process. This status may be stored in a textual jBPM variable or maybe the ESB service returns the root token's current process node name or something. The point is, without the process instance ID that was created with the application submission, how will the status check web service be able to know which process ID to check for intermediate status?
                ---------------------------------------------------------------

                Second use case - something NAVTEQ was wanting to do and something another SA's customer was wanting to do (among others):

                Customer wants to do BPEL like service orchestration but leverage the JBoss ESB concept of a service rather than be limited to and deal with the complexity of SOAP services.

                For this case, let's use the same loan application example, but let's say that now the loan application process is completely automated. There is still the initial SubmitLoanApplication WS, but it actually synchronously returns an approved or denied message. There are several steps to the loan application. First, we need to validate the application (ValidateApplicationService), then, we need to do a quick credit check (CreditCheckService), then we need to check for existing loans (RetrieveExistingCustomerLoans), then, we need to calculate a loan rate based on customer info, credit score, etc. (CalculateLoanRate), finally, we need to kick off a separate process that starts the loan process or puts the loan in a queue or something. Of course, there are conditional paths in here too. For example, based on the type of loan, we may call different CacluateLoanRate service or something, or if the RetrieveExistingCustomerLoans actually returns some existing loans, we may need to do some checking on them, etc.

                So, customer has these individual services exposed on the bus. They've got the orchestration of the service defined in jBPM so they can create it easily, see it easily, change it easily without pausing acceptance of applications, etc. They want the initial SubmitLoanApplication WS to accept the request, synchronously kick of the jBPM orchestration, wait for the process to finish, and return the contents of a loanDecision jBPM variable or something more complicated.

                Currently, the only way to do something like this (that I'm aware of) is with Sync Continuation which means the customer really has to understand the internals of our ESB, has to create a separate replyTo service in the ESB and update the jBPM process to call back to it so that the replyTo service can response to the replyTo endpoint of the original SubmitLoanApplication WS that is basically hanging out there waiting for a reply-to message. I'm sure some of my terminology is wrong there, but the point is that the customer doesn't want to hear about that complexity. They want to just create a SubmitLoanApplication WS, have that service kickoff the jBPM service orchestration, and return a result when the jBPM process finishes.
                ---------------------------------------------------------------

                Please let me know if these do not make sense.

                • 5. Re: StartProcessInstanceCommand: Return token id/process ins
                  burrsutter

                  I do understand the need to maintain the "whole" reply-to EPR as it has the context of the last node which is important. With that said, it is fairly common to have the ESB expose a WS endpoint (using either the 181+SOAPProcessor or XSD based mechanisms) and have the WS provide a response, synchronously to its waiting client.

                  One goal is to orchestrate multiple ESB hosted services and have the "composite" response sent back to the waiting client. In this case, there are no wait states in the jBPM process. This same use case should be achievable via routers as well.

                  Another goal is to start the business process via a EBWS/SOAPProcessor, return some context variable to the waiting client (e.g process instance id) so that the client can return at some later date to inquiry about the process status, the process instance variables, cancel/kill the process instance or if it is waiting tell it to move forward (aka signal).

                  Kevin, I believe that you are very wary of signal because the command doesn't know if the process instance is in a known/consistent state. Signal doesn't take into account that it is possible that another user via some other mechanism has already signaled that same instance. Plus, there are challenges with forks which involve multiple token ids instead of the single process instance id.

                  So I believe the following use cases are valid:
                  - Inquiry about a process instances status (running, completed)
                  - Inquiry about a process instances variables (get me the order total)
                  - Cancel/kill a process instance

                  Do we agree on those?

                  Burr

                  • 6. Re: StartProcessInstanceCommand: Return token id/process ins
                    burrsutter

                    Cleaning up the callback EPR....

                    The current callback into an awaiting BPM process is via this chunk of XML injected into the reply-to:

                    <wsa:From xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"><wsa:Address>logical:JBossESB-Internal#JBpmCallbackService</wsa:Address><wsa:ReferenceProperties><jbossesb:esbToBpmVars xmlns:jbossesb="http://schemas.jboss.com/ws/2007/01/jbossesb">&lt;esbToBpmVars>&lt;mapping esb="BODY_CONTENT" bpm="theBody"/>&lt;/esbToBpmVars></jbossesb:esbToBpmVars><jbossesb:jbpmTokenId xmlns:jbossesb="http://schemas.jboss.com/ws/2007/01/jbossesb">406</jbossesb:jbpmTokenId><jbossesb:jbpmNodeId xmlns:jbossesb="http://schemas.jboss.com/ws/2007/01/jbossesb">30</jbossesb:jbpmNodeId><jbossesb:jbpmProcessInstId xmlns:jbossesb="http://schemas.jboss.com/ws/2007/01/jbossesb">166</jbossesb:jbpmProcessInstId><jbossesb:jbpmProcessNodeVersionCounter30_406 xmlns:jbossesb="http://schemas.jboss.com/ws/2007/01/jbossesb">0</jbossesb:jbpmProcessNodeVersionCounter30_406><jbossesb:type xmlns:jbossesb="http://schemas.jboss.com/ws/2007/01/jbossesb">urn:jboss/esb/epr/type/logical</jbossesb:type></wsa:ReferenceProperties></wsa:From>

                    It is injected via a custom action like so:
                    message.getHeader().getCall().setTo(epr);
                    where epr equals the content listed above.

                    This is essentially our "cookie" for managing the conversation between ESB & the process instance. This chunk of XML is a bit unwieldy, especially if must go out to a business partner or actual client application. It also requires a custom action to be created.

                    An easier to use correlation solution should be created so that it works more like the older model of signal (out of the box action) & token id (a simpler "cookie").



                    • 7. Re: StartProcessInstanceCommand: Return token id/process ins
                      camunda

                       

                      Kevin, I believe that you are very wary of signal because the command doesn't know if the process instance is in a known/consistent state
                      


                      You can set an "expected state" on the jbpm SignalCommand. And I think if you get some really external signal, you need he SignalCommand, or how do you want to get around this?

                      Returning the token id when starting a new process instance is indeed an important use case where already some workarounds have been created because it is not there! So I completly support this requirement.

                      In this thread I think it is a good idea to mention again, that the whole internals of the BpmProcessor are a bit, hmm, let's say hard to understand... Best thing in my eyes is to rewrite it... I wanted to collect issues I have with it but didn't had the time yet...

                      Cheers
                      Bernd

                      • 8. Re: StartProcessInstanceCommand: Return token id/process ins
                        burrsutter

                        Bernd,

                        Do you also find the need for synchronous processing within jBPM? Where the calling thread should be blocked as each node (assumes no wait-states) is executed and control is returned to the calling thread when the process instance is completed?

                        Burr

                        • 9. Re: StartProcessInstanceCommand: Return token id/process ins
                          marklittle

                           

                          "camunda" wrote:
                          I wanted to collect issues I have with it but didn't had the time yet...


                          Bernd, it'd be good if you could jot them down over the next few weeks. Use cases, concerns, etc. always appreciated. Cheers (and have a good festive season) :-)



                          • 10. Re: StartProcessInstanceCommand: Return token id/process ins
                            camunda

                            Hi Burr.

                            burrsutter wrote:
                            Do you also find the need for synchronous processing within jBPM? Where the calling thread should be blocked as each node (assumes no wait-states) is executed and control is returned to the calling thread when the process instance is completed?


                            I see this being a main advantage of jBPM over other architectures. It makes it easily usable in a lot of different scenarios, like even 2-tier architectures or Seam Pageflow. Since you can easily change the behaviour to use JMS to decouple it from the client thread (something I think the most process engines do) by just adding a "async=true" flag, I don't see any downside of it.

                            If you want to get rid of the async flag you could even introduce a global configuration property in jbpm to change the default, should be a too big problem.

                            Despite the value for different use cases of jbpm I think "borrowing the client thread" has the big advantage, that java developers can easily understand what is going on. And it is easy to write Unit-Tests (since you do exactly know what happened after a call returns). So actually I am a big fan of it! And I know quite some projects or developers who are on my side I think :-) I think, this is one of the unique selling points of jbpm. Don't make it as unnecessary complex as other big unhandy process engines (unless there is a very good reason to ;-)).

                            Hence, the interessting question is, with the async flag in place, why should it be changed? What would be the advantage of it?

                            Cheers
                            Bernd

                            • 11. Re: StartProcessInstanceCommand: Return token id/process ins
                              kconner

                               

                              "burrsutter" wrote:
                              I do understand the need to maintain the "whole" reply-to EPR as it has the context of the last node which is important. With that said, it is fairly common to have the ESB expose a WS endpoint (using either the 181+SOAPProcessor or XSD based mechanisms) and have the WS provide a response, synchronously to its waiting client.

                              Sure, but this can already be done.

                              "burrsutter" wrote:
                              One goal is to orchestrate multiple ESB hosted services and have the "composite" response sent back to the waiting client. In this case, there are no wait states in the jBPM process. This same use case should be achievable via routers as well.

                              Actually there are wait states, at least one with every invocation of an ESB service.

                              "burrsutter" wrote:
                              Another goal is to start the business process via a EBWS/SOAPProcessor, return some context variable to the waiting client (e.g process instance id) so that the client can return at some later date to inquiry about the process status, the process instance variables, cancel/kill the process instance or if it is waiting tell it to move forward (aka signal).

                              Kill is certainly valid, and we have agreed to include that. Signal is not however as there is insufficient information to do this safely.

                              "burrsutter" wrote:
                              Kevin, I believe that you are very wary of signal because the command doesn't know if the process instance is in a known/consistent state. Signal doesn't take into account that it is possible that another user via some other mechanism has already signaled that same instance. Plus, there are challenges with forks which involve multiple token ids instead of the single process instance id.

                              Not only forks but loops

                              "burrsutter" wrote:
                              So I believe the following use cases are valid:
                              - Inquiry about a process instances status (running, completed)
                              - Inquiry about a process instances variables (get me the order total)
                              - Cancel/kill a process instance

                              Status and killing are certainly valid, I agree with those.

                              I think we need to discuss the process instance variable scenario further before deciding on that course. It would be restricted to root tokens only (i.e. global variables) and there is absolutely no guarantee of the process state (and therefore variables) when you query it. Having the process instance inform a service would be a cleaner way of handling this scenario.

                              Kev

                              • 12. Re: StartProcessInstanceCommand: Return token id/process ins
                                kconner

                                 

                                "apestel" wrote:
                                So, they want to expose a second web service on the ESB that allows the requester to get back status on the loan application in process. This status may be stored in a textual jBPM variable or maybe the ESB service returns the root token's current process node name or something. The point is, without the process instance ID that was created with the application submission, how will the status check web service be able to know which process ID to check for intermediate status?


                                We already agree on returning the process instance and, with that, the status of the process is easy to retrieve. It should also be possible to provide restricted access to variables although it is often cleaner to have the process instance control this.

                                "apestel" wrote:
                                Currently, the only way to do something like this (that I'm aware of) is with Sync Continuation which means the customer really has to understand the internals of our ESB, has to create a separate replyTo service in the ESB and update the jBPM process to call back to it so that the replyTo service can response to the replyTo endpoint of the original SubmitLoanApplication WS that is basically hanging out there waiting for a reply-to message.

                                This is the only way for a very good reason, it is the only way that is guaranteed to work in all circumstances. What you have been asking for (and have already been telling customers) is not safe.

                                "apestel" wrote:
                                I'm sure some of my terminology is wrong there, but the point is that the customer doesn't want to hear about that complexity. They want to just create a SubmitLoanApplication WS, have that service kickoff the jBPM service orchestration, and return a result when the jBPM process finishes.

                                What they are interested in hearing, however, is that their integration works. Having something that appears to work for simple cases, then breaks once they change things, is not what they want.

                                Kev

                                • 13. Re: StartProcessInstanceCommand: Return token id/process ins
                                  kconner

                                   

                                  "burrsutter" wrote:
                                  Cleaning up the callback EPR....

                                  The variable mappings should not have been included in the EPR so we can definitely remove those. The token id, node id and counter are all required to guarantee that the signal does not interfere with the process flow.

                                  "burrsutter" wrote:
                                  This is essentially our "cookie" for managing the conversation between ESB & the process instance. This chunk of XML is a bit unwieldy, especially if must go out to a business partner or actual client application. It also requires a custom action to be created.

                                  It is certainly true that something needs to manage any correlation with the external services.

                                  "burrsutter" wrote:
                                  An easier to use correlation solution should be created so that it works more like the older model of signal (out of the box action) & token id (a simpler "cookie").


                                  The older signal functionality is broken and token id is not sufficient. There are very good reasons why we rewrote that functionality.

                                  Kev


                                  • 14. Re: StartProcessInstanceCommand: Return token id/process ins
                                    kconner

                                     

                                    "camunda" wrote:
                                    You can set an "expected state" on the jbpm SignalCommand. And I think if you get some really external signal, you need he SignalCommand, or how do you want to get around this?

                                    Unfortunately this is not sufficient to guarantee that the signal does not interfere with the process flow.

                                    "camunda" wrote:
                                    Returning the token id when starting a new process instance is indeed an important use case where already some workarounds have been created because it is not there! So I completly support this requirement.

                                    Why token id and not process instance id?

                                    "camunda" wrote:
                                    In this thread I think it is a good idea to mention again, that the whole internals of the BpmProcessor are a bit, hmm, let's say hard to understand... Best thing in my eyes is to rewrite it... I wanted to collect issues I have with it but didn't had the time yet...

                                    I completely agree. The BpmProcessor is overly complicated for what it needs to do and could do with simplifying.

                                    Kev

                                    1 2 Previous Next