12 Replies Latest reply on Jun 14, 2007 10:03 AM by lucabotti

    process patch files

    tom.baeyens

      the jboss deployer thread made me think about process patches.

      sometimes people just want to patch a process instead of redeploying. i have always been against this as it will be very hard to know for users, what the implications are of their updates. in some cases there will be no problems, but in many cases people need to know the jpdl implementation to see all the implications of a process update.

      so for this reason, i'm still very skeptical about process updates.

      but nevertheless, for production process updates, process updates might be necessary.

      what would you think about a new file xml schema for process updates ?

      <process-update>
      
       <!-- just some ideas: -->
      
       <node name="">
       ... this node will be parsed in the context
       of the existing process combined with these
       updates and replaces the existing node...
       </node>
      
       <sub-process-binding process-state="theprocessstatename">
       <sub-process name="theNewSubprocess" [version="optionallyASpecificVersion"] />
       </sub-process-binding>
      
      </process-upddate>
      
      


      another alternative is to keep the existing process definition format and implement a merge. that parses the xml, analyses the changes and merges them in the deployed process.

      thoughts ?

        • 1. Re: process patch files
          dmlloyd

          That's a novel approach - but I think there is one main problem with it as suggested: we don't want users to actually be able to change the graph structure of their existing process (more specifically, anything that we consider to be static in the process definition) - we want them to be able to change *anything else* but that (like classes, extra process files [like images] or auxiliary XML data like GPD information).

          Maybe there could be a way to provide a process patch where just classes and files are provided, and changes to the XML that are not related to graph structure. Maybe a separate schema/DTD for updates that can clearly express this requirement.

          • 2. Re: process patch files
            brittm

            Tom,

            I like that suggestion. Already we've found ourselves in a predicament in which a transition was omitted from a process, only to find out weeks later that users were wondering how to do something from where they were in the process (or we where wondering how we could retry a failed integration step after it had moved into a "manual" state, that we never thought would need to be retried).

            The only thing we could reasonably do was to re-point an available transition to the desired node, signal the problem processes while there was little chance of normal users being on the system, then re-point the existing transition back to its original destination.

            What you're proposing would give us the ability to properly insert the needed transition into the running process without cheating (like we did) or figuring out a bunch of API calls--As it is, we've already built into our own "Admin" UI the abilities to upload updated resource files (like action handlers and such) and to re-direct transitions. Maybe its good to be wary of giving this ability to the business analyst, but it sure is useful in the development department.

            Of the two proposals, I think the merge idea is a little more dangerous as it increases the likelyhood of introducing unintended changes.

            -Britt

            • 3. Re: process patch files
              tom.baeyens

               

              "david.lloyd@jboss.com" wrote:
              That's a novel approach - but I think there is one main problem with it as suggested: we don't want users to actually be able to change the graph structure of their existing process (more specifically, anything that we consider to be static in the process definition) - we want them to be able to change *anything else* but that (like classes, extra process files [like images] or auxiliary XML data like GPD information).

              Maybe there could be a way to provide a process patch where just classes and files are provided, and changes to the XML that are not related to graph structure. Maybe a separate schema/DTD for updates that can clearly express this requirement.


              could you explain why you make the difference between changing the graph structure and other changes ?

              • 4. Re: process patch files
                camunda

                The process patch frigthens me a bit. Of course it is nice to just patch some small errors (like typos or RuntimeExceptions in Actions). But with good tests ou don't need that too much.

                So the process patch application is really to find some business problems never thougth of before (like the example of Britt, I also have a bunch of examples for it). But there I like it more to deploy a new process definition and somehow move old processes to the new version.

                And if this is implemented by a Command in the jbpm core, it is not really dirty or hacking. And if yoi provide a mapping of old process states to new process states it is nor fare away from you process-update file.

                I see no need for other stuff, and as I like the idea of only having this way with new process version.

                • 5. Re: process patch files
                  kukeltje

                  I tend to agree with Bernd. IMO a much simpler solution in case there are only small (no fundamental structural) changes.

                  • 6. Re: process patch files
                    brittm

                    Bernd, deploying a new process and migrating is not a feasible option for large, complicated, long running process.

                    When we take a new order, we don't have one process. Under one new order we spawn an order validation process, a credit check process, multiple service processes for each type of service on the order, each service process can spawn various vendor service order processes, etc., etc. Each new order can easily represent 15 processes of various types that are all interwoven together at some level. There is nothing unusual about our business; its typical telecom.

                    In this environment, it is rare that migration is an option. While late binding of subprocesses helps a lot, there are simply times when we must update processes in place. What Tom was proposing greatly helps us do that (what we're going to do anyway) in as safe a way as possible.

                    • 7. Re: process patch files
                      kukeltje

                      @Britt: And you want to update complex processes with patches? Think that is going to work? If you leave updating variables etc out of scope (would be required in both solutions, if at all) updating a simple id in an process instance would be a lot simpler than patching a process.... although... if node id's also change... hmmm....

                      Oh... and btw... there is nothing typical about telecom.... they are the exception to the KISS rule ;-)

                      • 8. Re: process patch files
                        camunda

                         

                        Bernd, deploying a new process and migrating is not a feasible option for large, complicated, long running process.


                        I agree with Ronald. I don't get the point why it is not feasable. You can deploy a new process (which can also be just a small sub-process) and migrate one process instance in a given old version (or all in a batch) to the new version.

                        What's easier with patching?

                        Or do you think of some kind of ad hoc processes (say: jump around in one process instance)? That would be a complete different (but also very interessting) field.

                        All in all, this is a not trivial task, that's true, but I have no clever idea to change that ;-)

                        • 9. Re: process patch files
                          brittm

                           

                          And you want to update complex processes with patches? Think that is going to work?

                          We already do it (successfully) by directly manipulating the database--sometimes through screens developed for that pupose. (Unfortunately, the 10 or so departments that we serve can't seem to come up with all their real requirements until they start using the system in production. And I suppose we have had one or two *ahem* mistakes on our part.)

                          What's easier with patching?

                          1) With patching, I don't need to deploy configuration and code that I'm not changing. The larger and more complicated an environment becomes, the higher the risk of introducing unintented changes, and hence the more benefits patching provides.
                          2) If a business looses track of a PAR project for whatever reason (company is acquired, IT is restructured, developers leave, data loss, etc.), and jBPM can't hand back the par file for rebuilding, then patching is the only option other than rewriting the process.


                          • 10. Re: process patch files
                            camunda

                             

                            2) If a business looses track of a PAR project for whatever reason (company is acquired, IT is restructured, developers leave, data loss, etc.), and jBPM can't hand back the par file for rebuilding, then patching is the only option other than rewriting the process.


                            Sounds more and more scary to me ;-)

                            We already do it (successfully) by directly manipulating the database--sometimes through screens developed for that pupose. (Unfortunately, the 10 or so departments that we serve can't seem to come up with all their real requirements until they start using the system in production. And I suppose we have had one or two *ahem* mistakes on our part.)


                            I now this very good, we have the same problem in most of the projects. But we can solve that with new process versioning and the ChangeProcessInstanceVersionCommand quite well. Of course a new process version has overhead (for testing and qa), but I think that is worth the effort...

                            • 11. Re: process patch files
                              tom.baeyens

                              Interesting thread. Thanks guys and galls for all the input.

                              Migrating process instances to new versions is one of the highest priority feature requests, imo. It should be possible to add that in some form or another in one of the next versions. I'll take this discussion in to account.

                              But process updates are going to be hard in the short term for database reasons. Basically, we currently work with one set of hibernate mappings. Those contain the caching. Using hibernate to update process definitions would require to clear that cache and it's hence not possible in a clustered environment. We could think of a clustered JBossCache but that is going to lead us too far, i think.

                              • 12. Re: process patch files

                                Why do not simply add a compatibility flag for the new process description (or for any node of the process descritpion) which could state the compatibility of the new process definition version with older process instances.
                                When an instance token is signalled, instead of moving blindly in the path, it could check if next version is backward compatible.
                                Also a node old-to-new mapping kind of descriptor shoud be added, or asked to be resolved at deployment.
                                This way:

                                1) I am a process in a node - token - so when i need to move i check if newer process definitions exist and if they are compatible.
                                2) If it exists, i lookup in the node old-to-new mapping to check the node compatibility and lookup for transitions (and tasks, and actions, and so on...)

                                Just (a silly?) idea.