1 2 Previous Next 24 Replies Latest reply on May 10, 2008 7:34 PM by kukeltje

    Potential enhancement to jbpm

    argo516

      I want to submit a enhancement to jbpm and wanted to see what everybody thinks about it:

      Everytime our application starts it deploys the latest process definition files (packages in the same *.jar). This allows us to always keep our code and process definitions in sync - for example if somebody modified and packaged a new process definition.

      However, currently every time you deploy jbpm blindly creates new row(s) in the database. This is rather problematic considering that we run multiple hosts against the same db and they can start/stop many times a day (very decentralized model). This results in a lot of uneeded junk in the database.

      The change I wrote actually compares the process definition parsed from the xml file against the lattest version in the database and only deploys if it's different. This way we still get the "just deploy each time you start" behavior, however without unneeded load on the db.

        • 1. Re: Potential enhancement to jbpm
          kukeltje

          Just to make sure we are talking about the same 'problem': jBPM by itself does not deploy anything automatically or even automagically (and certainly not blindly ;-). Something has to call a method on the jBPM API to deploy things (sometimes a startup servlet).

          As you mentioned, your application does this, so it is the responsibility of that application to check whether a new version should be deployed. There are multiple reasons for this but one of the major ones is how to determine what a new version is... a hash? timestamp? version number? There has been a lot of discussion about this in the forum. You can have a look at it if you want.

          In our company we deploy a processdefinition, or rather a processarchive, the moment a new version of the full application is deployed, even if there is NO new processdefinition or whatever. We keep everything in sync by using CVS, a very reliable way if you 'tag' everything.

          • 2. Re: Potential enhancement to jbpm
            argo516

            Can you expand on "In our company we deploy a processdefinition, or rather a processarchive, the moment a new version of the full application is deployed".

            Is this part of your deployment process? If not - how does your application know that it's a new version?

            • 3. Re: Potential enhancement to jbpm
              camunda

              Short comment from me: Deployment via ant is also possible, so you can integrate in your "normal" deployment process with ant easily...

              Cheers
              Bernd

              • 4. Re: Potential enhancement to jbpm
                argo516

                I am looking for a more localized deployment process. To me, doing something like this when my application starts up:

                ProcessDefinition pd = ProcessDefiniot.parseXmlInputStream(<classpath resource>);
                context.deployProcessDefinition(pd);

                is the ideal way of deploying. To us, this has the following advantages:

                1) No external dependencies - I do not depend on a deployment script, that may fail, leaving me with an older version of process definition.
                2) Localility - the process definition xml is packaged in the same archive as the rest of my code.

                Like I mentioned, the only problem with this approach is that a new process definition record is inserted into the database when each instance of our app starts up (we are distributed across donzens of hosts).

                Ideal solution would be having context.deployProcessDefinitionIfDifferent(pd) method, that will simply do a field by field comparison of the latest version in the database and do a no-op if they're equal.

                • 5. Re: Potential enhancement to jbpm
                  kukeltje

                   

                  is the ideal way of deploying
                  .... is in the eye of the beholder

                  So you do not call any remote services? What if you start orchestrating services? Orchestration included in your war/ear as well? Oh wait... databasescripts as well? Or do you use a hibernate param like update or something like that? Locality has long gone in our environments. As mentioned we (and I know many others) control it at a different level.

                  Like I mentioned, the only problem with this approach is that a new process definition record is inserted into the database when each instance of our app starts up (we are distributed across donzens of hosts).


                  This is a problem because *you* do not check it before deploying. The same code can be added to your app.

                  that will simply do a field by field comparison of the latest version in the database and do a no-op if they're equal.
                  Field by field? first normalized? and what about the order of elements? Again... it's not that simple.

                  With regard to your previous question
                  Full application is web, database, esb (jpdl also in mule) rules (in the near future). All distributed with their own deployment model, sometimes on different servers. web/business tier may always use the latest version of the processdefinition for new processes. We are still looking for a way to have multiple versions of the webapp (using all in the jbpm console is not an option)

                  btw... dozens of hosts for one app with multiple restarst a day? What kind of company do you work for? A Telco? then again... multiple restarts.... ouch... My boss would kill me (if he could catch me)






                  • 6. Re: Potential enhancement to jbpm
                    brittm

                    Processes are generally defined at a business scope, not at application run time.

                    A process definition should be built and deployed in it's proper context--that is "Process Definition Lifecycle". Process Definition Lifecycle (and Process Instance Lifecycle for that matter) will rarely coincide with Application Lifecycle, so Application Lifecycle is generally a bad fit for managing either Process Definition Lifecycle or Process Instance Lifecycle.

                    To put it bluntly, unless your Process Definitions are dynamically generated each time your application starts, the act of starting your application shouldn't be deploying your Definitions.

                    All that being said, jBPM DOES need to support a user assigned Version/Release number that can be used by the application to make determinations as to UI requirements/various compatibilities/etc. I've recently been forced to bastardize the processDefinition description field to support this:

                    <description>uiVersion=1, release=1|Service Request for leased circuit</description>

                    ...sorry for this last rabbit trail, but Ronald brought it up, and I do hate parsing strings. :-)

                    • 7. Re: Potential enhancement to jbpm
                      kukeltje

                      Britt,

                      To put it bluntly, unless your Process Definitions are dynamically generated each time your application starts, the act of starting your application shouldn't be deploying your Definitions.


                      jBPM does *not* do this. The application (or framework) does

                      Now regarding versioning. It's not that I (we?) do not want versioning... au contraire. I do want it but in formally controlled way like you describe (without parsing the string of course ;-)) and not by comparing the xml of a pd. So a patch that does what you describe would be welcome.

                      The thing is that it is not a simple patch. Requires db changes, so it is only possible in a 3.3 release. Besides that, it requires good tests for backwards compatibility and maybe conversion. If Argo516 wants to build a patch like that he/she is more than welcome.

                      btw. did you know jbpm supports custom namespaces in the pd? You could have used that and use real xml attributes....


                      • 8. Re: Potential enhancement to jbpm
                        camunda

                        Oh no, not evil namespaces again ;-) After two days BPELing I have enough of namespaces...

                        • 9. Re: Potential enhancement to jbpm
                          kukeltje

                          namespaces are really usefull and simple if applied with caution. The WS-* hell does not make it simpler. Heck the WS-* hell itself does not make life simpler.

                          I (we) 've not had the need for bpel yet, so I cannot comment on that

                          • 10. Re: Potential enhancement to jbpm
                            brittm

                             

                            jBPM does *not* do this. The application (or framework) does

                            I was responding to argo's issue, rather than your previous post. Sorry I wasn't clear about that.

                            Versioning...
                            jBPM's current version scheme does a great job for what it is intended to within the jBPM library; however, from an administrative perspective there are three pieces of info that a developer or admin are very interested in having regarding every ProcessDefinition deployment:
                            * Which version of UI requirements are needed, if any.
                            * When was the Definition deployed (this simple timestamp can answer a lot of questions very quickly).
                            * Is this a major release that breaks compatibility, or is it a minor release that simply addresses a small bug, typo, etc.

                            These three pieces of info are pretty important for management of a large application--and they are simple--just three fields that ought to live on the ProcessDefinition itself. The developer can apply any standard for versioning and can use the data any way he sees fit.

                            <processdefinition name='myDef' release='1.12.4' ui='1.8' >

                            Since the meta data is purely user supplied and user consumed, jBPM should not bother with any type of comparison or validation. It just needs to store it. No compatibility issues.

                            On the topic of UI requirements: Two types of screen requirement versioning are appropriate--one at the process level for summary screens, and one at the task level for task screens. So, we would also apply ui= to the task entry as well:
                            <task name='myTask' ui='1.9'>

                            ProcessDefinition ui= can be used to apply screen requirement versioning to process summary screens, or to apply screen versioning to tasks across the board.
                            Task ui= can be used to apply screen requirement versioning on a task by task basis if that level of detail is desirable.

                            This solution would provide a needed mapping in a place where it is easiest to apply and use, and still keeps UI concerns at arms length from the process defintion. We're only acknowledging that there is a UI requirement for this process/task and referencing that requirement at the highest level.

                            When the solution can be so simple, it seems a shame to not provide for a user request that has become so common. If this is a lowest-common-denominator solution that doesn't raise major objections, I'll create a JIRA issue for it. I may also be able to provide an implementation if my current schedule permits, but that's not a promise.

                            • 11. Re: Potential enhancement to jbpm
                              kukeltje

                              Britt,

                              The ui requirements for a process based (web)app are in our case so complicated that they cannot be solved by the jBPM webconsole. They have to be in a war, so your ui versionnumber wil not work, at least not in a way I know of. Still I'm curious for how you think it can solve things since I cannot follow your line of reasoning (maybe due my lack of knowledge of the English language)


                              • 12. Re: Potential enhancement to jbpm
                                brittm

                                I mentioned 4 additional pieces of data that I thought should be recorded--three on the ProcessDefinition and one on the Task. Lets just consider one of them: the ui attribute on the Task.

                                Lets say we have a long running process (1+ month) called "order". On day 10 after deployment we have 200 new processes started, and the business realizes that they would like something done differently on the "shipIt" Task. The change will require a change to the Definition as well as a change in the Task's UI functionality. We determine that the UI functionality for the old and new versions of the "shipIt" Task will NOT be compatible.

                                Assuming we have a "ui" attribute on the Task object, in the two versions of our ProcessDefinition we would see something like this:

                                The first version of the "order" ProcessDefintion was originally deployed with all Task ui attributes set to 'ui=1'.
                                Knowing the UI requirements are changing for the "shipIt" Task, the new version of the "order" ProcessDefintion is deployed with the "shipIt" Task's ui attribute set to '2'

                                <task name="shipIt" ui="2">

                                On the UI side, the developer could utilize this information many ways; here is just one implementation example:

                                The developer stores screens in a folder structure similar to ".../tasks/{ProcessDefinitionName}/{TaskName}/{TaskUi}.xhtml".
                                The UI is responsible to provide a screen for each Task/ui requirement, so before deploying the new version of "order" the developer would create a new screen
                                .../tasks/order/shipIt/2.xhtml


                                Now from the UI application, the user selects a task from their work queue. The UI fetches the TaskInstance, examines the associated Task.getUi() value and loads the appropriate screen.

                                In this example, the developer creates a new JSF/Facelets/whatever page and backing beans for each unique ProcessDefiniton_name/Task_name/Task_ui combination; however, the same Task.ui data could be used just as well by screen configuration files or even a content management system. I'm sure that each person here could come up with a totally different and equally viable way to utilize the Task.ui data in their own UI.


                                As for the three other pieces of data I mentioned...
                                * The "ui" attribute on the ProcessDefinition itself can be used in exactly the same ways as mentioned above for identifying appropriate versions of Process summary screens.
                                * The deployment timestamp on the ProcessDefintion is just a very simple change that can make at-a-glance trouble shooting or answering some questions VERY easy.
                                * The release number on the ProcessDefinition makes so much sense I don't even know where to start--Every major software product records and displays a developer-provided release number--and remember that long running jBPM processes and their definitions exist outside the scope of any running application (or database for that matter)--they need to carry developer-provided versioning info just like any other solution.

                                All four of these items are simple attribute adds, with no impact to existing functionality.

                                • 13. Re: Potential enhancement to jbpm
                                  camunda

                                  I agree with Britt.

                                  We developed a Swing framework for jbpm UI's some time ago ([utl]http://www.camunda.com/toolkit_for_jbpm/tk4jbpm_overview.html[/url]). There it would be also easy to support such a versioning of the gui IN the framework.

                                  Currently we do that in one clients application by changing Node names to "ShipIt.2", so we can configure a different panel for it.

                                  But Britt's solution is much nicer!

                                  Cheers
                                  Bernd

                                  • 14. Re: Potential enhancement to jbpm
                                    tom.baeyens

                                    Let's see if i get this right.

                                    In jPDL 4 i want to add annotations. Basically, annotations are key-value pairs that you can associate to all process elements.

                                    As extensions would like more infomation inside of the process (UI forms, simulation, prediction,...) they can be added in a standard way without the need to write a parser for it.

                                    e.g.

                                    <task name="ship">
                                     <annotations>
                                     <string name="task.form" value="ship.xhtml" />
                                     ...
                                     </annotations>
                                    </task>


                                    in jPDL 4, there will be one way on how to write objects in xml. it will be used here for annotations, but also for the configuration file and to specify initial values for variables.

                                    alternatively users should be able to write their own parsers to extract the information from a process archive and put that information somewhere in the process definition.

                                    does this feature match the requirement that you both are explaining ?

                                    1 2 Previous Next