Skip navigation
1 2 Previous Next

Tom Baeyens's Blog

23 posts

 

Let’s have a look at four example scenarios that could benefit from a workflow, BPM or orchestration engine.

  1.  

    Workflow: A simple web application that is responsible for managing tasks for people. Imagine an issue tracker where each type of starts a different type of workflow that involves several people. Or a DVD-store that manages the tasks done by people in the back-end after a sale has been submitted.

  2.  

    Pageflow: A web application is made up of multiple pages. Most web frameworks have a mechanism of invoking a piece of user code, and then the user code returns the outcome. Based on the outcome, the web framework will show the next page. This maps very well on finite state machine diagrams. In some frameworks, this is called workflow, but I prefer the term pageflow. The added value of a pageflow engine in this context is that a pageflow process shows the overall picture of the web application. Without this overview, developers will have to scan through the pages, the invoked user code and the outcome to find out how the application is working.

  3.  

    Legacy messaging: Legacy applications are often connected with newly developed applications by means of asynchronous messaging. Until recently, this kind of integration was often based on a proprietary protocol instead of the XML based technologies that are available now. In such an environment it can be a challenge for an application developer to keep track of what messages were sent and what messages are expected from other applications. That is another common situation where a state machine based workflow engine can be applied.

  4.  

    Service orchestration: Suppose that you have already standardized your organisation on web services technology. All of the departments communicate over one backbone on which XML messages are exchanged asynchronously. (By the way, such a backbone is called a Enterprise Service Bus (ESB) and you should only use JBI-compliant implementations of such an ESB to make sure that you can by components from different vendors). Now suppose that you want to write a new web service as a function of existing web services. That’s a domain where the Business Process Execution Language (BPEL) is at its best. A BPEL engine is a component in an ESB-architecture. A BPEL processes is a service that is a combination of existing services.

 

The first point I want to make in this blog is that one process language is not enough. All attempts to find the ultimate process language have failed. Instead, each of the scenarios require their own process language. This is a very important statement because at this moment, some people still try to push BPEL as the ultimate process language. This perception is also fuelled by the fact that no consensus is being reached on other process languages as described in the other scenarios.

 

BPEL is the only standard that indeed gets consensus in one particular application domain. And BPEL is a necessary component of an ESB. But BPEL is not suitable in the other scenarios. The first 3 scenarios do not lend themselves to be handled by XML based technologies like web services. Many projects are realized with a simple .war web application that is deployed on Tomcat. Using BPEL in those environments just doesn’t make any sense.

 

The second point I want to make is that we should be searching for the ultimate foundation for all of these workflow and BPM engines, rather then searching for the ultimate process language. We started by identifying the two common functions that can be found in all workflow and BPM engines:

  • Managing state: Processes are able to cope with wait states. This is something for which plain java just doesn’t have the proper support. See What’s wrong with the JVM
  • Graphical representation: A graphical representation of processes can create a common language between business analysts and technical developers. As described before, this is another point that cannot be done with plain java.


 

After finding those two common functions of all workflow and BPM engines, we distilled a very simple approach to adding support for these functions on top of an Object Oriented Programming (OOP) language. You can find more information about it at the JBoss jBPM documentation.

 

These foundations are more then a theoretical exercise. In JBoss jBPM we already support 3 process languages today. Each of those languages are targeted at very different environments:

  1. jBPM Process Definition Language (jPDL) : The core jBPM process language that focusses on task management and a clean integration with Java. All of this is packaged as a simple .jar Java library that can be used in simple web applications.
  2. BPEL: Our BPEL product is an extension of the base jBPM library.
  3. pageflow: In JBoss SEAM, jBPM is used for describing the pageflow of web applications.


 

The conclusion is that there are different types of workflow, BPM and orchestration. Each scenario requires its own process language. We have found Graph Oriented Programming to be a very solid basis for creating those process languages.

 

The typical BPM demo is actually hilarious. It gives an impression that is completely wrong: A business analyst draws a diagram of a simple hello world process, presses a button and a working system comes out. “Look, Ma! No coding.” Managers get absolutely wild when they see this… “So I can get rid of my unpredictable and slow developers?” Regrettably for the managers, nothing could be further from the truth.

Roles

 

Essentially, the development of business processes with a BPM system is not that different from regular software development. Though, there is one aspect that I would like to emphasize: Developing software with a BPM system can significantly improve the communication between the requirement’s stakeholders, the business analysts and the software developers.

 

In the development of software to support business processes, we can distinct 2 roles: The Business Analyst (BA) and the software developer. While these are 2 clearly distinct roles, note that in smaller organisations, the same person in an organisation can fulfill them.

 

In larger organisations, there is more specialization and the function business analyst and software developer is split up into separate full time positions. Even when one person does the business analysis and the technical implementation, the insights on how the roles work together leads to better quality of the software.

Executable business processes

 

To talk about how the roles of Business Analyst (BA) and developer interact, let’s first differentiate between abstract business processes and executable business processes. A business analyst is only concerned with the abstract business process. This includes mainly the graphical diagram. It does not include all the technical details needed to make the process run in a production environment. Let’s define an execution business process as a process that includes all these details so that it is a self-contained package that can be executed in a BPM engine.

 

The JBoss jBPM team thinks it is important that the business analyst is in control of the graphical representation of the business process. The graphical view of the business process should be a projection of the executable business process. Meaning that the graphical view should hide details and only show the aspects that are important to the business analyst.

 

We believe that an executable business process is a piece of software just like regular software written in an OO programming language and therefore, the same techniques of software development apply: namely iterative development and test driven development. Iterative development means that business analyst and technical developer each are responsible for different tasks in the development.

Business analysts do not create software

 

The business analyst is responsible for understanding the procedures in an organisation, how software will be used to support the procedures and documenting the way that people and software will work together. Mostly, business analysts use diagrams as the backbone for describing a business process, accompanied with a textual or sometimes oral:-( explanation. Creating a good visual representation of the business process that shows the important aspects and hides the details is a crucial factor.

 

As you might already know, all BPM systems support a graphical notation for their processes. But there’s this really stubborn myth that the goal of BPM systems is to eliminate the role of the software developer and eventually, the analyst will create executable software.

 

As we speak, this myth still is pursued by many of today’s BPM software vendors! As if the BPM systems would be able to automagically guess the best transaction-, locking-, asynchronous execution strategy and knows how to integrate with all of your enterprise’s systems... yeah right.

 

The “look ma, no single line of code for this process” works perfect for hello world examples and demos, but in real development scenarios it becomes painfully obvious that business analysts do *not* create software.

Hiding details

 

We at JBoss jBPM think that it is *vital* that The technical developer is responsible for adding technical details to the diagram to make the process executable. Developers do this in an iterative fashion, influencing the work of the business analyst. Their collaboration is now greatly improved because they have a common graphical model that they can use during their discussions. In JBoss jBPM we have created a model that allows for the business analyst’s important ideas to be expressed in the graphical diagram. But –similar as with UML class diagrams— developers must be able to hide the technical details for turning the graphical model into an executable piece of software.

 

Separating the responsibilities has an important implication. It means that the graphical view of a process should be decoupled from the technical details that have to be added by the technical developer. Most of today’s BPM systems fail at this.

 

If a developer needs to add e.g. a technical database update in which the business analyst is not interested, these systems still require the graphical model to be changed. That is unacceptable. A developer must be able to add details and implementation *without* touching the graphical diagram that is used as communication between the BA and the developer. In one of the later blogs we will refer to “Graph Oriented Programming” as a simple technique to achieve this decoupling.

 

Have you ever wondered why Business Process Management (BPM) ‘engines’ need a separate runtime server ? Why is a plain JVM not sufficient as a runtime server for business processes ?

 

The best way to see the problems that BPM solutions try to solve is by identifying the problems with the current generation of Object Oriented (OO) programming languages. Current generation of programming languages is lacking a two crucial features :

  • no support for long running processes
  • no visual representation


The limitations of the JVM

 

Let’s look at those features a little closer. First of all support for long running processes. Consider all requests handled by a server related to the processing of one order. Typically, there are quite a few. So there is a relation between the different requests to the server. A first request might be from a customer that creates a new order via the public web application. A second request might be from a employee that verifies and approves the order for further processing. The third request might be a response from the warehouse to confirm the stock availability and so on.

 

With support for long running processes, we mean that we want to be able to write a piece of software that covers this whole scenario, including the wait states and how requests and execution of software is combined. Let’s call that kind of software that expresses a long running process and includes wait states “executable business processes”.

 

The basis of OO programming languages is that they allow a programmer to express a sequence of instructions. That is very good to serve requests. Meaning, a request comes into the server, then the software executes as fast as possible to perform the request. And when the server has completed the request, the server waits for a next request.

 

But OO programming languages cannot express programs that span wait states. It is impossible to write an instruction as for instance: “wait for an asynchronous message from another system” or “wait for that user to complete this task”. Ok… yes, it is possible to write that kind of instruction, but the problem is that the execution cannot be persisted by the programming language during the wait state. So if your thread is blocked in such a wait-instruction and you reboot the server, you have lost your execution. To be precise: programming languages don’t support persistence of the execution during wait states.

What do BPM engines add to plain OOP ?

 

So the first goal of BPM solutions is to support the specification of executable business processes that can include wait states. This capability brings the specification of software much closer to the requirements that originated from optimizing business processes with IT. Without a BPM solution, the state of the overall process and the relation between different automated functions would have to be managed by the programmer in each request manually.

 

The second goal of BPM solutions is support a graphical view for the executable business processes. This is important because it creates a common language between the business analyst and the technical developer. Furthermore, it gives the programmer an extra technique to structure software.

Another technique to structure software

 

Structured programming added abstraction, Object Oriented Programming (OOP) grouped data with code and Aspect Oriented Programming (AOP) groups crosscutting logic. Those are all techniques to structure our software. Structuring software in a proper way can lead to considerable cost and time savings.

 

All of those techniques attempt to lower the complexity of software development. Since the cost of software development grows exponentially with the complexity, it becomes clear that all techniques to structure software are important. BPM solutions give the programmer the ability to structure the software around a graph.

Conclusion

 

The conclusion of this is that OO programming languages are not suited to specify long running processes that include wait states. BPM engines provide developers with a process language that can express long running processes.

 

The runtime part of a BPM system keeps track of the state of every process execution on a persistent storage such as a database. The net result for developers is that they are given another technique to structure their software around the business requirements.

 

So developers can spend more time trying to understand the requirements of the problem domain, rather then finding a technical solution for managing the state of each business process manually.

 

Many developers are still sceptical about the benefits Business Process Management (BPM) systems and workflow engines. BPM visionaries and management executives seem unable to pass their enthusiasm about BPM to developers. In a couple of blogs, I will show what the essence is of this technology and how developers can take advantage of it. In this first blog, we’ll sketch the context of BPM and what it brings to the table for developers. Then, in the next post, we’ll show the limitations of the JVM that BPM solutions are trying to resolve. Further blogs in this series will clear some of the myths and misunderstandings about BPM.Especially the relation between BPM and integration is often misunderstood (even by many of the BPM vendors:-). Also, we'll cover BPM in relation to other buzzword technologies like SOA, ESB, Orchestration, and BPEL.

Opinions on BPM systems

 

Opinions on the necessity of BPM systems vary from “It’s always necessary... I couldn’t write software without it” to “up to now, I was able to implement my requirements without a BPM system so I don’t see why I would need one”. So I think it’s time to learn about what BPM is, what the typical myths are and how developers can benefit from it.

 

First, let’s look at the most cited pros and cons. The most often heard benefit is business process agility. But for a typical developer this gets interpreted as “PR-fluff”. In this series of blogs I want to show that there are actually some real benefits for developers as well.

 

Most often heard disadvantages are:

  • “I don’t know it, so I don’t like it” ok... let’s do something about that with these blogs
  • “I just have simple workflow requirements, a workflow or BPM engine is overkill for me” This is because BPM systems tend to be a pain to integrate in simple web- and other applications. Also there is this myth that BPM has to be build on top of XML and web services technology. Working with WSDL, XML and web services is indeed not very light-weight, but what a few people seem to know is that the BPM features can be offered in a plain Java environment *without* any dependency on WSDL and related service oriented technologies. It is a matter of focus. In JBoss jBPM we chosen resolute for simplifying BPM on the Java platform. Offering all of the BPM features in an easy accessible way for the Java developer, ranging from the simplest applications, over common web applications all the way up to a high-throughput, clustered server solution.

 

To put all of this in the right perspective, we are going to start with a look at what BPM is and in which context it can have significant advantages for Java developers.

BPM Introduction

 

Explaining Business Process Management (BPM) makes more sense if we start with providing a working definition of a business process : A business process is a recurring procedure in an organisation to achieve a goal.

 

Without explicit management, business processes tend to grow organically. New procedures have to be created or invented. Mostly, this is done informally, without documenting or thinking about it. Basically, people that are concerned with the goals of the organisation, do a set of activities in order to reach the organisation goals. When many of those activities start to be done frequently, we can speak of a procedure.

 

After procedures are formed, small local changes occur quite frequently. Many small local changes to procedures can lead to overall inefficiencies. BPM can now be defined as the effort to improve the efficiency of an organisation by examining and improving the procedures that are used to reach the organisation’s goals.

IT can improve efficiency of business processes

 

In this section, we’ll explain how the above definition relates to the process management and workflow software packages that can be found in today’s market.

 

In current midsize and large organisations, there is a large amount of information that needs to be processed to execute the business processes. So the big opportunity is to make this processing much more efficient with proper use of IT technology.

 

Developing software for supporting business processes in *not* different from any other software development. So what is then the purpose of the BPM and workflow solutions out there, all with graphical designers and special runtime servers ?

 

Current Object Oriented Programming (OOP) languages are not very suited for defining long running processes. For example the handling of an order through an order processing system. There are many different automated steps in processing one order through the system. In traditional OOP, the state of the order has to be managed manually. What this means is that the developer must add a field in the database that reflects the current state of the order. This state must then be checked and updated in every request related the order. Especially when multiple concurrent paths of execution are involved, this can be complex to think about and hard to debug.

Developers should care

 

Managing the state of long running processes is the key feature why Java developers should care about BPM systems. It abstracts the state management from the business logic and relieves the developer from a tedious and error-prone job. With this capability the developer can structure the software around the overall process (e.g. the order process). This brings the software design much more in sync with the software requirements.

 

Another reason why developers should learn about the BPM technology is the automatic audit trail logging that is included in all the BPM engines. These audit trails provide detailed information about each step in each process. These process logs plus the statistics that can be derived from those logs gets a programmer from the BPM system as an extra. Note that the statistics are highly valued by managers since it allows them to detect bottlenecks and inefficient steps in the processes.

 

The next blogs in the series will dive deeper in the missing features of OOP languages and how we can make the BPM feature set easily available to Java developers.

 

regards, tom.

 

The last few years, XML has become the preferred format to exchange information between heterogeneous systems. Conversions between the XML format and the programming data structures like java objects can be a bit slow and those conversions are hard to develop. This creates a virtual barrier between the java world and the XML world.

 

Of course, developers wouldn't be developers if they didn't find a way around this little problem. In a first wave, technologies like XPath and XSLT were created. So with those technologies at hand, developers didn't need to convert the XML to an Object Oriented Programming (OOP) environment to do the transformation, that way avoiding the barrier.

 

The second wave was WSDL, SOAP and web services. First the focus was on RPC-style web services, now a shift is finalizing towards document style web services. Once this dust was settled the net result was a standard interface language for XML based integration.

 

Recently, a third wave has started to unfold that builds on top of the WSDL interface language. On the XML specifications front, there's the WS-* specifications, of which WS-BPEL is the one that most clearly shows the trend. In Java land there is Java Business Integration (JBI). Let's take a closer look at those two.

 

The basis of WS-BPEL can best be described as an XML scripting language that has WSDL service invocations as its primitive operations. Simply spoken, you write a function in XML that is comprised of control constructs and WSDL operation invocations. That function can now be exposed as a new WSDL service. This is of course oversimplified. In reality, WS-BPEL adds a lot of syntactic sugar on top of this. But you can sense that this technology allows you to do some processing/programming in the XML world without crossing the barrier of XML conversion to Object Oriented Programming (OOP) objects.

 

Java Business Integration (JBI) specifies a component platform for WSDL based services. It has a similar structure as other component technologies like e.g. JMX, OSGi, Session EJBs. But the difference is that JBI targets this bus and component structure in the XML world. This is achieved by using WSDL as the interface language for the components (Service Engines).

 

This third wave clearly marks a trend towards a complete programming paradigm in the XML landscape. So we started off by avoiding the XML parsing problem. But now, the XML programming has introduced a new problem. Development of XML software is not as easy as we'ld hoped for. In traditional OO programming languages we have powerfull debuggers, unit testing, refactoring and more. Will that be the next step in this XML evolution ? Duplicating that effort in the XML landscape ?

 

Currently the $64 question that this trend puts forward is "What is the natural balance between OOP and XML programming ?". What kind of programming should we be doing in plain OOP and what processing is appropriate in the XML world ?

 

Stay tuned here on this blog for my answer on these questions in my next blog: "My appreciation of the Enterprise Integration Patterns".

 

Here's a retrospective to the JBoss jBPM activities and feedback we received at this year's JavaOne conference.

 

Workflow and BPM systems have been around for a while now and orchestration engines are mushrooming all over the place. Despite the fact that all of these systems have similar goals, it is impossible to find 2 concrete solutions that are based on a common foundation. Each solution is a unique mix of functionalities. Unique mixes might be great at a coctail party but regrettably the analogy doesn't stand. You can safely trust us on this one, we seached hard but couldn't to find that analogy...

 

This fragmentation of the workflow, BPM and orchestration landscape leaves the industry in an orphaned position. Without common foundations model, there is no mindshare and almost no progress. The value of such a foundations model can be best understood if we look at the database world. There, you have 3 well known paradigms: the relational model, hierarchical databases and object orietented databases. No such model existed yet for workflow, BPM and orchestration.

 

Many people didn't realize such a model was missing and only felt confused by the diversity of the BPM offerings. In my session, we launched the idea of a unified model: Graph Oriented Programming. We had massive positive feedback after the talk and at the booth. People were very enthousiast and hopefull that this new model will bring unification and thereby opening up a whole new array of possibilities.

 

It took us 4 years of extensive research to develop the Graph Oriented Programming model. During that time, we developed a simple but powerfull kernel module. The open source community continiously challenged us and helped in defining solutions. Now, we have delivered proof that this model is capable of supporting all 3 domains: workflow, BPM and orchestration. The JBoss jBPM product is build on this model and is the only BPM engine that is able to support multiple process languages on the same kernel (plain POJO) software. JBoss jBPM has support for JPDL, which is a language focussed on managing tasklists for people and it also has a very clean integration with standard Java. But jBPM also has support for BPEL, an orchestration process language.

 

This unification in process modelling foundations also removes the necessity for multiple process engines in one software project. Instead of having to use a workflow engine and a separate orchestration engine, you now can just use the workflow extension and the orchestration extension of the same product.

 

So don't wait any longer. JBoss jBPM version 3.0 has been released just before JavaOne and you can download it today and take a look at what this means in practice.

 

We just released jBPM BPEL extension 1.0 alpha 1. This product brings jBPM into the arena of standards-based enterprise application integration. Also, it adds value to the Web services stack of JBoss by enabling a developer to assemble individual services into a process flow. These concepts will be particularly appealing to those interested in implementing service-oriented architecture using JBoss as middleware platform.

 

This release covers five of a total of eight functional areas of BPEL, just enough to model simple to moderately complex business scenarios. The documentation bundle contains:

  • an introduction to BPEL, including the role within SOA, the industry support and the OASIS submission
  • a comparison with the jBPM process definition language, with guidelines for choosing the most appropriate language
  • instructions for getting started
  • two complete examples, a simple request/response and a moderately complex interaction of an automated teller machine with the bank business façade


 

To ease the deployment of web services implemented as processes, we supply the first iteration of a toolset for automating the generation of artifacts required in compliant WS4EE deployments. In the future, the jBPM team will collaborate with the JBossWS team to push the implementation of advanced Web services standards, such as WS-Security and WS-Addressing, in the benefit of the user communities of both projects.

 

Download jBPM BPEL extension 1.0 alpha 1

 

A big thanks and a pat on the back to desperados Alex and Juan, who made it happen. Great work, guys !

 

regards, tom.

 

Today we released JBoss jBPM 3.0 alpha 2. It gives already a quite comprehensive overview of how jBPM 3.0 final will look like. This release includes persistence and improved documentation.

 

The two areas where we are focussing on is documentation and database compatibility. This release includes about 60 pages of userguide, including getting started, tutorial, what is workflow and XML reference.

 

For improved database compatibility we created a separate package: jbpm-3.0-alpha2-db.zip Currently only hypersonic and postgresql are included, but the structure is set to add support for other databases quite easily. We encourage the community to contribute validations of jBPM against your database.

Download jBPM 3.0 alpha 2

 

Also the graphical process designer alpha 2 has been released. The graphical process designer is an eclipse plugin for authoring the jBPM business processes.

 

Download jBPM Process Designer 3.0 alpha 2

 

For questions and feedback, check out the forums.

 

Enjoy !

 

regards, tom.

1 2 Previous Next

Filter Blog

By date: