BPEL component for switchyard

[objectiser]

Wanted to discuss the issues that may be involved in deploying riftsaw as a component implementation in Switchyard. Switchyard uses the SCA assembly model, and may incorporate some other aspects of SCA over time.

Some of the issues that need to be resolved are:

1) Interface to BPEL implementation

The SCA spec allows components to be defined by an interface, and currently supports Java, WSDL 1.1 and WSDL 2.0 as the interface type systems.

If Java was to be used, then we would need tool support (either prior to packaging or on deployment) to create the relevant Java types for the WSDL interface.

Therefore for some deployment, support for the interface.wsdl construct would be better. Need to check whether this is supported in switchyard, or on the plans.

If we want the BPEL process to be used from both a WSDL interface and other components, then it may be good to have the WSDL to Java mapped classes available.

2) Versioning

SCA and I am assuming switchyard have no concept of versioning - dispite the fact that SCA supports stateful components. This is generally because an ESB is used for processing mediation between services, and therefore only one version of the mediation rules/policies is relevant at a time.

However BPEL process instances are potentially long lived and therefore process versioning is necessary. In RiftSaw currently versioning is handled by leaving the previous deployed process definition in place, and deploying a new jar with the new BPEL process definition as a new version number. This enables existing process instances to continue to use the old process, while new processes use the new version.

One of the issues with the current approach is that, even though the old process instances use the old version of the BPEL process, only the web services associated with the latest version are made available. So if the WSDL interface has changed, then this will impact the older process instances.

Therefore one possible solution is to package the new BPEL process, along with the older process definitions that are stil relevant, in a single switchyard deployment - with the WSDL interfaces that need to be relevant to all of the versions. This ensures that the user continues to be aware that the WSDL interface needs to be backward compatible.

One other issue is that this would mean the BPEL process definitions must be unique across all switchyard deployments, as otherwise it would cause a name conflict. Therefore if a BPEL process needs to be reused, it must be done by invoking the service made available through a particular switchyard deployment.

3) Long running processes

BPEL processes can live for a long time, and the persistence support in riftsaw can cater for this. However individual invocations may also take some time to complete. Generally this needs to be within two minutes, as this is the default timeout for the http/soap transport - so most BPEL processes will generally be written to complete a req/resp within this timescale.

However we need to understand whether switchyard would be capable of handling a large number of invocations of this nature.

4) Clustering support

Need to understand how any clustering support in switchyard would work with riftsaw. Should not be a problem though.

[jeff.yuchang]

Gary Brown wrote:

2) Versioning

SCA and I am assuming switchyard have no concept of versioning - dispite the fact that SCA supports stateful components. This is generally because an ESB is used for processing mediation between services, and therefore only one version of the mediation rules/policies is relevant at a time.

However BPEL process instances are potentially long lived and therefore process versioning is necessary. In RiftSaw currently versioning is handled by leaving the previous deployed process definition in place, and deploying a new jar with the new BPEL process definition as a new version number. This enables existing process instances to continue to use the old process, while new processes use the new version.

One of the issues with the current approach is that, even though the old process instances use the old version of the BPEL process, only the web services associated with the latest version are made available. So if the WSDL interface has changed, then this will impact the older process instances.

Therefore one possible solution is to package the new BPEL process, along with the older process definitions that are stil relevant, in a single switchyard deployment - with the WSDL interfaces that need to be relevant to all of the versions. This ensures that the user continues to be aware that the WSDL interface needs to be backward compatible.

One other issue is that this would mean the BPEL process definitions must be unique across all switchyard deployments, as otherwise it would cause a name conflict. Therefore if a BPEL process needs to be reused, it must be done by invoking the service made available through a particular switchyard deployment.

This approach seems good to me, we may can add version number into the bpel process definition name to distinguish.

Regards

Jeff

[objectiser]

The following extends the previous discussion, although focused more on the technical issues related to different possible riftsaw configurations that could be used.

Generally SCA components are defined in the context of a configuration, and are only applicable in that context.

BPEL components are slightly different as it depends on the scope of the engine and backing db. If shared with other BPEL components in different SCA composites, then there is potential for conflicts between process definition versions, and we also need to ensure messages are routed to the correct SCA composites when sent by the BPEL process instances.

So the obvious solution would be for each SCA composite, and the configured BPEL components within it, to have their own isolated BPEL engine. However this approach has its own set of issues, as outlined below:

1) Multiple engines and databases

So if we have a configuration where each BPEL component, or atleast composite (application), has its own engine and database, then isolation in terms of the application will be ideal. Each deployment does not need to care about other SCA applications.

The issue is how to handle management of the BPEL process definitions and instances from a console. In ODE, the engine and the database have information relevant to the console. If we have multiple of each, then it becomes difficult for the console to locate and manage all of this information.

It would be difficult from a deployment perspective, as each SCA/switchyard application would be required to contain the engine configuration, and the administrator would be required to manage multiple databases.

Although the isolation benefits make this option attractive, in general it is better to keep the configuration information for the BPEL engine as part of the container, managed by the container administrator, rather than within each application.

2) Multiple engines and single database

Slight variation on the previous setup.

As this configuration shares a database, the process definitions deployed across different SCA composites/applications could impact each other. Only one version of a particular process definitions would be active at any one time, so if used in multiple composites, then a version change would need to be synchronized.

On initial consideration this approach offers the benefit of an engine that is specific to the SCA application, and therefore avoids routing issues on requests issue from a process in the engine to the containing SCA application. However in reality each of the multiple engines would effectively be acting as a cluster around the single db, and therefore under certain circumstances the process instance may be loaded into another engine instance (possibly running the same process definition) and send the request message into the wrong SCA application.

So this is not really a suitable option, and therefore will be ignored.

3) Single engine and database

The various issues have been discussed in the previous two options, so we only need here to discuss what this configuration would offer.

Advantages are related to the console, being able to access active and historic information as before (in Riftsaw 2) from a single source, and the ease of database and container administration.

Disadvantages are due to no isolation, so process definitions/versions between SCA applications could impact each other, and routing of requests from processes running in the engine to the appropriate SCA application. To overcome this second issue it will be necessary for the process instance to carry context information regarding which SCA application to route messages to - to be investigated.

Thoughts:

If a single engine/db approach is used, some of the problems can be avoided by scoping the process definitions to the application by name. This may need to be manually done, so puts the burden on the user to ensure this level of isolation. Would be good if this could be achieved automatically, but may not be possible without significant changes to ODE.

Although this approach may cater for sharing process instances across SCA applications, not sure there is a use-case for this, and would be problematic, as we would not be sure where to route outgoing requests from the process instance.

In an SCA (ESB) context, its possible that more transient process definitions would be used. However this would still have the same process definition management, and outbound request routing, issues that a persistent process definition would have.

[objectiser]

Currently I think we need to work on the assumption that we have a single engine and database, and require the user to deal with process definition version issues across SCA applications. This is not that different to the user's responsibility with riftsaw 2, where they still needed to be aware of different process definitions and versions deployed within the app server.

Therefore the remaining issue that needs to be solved in this scenario is the routing of messages to the correct SCA application.

[objectiser]

Therefore the remaining issue that needs to be solved in this scenario is the routing of messages to the correct SCA application.

I think the best way to deal with this at the moment is to simply state that SCA/switchyard apps within a particular SCA domain must have distinct BPEL process names - so if it is necessary to have the same process in different composites, then they would need to have different names.

Based on having unique names, we can statically initialise an understanding of what processes have outbound references to which services.

In terms of management/database issues, this could also be on an SCA domain basis - but that is for later consideration when looking at how best to integrate the BPEL console with other Switchyard management tools.

[jeff.yuchang]

Gary Brown wrote:

Therefore the remaining issue that needs to be solved in this scenario is the routing of messages to the correct SCA application.

I think the best way to deal with this at the moment is to simply state that SCA/switchyard apps within a particular SCA domain must have distinct BPEL process names - so if it is necessary to have the same process in different composites, then they would need to have different names.

Agreed here.

As you stated in previous reply, the single engine and single database would be the best option. For now, we will ask users to check for the same process, but for the future release, we can try to add some contextual information in our database, more like following presentations.  Multi tenant for BPM. (http://www.slideshare.net/hemapani/a-multitenant-architecture-for-business-process-executions)

Regards

Jeff