BPEL (Riftsaw) Integration

[rcernich]

Hey all,

I'm currently working on adding support for BPEL components to the SwitchYard tools, which means I've also been learning about BPEL and how it is used within SwitchYard.

To summarize what I've learned, here's what is needed to use BPEL within a SwitchYard application:

1. WSDL which defines the interface for the service.  This must also include a port definition for use by deploy.xml
2. Artifacts WSDL which adds partner links to the interface WSDL.
3. deploy.xml which identifies partner links for publish/invoke for each process.

Adding to that, the type used for "myRole" aligns with the service interface, while other roles align with interfaces on the reference.  Knowing what we know, it seems like we should be able to generate items two and three as part of configure or during deployment.  I'm no WSDL or BPEL or ODE expert, but it seems like we should be able to use the partner links in the BPEL plus the services and references to create the artifacts wsdl and the deploy file.  That said, I'm guessing the BPEL tools will require the artifacts file (they generate one during creation), so generating the artifacts file during build/deployment may not be the best thing from a usability perspective.

Also, I don't think it makes sense to require the interface wsdl to define ports.  We aren't really using them.  I don't know how hacky this is, but maybe we could just specify the qname for the porttype instead and use the switchyard service identifier for the service.

Anyway, just throwing this out there to see what you guys think.

Rob

[objectiser]

Hi Rob

Rob Cernich wrote:

I'm no WSDL or BPEL or ODE expert, but it seems like we should be able to use the partner links in the BPEL plus the services and references to create the artifacts wsdl and the deploy file.  That said, I'm guessing the BPEL tools will require the artifacts file (they generate one during creation), so generating the artifacts file during build/deployment may not be the best thing from a usability perspective.

Probably best for Bob to comment on the BPEL tools and the artifacts file, but we need to take into account situations where the partner link types are defined in the same wsdl as the port type and port.

Regarding the deploy.xml, we have recently had a jira (https://issues.jboss.org/browse/SWITCHYARD-1283) that requested just that. So if the tooling can handle that it would be great. However we also must deal with cases where the deploy.xml is provided by the user, as this file is also used to set audit/logging information.

Rob Cernich wrote:

Also, I don't think it makes sense to require the interface wsdl to define ports.  We aren't really using them.  I don't know how hacky this is, but maybe we could just specify the qname for the porttype instead and use the switchyard service identifier for the service.

I think this would impact the ODE BPEL execution.

Regards

Gary

[rcernich]

Hey Gary,

Gary Brown wrote:

Hi Rob

Rob Cernich wrote:

I'm no WSDL or BPEL or ODE expert, but it seems like we should be able to use the partner links in the BPEL plus the services and references to create the artifacts wsdl and the deploy file.  That said, I'm guessing the BPEL tools will require the artifacts file (they generate one during creation), so generating the artifacts file during build/deployment may not be the best thing from a usability perspective.

Probably best for Bob to comment on the BPEL tools and the artifacts file, but we need to take into account situations where the partner link types are defined in the same wsdl as the port type and port.

Regarding the deploy.xml, we have recently had a jira (https://issues.jboss.org/browse/SWITCHYARD-1283) that requested just that. So if the tooling can handle that it would be great. However we also must deal with cases where the deploy.xml is provided by the user, as this file is also used to set audit/logging information.

It would be nice if these details could be configured in the SwitchYard configuration (i.e. switchyard.xml), so there is a single place where all configuration information gets stored/edited.  We already have a trace setting in switchyard.xml and I wonder if, when that is set, we could apply it to the component infrastructure as well (as opposed to having a separate log/trace/debug setting for each component).

I was doing a little more thinking about this over the weekend and put together a summary:

1. Use single partner link type which defines all roles used by the service.
2. Component service/reference names should map to partner link role names.
3. Component service/reference interface should mape to partner link role portTypes.
   1. myRole maps to service name
   2. partnerRole maps to reference name
4. deploy.xml
   1. provide maps to service partner link
   2. invoke maps to reference partner link
   3. provide/invoke service@name maps to switchyard service/reference qname
   4. provide/invoke service@port maps to switchyard service/reference interface (portType)

Files required:

wsdl - defines interface implemented by process

bpel - defines process, references artifacts wsdl defining partner link type

artifacts wsdl - adorns interface wsdl with partner link type

switchyard.xml - defines service/reference interfaces provided/required by process

Files generated:

deploy.xml - uses information from bpel, artifacts wsdl and switchyard.xml to create process elements and their child provide/invoke elements

BPEL component activator will need to be modified so the ServiceLocator treats service@name as switchyard service/reference qname.  I suspect service@port can effectively be ignored, but might be useful for resolving a specific named service.

Obviously, there's nothing that says the interface and artifacts wsdl need to be separate files, but I think that's how our tools would operate, as I think it provides the best flow for supporting both top-down (i.e. wsdl provided) and bottom-up scenarios (i.e. wsdl generated).  It also prevents implementation details from bleeding out in the bottom-up scenario (e.g. no partner links in wsdl for a SOAP endpoint).

Anyway, just some ideas to chew on.

Best,

Rob

[objectiser]

One question - "Component service/reference names should map to partner link role names." - does this imply that users need to know that when creating the BPEL process, that the role names must match the service/reference names?

Might be better if you could outline the steps the BPEL developer would take, whether starting from scratch using the tools, or if they are looking to deploy existing BPEL process/wsdl(with partner link types) and possibly deploy.xml?

It might also be useful if you could provide a modified version of the loan_approval quickstart, with a version of the deploy.xml that you would expect to generate, so that we can see what issues the runtime may have?

Regards

Gary

[rcernich]

Hey Gary,

Thanks for the feedback.

Gary Brown wrote:

One question - "Component service/reference names should map to partner link role names." - does this imply that users need to know that when creating the BPEL process, that the role names must match the service/reference names?

I think this cuts both way, so if it were an existing BPEL, I would assume the component service/reference names matched the existing role names.  This really isn't any different than the way the service/references are defined in a Java class for a bean service (e.g. @Refence(name="MyReference")).

Might be better if you could outline the steps the BPEL developer would take, whether starting from scratch using the tools, or if they are looking to deploy existing BPEL process/wsdl(with partner link types) and possibly deploy.xml?

I'll try to scratch something up when I get some time.  Is there a reason we would support using an existing deploy.xml?  Also, is it possible to deploy an existing BPEL that runs natively on Riftsaw directly within a SwitchYard application?  I think there's a difference between using BPEL to implement a SwitchYard component and a more generic BPEL process.  That said, I'm far from a BPEL expert.

It might also be useful if you could provide a modified version of the loan_approval quickstart, with a version of the deploy.xml that you would expect to generate, so that we can see what issues the runtime may have?

I'll mock up the existing quickstart when I get a chance.

Best,

Rob

[objectiser]

I believe it should be possible to use existing BPEL, WSDL and deploy.xml in a switchyard app, just defining the switchyard.xml. The only constraint is that only doc/lit is supported, whereas a lot of the existing riftsaw examples were rpc.

Regards

Gary

[rcernich]

Hey Gary,

Would you need to redefine the endpoints in SwitchYard (i.e. promote and add service bindings)?  Or would the process be exposed natively by Riftsaw, but also made accessible, internally, to other components within SwitchYard (e.g. a to a bean or camel route via a reference)?  (Just trying to understand the mechanics.)

Best,

Rob

[objectiser]

Riftsaw in switchyard does not support its own bindings, only via switchyard. So its possible that some of the endpoint URLs would need to change, as switchyard has fixed URLs i.e. in cases where two BPEL processes were defined, one as a client of the other.

The only other thing to note is that if two communicating BPEL processes are deployed to ODE, it will use an internal communication channel - so to avoid this, and ensure that all comms go via switchyard, the deploy.xml has to define a usePeer2Peer="false" property.

Regards

Gary

[kcbabo]

This has been a productive conversation, so it's about time I jump in and try to change all that. ;-)

We need to support two scenarios for dealing with BPEL processes:

1. From scratch : user wants to create a new service implementation based on BPEL.  In this scenario, we need to make it as easy as possible for users to get up and running quickly.  As it turns out, a BPEL service requires quite a bit of metadata (WSDL, XSD, partnerLink, deploy.xml).  If we can find a way to generate most/all of this (or optimize it away in the case of deploy.xml) as part of new service creation, then that's great.
2. Existing BPEL : in this case, the user comes to the party with BPEL and likely has some other artifacts as well (WSDLs with partnerLinks, deploy.xml, BPEL process definition).  For this scenario, it's all about making things "line up". We can't be overly proscriptive in terms of how the user designs their BPEL and whether they use ODE-specific metadata in deploy.xml.  On this path, the tooling might not do much other than provide the correct details in the implementation.bpel config and creating the component service.  After this, the user is responsible for evolving the service configuration to match up references to partner links.

Some other random thoughts based on points made in the thread:

• Although the vast majority of BPEL services will invoke other services (that's the point after all), we shouldn't necessarily assume that the user will define any/all references and partnerLinks during new service creation.
• Related to the above, we need to balance the amount of information required to create a new service vs. generating stuff the user doesn't need.  Perhaps this is addressed with the dreaded multi-page wizard with optional extra pages (finish available on first page).
• We don't have to generate deploy.xml if we can configure this in another way, but we can't discard it altogether unless we can completely replace all ODE configuration possible through deploy.xml.  That could be a maintenance nightmare and is probably not the friendliest thing for existing Riftsaw-based applications moving to SY.

[rcernich]

Here's what I've come up with, which should handle both cases:

• For "provide" entries, service name will exactly match the component service name in switchyard.xml
• For "invoke" entries, service names will exactly match the component reference names in switchyard.xml
• The "port" attribute is ignored.
• usePeer2Peer must be false for invoke elements which describe component references.

Basically, what this means is that for any BPEL component, the deploy.xml will have provide/invoke elements which correlate directly with with each service/reference on the component (i.e. the BPEL endpoint _is_ a SwitchYard service).  This alleviates the need to specify services and ports in the wsdl files associated with the process, allowing processes that implement an abstract service interface (i.e. portType) to be used as SwitchYard components, without extra configuration.

That said, I'm guessing you could still access external processes by specifying invoke entries without usePeer2Peer=false, but I haven't tested this scenario.  However, if this is not a requirement (the ability to access partner links not hosted by SwitchYard), we should be able to generate the deploy.xml file for the user.  If we decide to do this, I propose adding configuration properties to our bpel component configuration to support configuration of additional details (e.g. properties, in-memory, etc.).

Attached is a sample project based on the loan service quickstart.

Here's a link to the component changes:  https://github.com/rcernich/components/tree/bpel_deploy

Let me know what you think.

[objectiser]

Hi Rob

Was trying to build the example, but got:

[ERROR] Failed to execute goal org.switchyard:switchyard-plugin:0.8.0-SNAPSHOT:configure (default) on project switchyard-bpel-example: Model [org.switchyard.config.model.switchyard.v1.V1SwitchYardModel] is invalid: UndeclaredPrefix: Cannot resolve 'loanService:LoanService.bpel' as a QName: the prefix 'loanService' is not declared. -> [Help 1]

Any idea?

Seems strange, as the prefix is definitely defined. Have also updated all repos (except components where I cloned yours), but didn't seem to sort it out.

In terms of the config from the deploy.xml, not sure if you have already seen, but it is documented in the filtering section of this page: https://ode.apache.org/ode-execution-events.html

By default we use "<dd:process-events generate="all"/>". Not sure we need the scope level logging, but might find some users use it, in which case it would be good to understand how it could be supported later.

For the list of enable-event types, possibly it could just be a comma separated list. But generally I think we would just want the 'all' or 'none', and then have a third value which indicaes a list will be provided?

Regards

Gary

[objectiser]

Hi Rob

Looks like the switchyard.xml processing has removed the namespace definition for 'loanService'. The src/main/resources/META-INF version has it, but the expanded target/classes/META-INF version doesn't.

The prefix is only used in the bpel impl's process field, so looks like it has been filtered out.

Regards

Gary

[rcernich]

Hey Gary,

Sorry about that.  There's a bug in the configuration mojo which causes "unused" namespace declarations to be stripped from the xml.  Here's the JIRA: https://issues.jboss.org/browse/SWITCHYARD-1299

You might get it to work by hand editing the switchyard.xml, moving the namespace declaration onto the implementation.bpel element.

As for the event stuff, I just pulled that in from one of the other quickstarts.  I hadn't actually given them any thought beyond: stuff like that should probably be configured on the implementation.bpel element, so people don't need to worry about deploy.xml.

For me, this exercise was about getting more familiar with BPEL in terms of SwitchYard, so I could get the tooling right.

You can also look at the patch containing the changes to the BPEL component.  The unit tests were updated and they do pass.  It might be easier to see the differences.

Best,

Rob

[objectiser]

Hi Rob

I'll have a go trying to get it deployed - I had a look at the patch and it looked good, but wanted to play with it.

At what stage in the process were you thinking about the deploy.xml being created? One possibility is in the Eclipse tooling, but then switchyard projects created outside of Eclipse (e.g. the quickstarts) would not benefit. The other possibility is in the maven mojo, as you do with the switchyard.xml. Or were you thinking it should be in the BPEL component upon deployment?

My preference is second option (mojo) but wondered what your thoughts were?

Regards

Gary

[rcernich]

Hey Gary,

Based on what little I know, I was thinking a mojo.

It would be nice if it could just be generated by the deployer, but I think the engine requires a file, and I don't know how to support that.  You'd also need to walk the entire configuration, which means it would need to be handled outside the component activator, most likely as a separate deploy step.

I don't think generating it in the tooling is any better than what we have today, although we might be able to add some validation logic to ensure all the pieces are lined up correctly (i.e. each partner link has a corresponding component service/reference and invoke/provide).

So, I think using a mojo is the most flexible and least error prone.

That said, I think this definitely qualifies as gravy.

As I said previously, my main purpose was to better understand this to give me a better idea of what I need to do in the tooling to support BPEL components.

Best,

Rob