Adapter concept to fully support service versioning

[goc]

Hello,

I would like to discuss a new feature, I developed for SwitchYard [1].

My changes solve a similar issue as discussed in [2].

Essentially, my changes treat the problem of service versioning - for promoted component services.

An example [3] demonstrates the solution.

A component (OrderingServiceBean) provides certain functionality. This component is promoted twice, OrderingService and OrderingService2.

The real problem is, that the two service contracts differ, they are mutually incompatible for varios reasons.

For example the parameter datatype changed - can be solved using the SwitchYard's transformer ability.

A new method was added to the service interface ... this doesn't harm old service consumers.

But in some cases you wish to rename a method or have the ability to use another service for a certain operation - this issue is solved by my extension.

In the example, a method in service OrderingService2 was renamed. The component bean implements this service interface (Java interface).

Hence, consumers using the OrderingService will run into trouble accessing the new service.

My extension to SwitchYard is quite similar to the transformer concept. In switchyard.xml the following modification are required:

 <adapters>
 <adapt.java xmlns="urn:switchyard-config:adapter:1.0" 
 from="{urn:com.objectbay:switchyard.adapter.example:0.0.1-SNAPSHOT}OrderingService"
 to="{urn:com.objectbay:switchyard.adapter.example:0.0.1-SNAPSHOT}OrderingService2"
 class="com.objectbay.switchyard.adapter.example.adapters.OrderingServiceToOrderingService2Adapter" 
/>
</adapters>

The attributes "from" and "to" are the two services, promoted by the component's bean.

"OrderingServiceToOrderingService2Adapter" is the java class, where the user can choose the new service operation:

public ServiceOperation lookup(String consumerOperation, ServiceInterface targetInterface) {
    if ("order".equals(consumerOperation)) {
        return targetInterface.getOperation("orderIt");
    }
    return targetInterface.getOperation(consumerOperation);
}

Well, what do you think? Could this be a feature for SwitchYard?

Let's discuss further here!

Christoph Gostner

[1] https://github.com/TrueMoe/core

[2] https://community.jboss.org/message/744664

[3] https://github.com/TrueMoe/switchyard-adapter-example

[kcbabo]

Hey Christoph,

Thanks very much for taking this on and sorry for the late reply - just getting caught up after a week of full-day meetings.  I will review the code you've submitted and the earlier discussion around contract adapters and reply back by the end of the week.

cheers,

keith

[objectiser]

Hi Christoph

Was wondering how you would handle signature changes?

I guess one possibility would be where the same parameter name is used in both ops, but with different types, it could use the transformer mechanism to convert the values.

However if additional parameters are added, how would you contend with this situation? Somehow specify a default value?

Regards

Gary

[kcbabo]

However if additional parameters are added, how would you contend with this situation? Somehow specify a default value?

Service contracts in SY support a single input and (optional) output type, so I don't think we will need to address this case. :-)

The thing I like about the adapter approach is that it provides a mechanism by which a provider can introduce a new version of a contract while remaining contract-compatible with existing consumers.  In fact, I think this is the primary (perhaps only) use case for adapters.  I'm sure we could dream up some edge cases and theoretical situations, but I think it really just boils down to the responsibility of a provider to be contract-compatible with consumers.  Please let me know if I've missed something there.

[objectiser]

Keith Babo wrote:

However if additional parameters are added, how would you contend with this situation? Somehow specify a default value?

Service contracts in SY support a single input and (optional) output type, so I don't think we will need to address this case. :-)

Woops - yes I forgot that

I also like the adapter idea, as it encourages service developers to consider how their service implementation should evolve over time to support new and existing consumers. This also makes it easier from a design time governance perspective, as the service implementation identity remains constant, so easier to track its history.

[goc]

Hey Gary,

Keith is right. Services accept at most one parameter - which is ideal regarding service versioning.

Marino and Rowley outline the advantage of one parameter in their book "Understanding SCA".

They describe the situation as follows (in my words):

The service contract’s operation should only require a single parameter. If required, this parameter can be a complex data type.

The main reason for this is, that services evolve over time as new requirements arise.

From the point of view of the service consumers, it is better to modify a single parameter than adding a new parameter.

Existing service consumers will still be compatible, as the original binding is still available.

The adapter concept has nothing to do with parameters. Parameter changes can be handled using transformers.

Adapters are only neccessary to map old operation names to the new service contract.

A fact that I have not yet taken into account are exceptions.

It is possible to map a method to another method with incompatible exception types.

In case of failure, a SwitchYardException will be thrown.

@Keith: Have you already reviewed the code?

I updated my branch today, it's now on 1.0.0-SNAPSHOT - no conflicts!

Should I make a JIRA as recommended in https://community.jboss.org/wiki/DevelopmentOnSwitchYard?

[kcbabo]

I've been thinking about this off and on for a couple weeks now.  On the one hand, the idea of adapting contracts when they change is very attractive.  We have all felt the pain of introducing changes to a service contract and having to deal with fallout of all consumers for the modified service.  I absolutely agree that there is a need there w/r/t change management and service lifecycle.  On the other hand, I'm having trouble warming up to the idea of a first class construct called a contract adapter.  Maybe it will seem more natural the more I talk about it. :-)  Let's review what can happen when a contract changes:

1) An operation can be added.

2) An operation can be removed.

3) An operation name can change.

4) The input type of an operation can change.

5) The output type of an operation can change.

6) An output type can be added.

7) An output type can be removed.

8) A fault can be added.

9) A fault can be removed.

10) A fault can be changed.

First, let me check to make sure I hit all the possibilities.  Let me know if I missed something?

Working off the above list, I think we can say a few things:

1, 6, and 8 are additions and unlikely to impact service consumers.  It's true that the interaction contract has changed, but from a consumer standpoint it still accepts a message in the way it used to.  I don't think an adapter as described in this thread wil address these cases anyway.

4, 5, and 10 can be handled with transformation in today's implementation.  It does require adding a completely new transform instead of piggy-backing on an existing transform via a multi-step transform (A -> B -> C).  Incidentally, I *really* want to see multi-step transformations added to SY, so if you are interested within the context of this problem please let me know. :-)

7 is nasty and I think an adapter will likely be of limited help.  The adapter would have to create the reply message on its own, which is probably asking too much.

9 is likely to have no impact on service consumers as they simply won't receive faults of that type any longer.  More of a functional issue at that point.

3 is the one thing that could happen that transformers can't handle that an adapter can do quite well.

2 is really borderline w/r/t adapters.  You could mask a removal by dispatching to another available method perhaps.  Seems more likely that an instance of 2 is more likely to be a combination of 3 and 4 and/or 5.

Hopefully that whole numbering business isn't too confusing.  Perhaps this is a window into my mind and why I'm not quite settled on the idea yet. :-)  Based on my impressions above, it seems like the unique value of an adapter is to handle changes in an operation name.  Any other cases are difficult to handle (even with an adapter) or can be handled now with transformers and even better in the future with multi-step transformation.

This is a great conversation, so let's keep it going and get to the heart of solving this problem of reacting to contract changes.

[goc]

1, 6 and 9 won't impact existing service consumers.

8 - a fault can be added. Are you sure that existing service consumers won't be affected? (I updated my sample project [1])

3 - an operation name can change. This is already covered by the implementation. In combination with transformers (4,5, 10), also 2 might be covered, if the operation can be dispatched to another existing or new operation.

As you have already pointed out, adapter's "unique value" is to handle changes in an operation name.

7 - an output type can be removed. This will render the service operation incompatible.

If such a design decision is made, compatibility is left completely out of consideration.

But the idea of multi-step transformation is definitely interesting.

[1] https://github.com/TrueMoe/switchyard-adapter-example

[kcbabo]

Christoph Gostner wrote:

8 - a fault can be added. Are you sure that existing service consumers won't be affected? (I updated my sample project [1])

I guess it depends on whether that fault ever occurs. :-)  You're right, a new fault would break a consumer if it had no idea it was coming.  I think this can actually be broken down into two scenarios:

1) The service already returns a fault and were adding another.  We don't support multiple fault types at the moment, but this will be introduced soon(ish).

2) There was no fault before, but now a fault is added.  This is equivalent, in my mind, to the introduction of an output type when one did not exist before.  You are changing the MEP for an operation and the consumer is unlikely to do anything with the response (fault or otherwise).  In fact, in the SY runtime we check the MEP and replies on an in-only MEP should be logged at WARNING to notify the user that there's a mismatch.

As you have already pointed out, adapter's "unique value" is to handle changes in an operation name.

After this conversation, the role of an adapter is much clearer to me.  The concern/confusion I had earlier was mainly around the relationship between transformation and contract adaptation, specifically how to express to users the right situation to use one over the other.  Based on this discussion, I believe that adapters can be used in the following situations:

a) Operation name changes.

b) An operation is removed.  In my mind, this is simply an advanced form of (a) since the adapter will need to map calls to the removed operation into existing or new operations on the service interface.

c) An output type is removed.  Strictly speaking, an adapter could help with this by returning dummy data but I doubt that will be of much use.  Anyway, it's possible.

Did I miss anything in that summary?  Based on those use cases, is a distinct adapter approach a useful feature?  Personally, I have an open mind about it.  I can see this being used for sure. We definitely need to avoid confusion between the roles of adapters and transformers, however.

[kcbabo]

I would also like to get your thoughts on the lifecycle and scope of these adapters.  Based on how the conversation has played out so far, I'm guessing that adapters are usually (perhaps always) introduced by the service provider as a protective mechanism when changing a service contract.  Do adapters apply to a specific service or do they apply to all instances of a contract?

[goc]

Regarding faults, old consumers are only affected, if an exception occures.

Keith Babo wrote:

1) The service already returns a fault and were adding another.  We don't support multiple fault types at the moment, but this will be introduced soon(ish).

A possible solution could be to define a transformer, that maps the new exception to an existing one.

Keith Babo wrote:

2) There was no fault before, but now a fault is added.  This is equivalent, in my mind, to the introduction of an output type when one did not exist before.  You are changing the MEP for an operation and the consumer is unlikely to do anything with the response (fault or otherwise).  In fact, in the SY runtime we check the MEP and replies on an in-only MEP should be logged at WARNING to notify the user that there's a mismatch.

Maybe I did not quite understand.

I think a change of the MEP is not required (if an output type is added). Consider the following service operations:

void outputTypeAdded(Order order); // version 1

Order2 outputTypeAdded(Order2 order); // version 2

Consumers, using version 1 of the service operation, will use an InOnly MEP, in test: service.operation("outputTypeAdded").sendInOnly(message);

Consumers, using version 2 of the service operation, will use an InOut MEP, in test: Message response = service.operation("outputTypeAdded").sendInOut(message);

So, a change of the MEP is not required, since the consumer actually doesn't recognises the change in the service contract.

Please tell me if I'm wrong.

A clear description of both concepts is important. This will be an interesting chapter in my master thesis.

a) and b) are covered by the implementation as it is currently.

c) is certainly worth considering but is not covered yet. Here, the MEP will change.

Resuming the example from 2), the adapter will be applied, when a service consumer tries to access the service using version 1 of the service contract.

A consumer of the service contract 2 won't be affected by the adapter (except for the check, that is required to see if an adapter is defined for the consumer).

private ServiceResolver selectServiceResolver(QName name) {

    if (_adapterRegistry.hasAdapter(name))

        return _adapterResolver;

    return _defaultResolver;

}

So, an adapter is only used for the specified, older version of a service contract.

As that, the deciding factor is the service contract of the consumer.

Keith Babo wrote:

Based on how the conversation has played out so far, I'm guessing that adapters are usually (perhaps always) introduced by the service provider as a protective mechanism when changing a service contract.

Exactly

[goc]

I created a table to summarize the results of this conversation:

	concept	consumers of old service contract	consumers of new service contract
An operation can be added.	-	consumer will not be affected	can use the new operation
An operation can be removed.	adapter	operation is no longer available, adapter can be used to map to another operation	can’t use removed operation
An operation name can change.	adapter	operation is no longer available, adapter can be used to map to the new titled operation	can use the new titled operation
The input type of an operation can change.	transformer	a transformer can handle type changes.	Can use the new input type
The output type of an operation can change.	transformer	a transformer can handle type changes.	Retrieves the new output type
An output type can be added.	-	an additional output type will not break compatibility	Can use the new output type
An output type can be removed.	adapter?	a dummy return value could be created	Output type no longer available
A fault can be added.	-	might break interoperability	Consumer has handle fault
An additional fault type can be added	-	currently not supported by SwitchYard	currently not supported by SwitchYard
A fault can be removed.	-	the fault will no longer occur, also, no changes are necessary	no faults have to be treated
A fault can be changed.	transformer?	Could be implemented using a transformer*	Consumer has to handle new fault type

* transformation of exceptions seems not to work correctly.

[kcbabo]

Did you run into an issue with testing fault transformation?  That should work.  I agree with the contents of the table.

[goc]

Keith Babo wrote:

Did you run into an issue with testing fault transformation?

In my tests, exceptions are not transformed. In my test-project [1], the tests fail. Also, when I'm using clean version of SwitchYard directly from github.

I disabled the failing tests for the moment.

https://github.com/TrueMoe/switchyard-adapter-example

[goc]

I didn't find a testcase, which demonstrates a fault transformation.

So I created an example on my own [1].

There is a service contract for the consumer where an operation throws a ConsumerSideFault, the provider side throws a ProviderSideFault.

Next to the fault transformer:

public class FaultTransformer {
    @Transformer
    public ConsumerSideFault transform(ProviderSideFault fault) {
        return new ConsumerSideFault(fault);
    }
}

I had to create another one:

public class HandlerExceptionTransformer {
    @Transformer
    public ProviderSideFault transform(HandlerException fault) {
        return (ProviderSideFault) fault.getCause();
    }
}

Without this second transformer, the first transformer isn't called.

Unfortunately, this doesn't solve the issue in the adapter example.

[1] https://github.com/TrueMoe/switchyard-fault-example

Edit: I have to correct myself ... It solved the problem - I had to set the fault type explicitly:

QName faultType = JavaService.toMessageType(FaultV1.class);
service.operation("faultCanChange").expectedFaultType(faultType).sendInOut(null);