SOAP header properties

[objectiser]

Have found a problem with handling of header values in the BPEL component. When a header value is included in the SOAP request header, it is passed through (assuming the context mapper is set) to the BPEL component which then passes it into the BPEL process - so that is fine.

The problem is when the response is returned. The BPEL process does not include any header values with the response, however the SOAP envelope being returned from switchyard has the header value that was previously supplied with the request.

It appears that the header values are being retained by the exchange and passed back with the response. I was originally thinking that perhaps I needed to clear them, but then on inspection of the SOAP binding code, it looks like the SOAPContextMapper uses Scope.IN in one place (related to fault info) and Scope.EXCHANGE in another.

So my question, is there a reason for the scope to be EXCHANGE? If so, I assume that I should clear the existing soap_message_header labelled header values before setting the headers for the response.

Alternatively should this be Scope.IN? In which case I'll apply the change in the SOAPContextMapper.

I've already created a jira for this issue: https://issues.jboss.org/browse/SWITCHYARD-1289

Regards

Gary

[kcbabo]

SOAP:Header entries on a request message should be mapped to Scope.IN by default.  If an implementation needs headers in the request message to be returned in the response message then it needs to set those manually.  The motivation for message-scoped properties is to not require implementations to clear properties on their own (i.e. opt in instead of opt out).

[dward]

Just so everyone is clear, right now, the SOAPContextMapper, on a request:

1. Takes the HTTP request headers and maps them as Scope.IN.
2. Takes the SOAP headers in the request and maps them as Scope.EXCHANGE.

Then, on a response:

1. Takes the Scope.OUT properties, and adds them as HTTP response headers
   
2. Takes the Scope.EXCHANGE properties, and adds them as SOAP headers in the response
   

We should be careful about making blanket changes here.

There might be implementation components that make an assumption of under which scope certain properties it's handling reside.  For example, the BPM component puts the processInstanceId into Scope.EXCHANGE so that it will end up in the SOAP message headers of the response, so that it can be included in future interactions in the request.

[kcbabo]

Unfortunately, this is problematic for a number of scenarios.  Take any case where a SOAP header is specifically tied to the request message (digital signature, message id, timestamp, etc., etc.) and returning the value in the response message doesn't make sense.  100% agree with you that some headers should be returned, but I believe that is the responsibility of an implementation to handle or via a custom context mapper (or perhaps config on a built-in context mapper if we want to go that route).

[dward]

Implementations can never be the ones to decide how properties get mapped, as they only ever see one side of it - the Context Property side.  Only the interfaces see both sides (the Context Property side AND the binding/gateway-specific side).

Am I right, then, in my understanding that the desire might - might - be that all OOTB context mappers' mapFrom should change to put EVERYTHING they get from a request into Scope.IN, and similary, that all OOTB context mappers' mapTo should change to put EVERYTHING they encounter in Scope.OUT into a response?  And ignore anything at Scope.EXCHANGE in either direction?

If we do that in one interface, we should do it in all of them, and then we'd have to comb through our implementations and fix any assumptions they have.  This was my point of why whoever handles (I said "handles" not "fixes" on purpose) that jira should be aware of the implications of such a blanket change.  However, after that at least we'd be consistent, and if you don't want consistent, you write your own context mapper.

Last, there is a questions of what part of a response to things get put in to.  Some interfaces have different "buckets" where things can go.  In our example, how do we know that a Scope.OUT variable should go inside a SOAP response message's soap header element area, vs. being an HTTP response header?  That's just one example...

[dward]

PS: By "everything they get" I mean everything they get that also matches the include/exclude rules, which by default is "exclude everything" I believe. (Hope that made sense, LOL.)

[kcbabo]

Implementations can never be the ones to decide how properties get mapped, as they only ever see one side of it - the Context Property side.  Only the interfaces see both sides (the Context Property side AND the binding/gateway-specific side).

I'm afraid I don't agree with this statement.  Implementations have access to the Context, which contains properties for IN, OUT, and EXCHANGE.  An implementation that is servicing a request is quite capable of copying an IN-scoped property to an OUT-scoped property.

Am I right, then, in my understanding that the desire might - might - be that all OOTB context mappers' mapFrom should change to put EVERYTHING they get from a request into Scope.IN, and similary, that all OOTB context mappers' mapTo should change to put EVERYTHING they encounter in Scope.OUT into a response?  And ignore anything at Scope.EXCHANGE in either direction?

Scope.EXCHANGE is never ignored.  We have the ability for component-specific context mappers for a reason, so that individual gateway can deal with headers/properties in a manner consistent with their native protocol.  I would say that absent protocol-specific context information, all request properties/headers should be mapped to Scope.IN.  An example of an Exchange.SCOPE property might be something like a conversation or session id for a specific protocol.  Scope.EXCHANGE properties should also be mapped to native response headers consistent with the context mapper configuration and the native protocol being used.

Last, there is a questions of what part of a response to things get put in to.  Some interfaces have different "buckets" where things can go.  In our example, how do we know that a Scope.OUT variable should go inside a SOAP response message's soap header element area, vs. being an HTTP response header?  That's just one example...

I thought we added the ability to select which properties are returned via a context mapper?  I've always been against the idea of mapping context properties "blindly" - in other words, grabbing everything and including it as a header.  Being able to configure a context mapper with property names (or even label names) would be a good way to go and still could allow for wildcards to grab everything.

[kcbabo]

Yes, that include/exclude stuff is the exact stuff I mentioned indirectly in my last post.  Thanks. :-)

[kcbabo]

BTW, I totally agree that everyone should be aware of this change and we want to make sure we deal with any concerns before cutting over.

[objectiser]

Ok, I'll leave this one to you guys then

[dward]

Implementations can never be the ones to decide how properties get mapped, as they only ever see one side of it - the Context Property side.  Only the interfaces see both sides (the Context Property side AND the binding/gateway-specific side).

I'm afraid I don't agree with this statement.  Implementations have access to the Context, which contains properties for IN, OUT, and EXCHANGE.  An implementation that is servicing a request is quite capable of copying an IN-scoped property to an OUT-scoped property.

I think our disagreement stems from choice of words.  Yes, you are correct that a component implementation has full access to the Context.  When I say "only the interfaces see both sides", my definition of interface is the gateway/binding piece, which is the only place where knowledge of the protocol is.  For example, yes, the BPM component has access to the Context, but it does not have access to the HttpRequest, or the SOAPMessage that initiated everything.  So to clarify my statement "decide how properties get mapped", I was talking about how Context Properties get mapped in/out of the binding/protocol-specific object (HttpRequest or SOAPMessage).

Am I right, then, in my understanding that the desire might - might - be that all OOTB context mappers' mapFrom should change to put EVERYTHING they get from a request into Scope.IN, and similary, that all OOTB context mappers' mapTo should change to put EVERYTHING they encounter in Scope.OUT into a response?  And ignore anything at Scope.EXCHANGE in either direction?

Scope.EXCHANGE is never ignored.  We have the ability for component-specific context mappers for a reason, so that individual gateway can deal with headers/properties in a manner consistent with their native protocol.  I would say that absent protocol-specific context information, all request properties/headers should be mapped to Scope.IN.  An example of an Exchange.SCOPE property might be something like a conversation or session id for a specific protocol.  Scope.EXCHANGE properties should also be mapped to native response headers consistent with the context mapper configuration and the native protocol being used.

The problem here, again, is the disconnect when a gateway/binding component (like soap or http) is doing the mapping on behalf of a protocol-ignorant component (like the BPM Component).  You say that an example of a Scope.EXCHANGE property is a conversation or session id "for a specific protocol". Well, in Web Services, there is the notion of a correlation id specific to the protocol.  But my point is, what about non-protocol aware correlation ids? Like the processInstanceId from the BPM component, which is placed at Scope.EXCHANGE? The BPM Component has no "native protocol". And how is the SOAPComponent's ContextMapper supposed to know anything about it?  This issue is the crux of my point.  We need to all agree upon default behavior for protocol-knowledgeable ContextMappers for when they are dealing with Context properties that come from NON-protocol-knowledgeable components they are "gateway-ing" for.  That default behavior should be consistent across our OOTB ContextMappers, and when someone wants to do something different, they write their own.

Last, there is a questions of what part of a response to things get put in to.  Some interfaces have different "buckets" where things can go.  In our example, how do we know that a Scope.OUT variable should go inside a SOAP response message's soap header element area, vs. being an HTTP response header?  That's just one example...

I thought we added the ability to select which properties are returned via a context mapper?  I've always been against the idea of mapping context properties "blindly" - in other words, grabbing everything and including it as a header.  Being able to configure a context mapper with property names (or even label names) would be a good way to go and still could allow for wildcards to grab everything.

ContextMappers don't "return" anything. They map properties in-and-out of the binding/protocol-specific objects they are aware of.  Yes, we do have the ability to select the properties that get mapped based on regex includes, includeNamespaces, excludes, excludesNamespaces.  So only those properties whose names (and namespaces, if applicable) match get mapped.  But everything is a candidate for mapping, is what I meant.

[kcbabo]

The implementation doesn't have to know where a given context property is mapped, that is up to the context mapper.  The context mapper has a default mapping, but this can be modified by user configuration (e.g. selecting which properties are mapped to SOAP headers).  I think we are talking about the same moving parts here.  The only difference I can see is that I'm saying that the default behavior of context mappers should be to map request message headers/properties to Scope.IN instead of Scope.EXCHANGE.  I gave quite a few examples of why this is not appropriate.  If there are instances where a native request message header should be included in a native response header, then there are two options:

1) The context mapper's default behavior can be overridden to map the header to a Scope.EXCHANGE context property.

2) The implementation logic for the service *that the user adds* , copies the context property from Scope.IN to Scope.OUT.

The approach taken depends a lot on the context of the interaction.  If the header is only used with a specific protocol, then the context mapper approach makes more sense so that you don't pollute your implementation with native binding details.  On the other hand, if the consumer->provider interaction implies that additional context will be available outside of the payload regardless of the native protocol in use, then it make sense to go with option 2.

Just to be clear, the example above is for inbound messages.

[dward]

I think we shoould talk about this in our team meeting tomorrow.  There are still unanswered questions.  One example is what a gateway-spectific contextmapper should do when there are 2 "buckets" to put Scope.OUT properties into.  How to choose?  For example, the SOAPContextMapper could put them in the HttpResponse as http headers, or it could but them in the response payload as SOAP headers.  (This might not be a perfect example, but it's a viable one.)  Perhaps we leverage our "property labels" capability here, so that protocol-ignorant components, when adding a a context property at Scope.OUT, can add labels like "protocol" or "payload" to give a "hint" on which bucket the output property should get mapped into by the protocol-aware context mapper.  Discussing this via voice might be more efficient than here, as we can brainstorm together.

[kcbabo]

Layered protocols (e.g. SOAP over HTTP) definitely provide a bit more to think about.  In this case, I would think that the SOAP context mapper would provide an ability to map specific context properties to headers.  So a bucket for SOAP and a bucket for HTTP.  Sensible defaults have to be the rule for all context mappers, so the default for the SOAP context mapper could be to map into SOAP:Headers but this can be changed via context mapper configuration.  Using labels is another interesting idea.

I agree that live discussion will help. Just documenting my thoughts above so that other people can chime in if they want.

[mageshbk]

Layered protocols (e.g. SOAP over HTTP) definitely provide a bit more to think about.  In this case, I would think that the SOAP context mapper would provide an ability to map specific context properties to headers.  So a bucket for SOAP and a bucket for HTTP.

Didn't somebody hint this already

https://community.jboss.org/message/638102#638102