9 Replies Latest reply on May 6, 2014 4:00 PM by paul.robinson

Distributed transactions using spring/tomcat

slybuford Apr 16, 2014 2:47 PM

I have a requirement to support distributed transactions in a spring/tomcat application. As much as I would love to utilize JBOSS AS to accomplish this, for a variety of reasons it is not possible. As such, I've been working through numerous Quickstarts (very helpful btw) and this particular post: https://community.jboss.org/thread/154364. The test application that I have created has the following components:

ws-at client to initiate end-to-end transaction

webservice A

webservice B

I am deploying webservice A within my IDE and webservice B to my local tomcat instance on different ports (to simulate txn propagation/coordination over the wire). the process is invoked by a JUnit test which pulls a client from the spring context and makes a call to webservice A. webservice A first makes a call to webservice B, which always successfully saves data to a mysql db. webservice A then inserts/updates data locally to the same mysql instance. i have the ability to simulate success or failure for this last step. success means 2 tables have records inserted/updated. if the second set of db operations (at webservice A) fails, no records should be present in the db, i.e. the db work performed by webservice B is rolled back.

I have managed to make all this work utilizing the standard CompletionCoordinator service, however the implementation is wonky. Ideally, I would like to utilize the "non-standard", JBOSS specific RPC coordinator to avoid the necessity of exposing an endpoint at the client (CompletionCoordinatorRPCPortTypeImpl). In this configuration however, I am encountering the following error. It appears the webservice response is not including the appropriate addressing headers, but I am thoroughly confused as to why or even why it would matter. I've tried to include as much pertinent information without going overboard (may have failed there). I am also working to clean my environment and get the entire test application up on a public repo, since it seems there are a few brave souls who have attempted this in the past and maybe a few more who could benefit in the future from my days of cursing... Any help would be greatly appreciated.

Apr 16, 2014 10:58:31 AM org.apache.cxf.ws.addressing.soap.MAPCodec restoreExchange

WARNING: Response message does not contain WS-Addressing properties. Not correlating response.

Apr 16, 2014 10:58:31 AM org.apache.cxf.ws.addressing.ContextUtils retrieveMAPs

WARNING: WS-Addressing - failed to retrieve Message Addressing Properties from context

Apr 16, 2014 10:58:31 AM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging

WARNING: Interceptor for {http://docs.oasis-open.org/ws-tx/wsat/2006/06}CompletionCoordinatorRPCService#{http://docs.oasis-open.org/ws-tx/wsat/2006/06}CommitOperation has thrown exception, unwinding now

org.apache.cxf.binding.soap.SoapFault: A required header representing a Message Addressing Property is not present

at org.apache.cxf.ws.addressing.impl.MAPAggregatorImpl.mediate(MAPAggregatorImpl.java:561)

at org.apache.cxf.ws.addressing.impl.MAPAggregatorImpl.handleMessage(MAPAggregatorImpl.java:143)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)

at org.apache.cxf.endpoint.ClientImpl.onMessage(ClientImpl.java:835)

at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1614)

at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1504)

at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1310)

at org.apache.cxf.io.CacheAndWriteOutputStream.postClose(CacheAndWriteOutputStream.java:50)

at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:223)

at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)

at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:628)

at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:272)

at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:565)

at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:474)

at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:377)

at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:330)

at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)

at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:135)

at com.sun.proxy.$Proxy51.commitOperation(Unknown Source)

at com.arjuna.webservices11.wsat.client.CompletionCoordinatorRPCClient.sendCommit(CompletionCoordinatorRPCClient.java:73)

at com.arjuna.wst11.stub.CompletionRPCStub.commit(CompletionRPCStub.java:44)

at com.arjuna.mwlabs.wst11.at.remote.UserTransactionStandaloneImple.commitWithoutAck(UserTransactionStandaloneImple.java:322)

at com.arjuna.mwlabs.wst11.at.remote.UserTransactionStandaloneImple.commit(UserTransactionStandaloneImple.java:120)

pom particulars:

<groupId>org.jboss.narayana</groupId>

<artifactId>jbosstxbridge</artifactId>

<version>5.0.1.Final</version>

</dependency>

<groupId>org.jboss.ws.cxf</groupId>

<artifactId>jbossws-cxf-client</artifactId>

<version>4.3.0.Final</version>

</exclusion>

</exclusions>

</dependency>

relevant spring config:

<jaxws:endpoint id="txnSoapService" implementor="com.hlc.soap.TestTxnSoapServiceImpl" address="/txnSoapService">

<jaxws:handlers>

</jaxws:handlers>

</jaxws:endpoint>

<jaxws:endpoint id="wsatParticipantService" implementor="com.arjuna.webservices11.wsat.sei.ParticipantPortTypeImpl" address="/ParticipantService"/>

<jaxws:endpoint id="wsatCompCoordService" implementor="com.arjuna.webservices11.wsat.sei.CompletionCoordinatorPortTypeImpl" address="/CompletionCoordinatorService"/>

<jaxws:endpoint id="wsatCompletionCoordRPCService" implementor="com.arjuna.webservices11.wsat.sei.CompletionCoordinatorRPCPortTypeImpl" address="/CompletionCoordinatorRPCService"/>

<jaxws:endpoint id="wsatCoordService" implementor="com.arjuna.webservices11.wsat.sei.CoordinatorPortTypeImpl" address="/CoordinatorService"/>

<jaxws:endpoint id="wsatActivationService" implementor="com.arjuna.webservices11.wscoor.sei.ActivationPortTypeImpl" address="/ActivationService"/>

<jaxws:endpoint id="wsatRegistrationService" implementor="com.arjuna.webservices11.wscoor.sei.RegistrationPortTypeImpl" address="/RegistrationService"/>

<jaxws:endpoint id="wsatCompletionInitService" implementor="com.arjuna.webservices11.wsat.sei.CompletionInitiatorPortTypeImpl" address="/CompletionInitiatorService"/>

<jaxws:client id="testClientLocal"

address="http://localhost:8181/ws/txnSoapService"

serviceClass="com.hlc.soap.TestTxnSoapService" >

<jaxws:handlers>

</jaxws:handlers>

</jaxws:client>

<jaxws:client id="testClientRemote"

address="http://localhost:8080/ws/txnSoapService"

serviceClass="com.hlc.soap.TestTxnSoapService" >

<jaxws:handlers>

</jaxws:handlers>

</jaxws:client>

properties used to drive XTS configuration for webservice participants (adapted from original xts-properties.xml)

<entry key="org.jboss.jbossts.xts11.bind.address">localhost</entry>

<entry key="org.jboss.jbossts.xts11.wsat.UserTransaction">com.arjuna.mwlabs.wst11.at.remote.UserTransactionStandaloneImple</entry>

<entry key="org.jboss.jbossts.xts11.wsba.UserBusinessActivity">com.arjuna.mwlabs.wst11.ba.remote.UserBusinessActivityImple</entry>

<entry key="org.jboss.jbossts.xts11.wsat.TransactionManager">com.arjuna.mwlabs.wst11.at.remote.TransactionManagerImple</entry>

<entry key="org.jboss.jbossts.xts11.wsba.BusinessActivityManager">com.arjuna.mwlabs.wst11.ba.remote.BusinessActivityManagerImple</entry>

<entry key="org.jboss.jbossts.xts.initialisation.xtsInitialisation_1">org.jboss.jbossts.xts.initialisation.CoordinatorSideInitialisation</entry>

<entry key="org.jboss.jbossts.xts.initialisation.xtsInitialisation_2">org.jboss.jbossts.xts.initialisation.ParticipantSideInitialisation</entry>

<entry key="org.jboss.jbossts.xts.initialisation.xtsInitialisation_3">org.jboss.jbossts.xts.initialisation.ClientSideInitialisation</entry>

<entry key="org.jboss.jbossts.xts11.coordinatorURL">http://localhost:8181/xts/ActivationService</entry>

properties used to drive XTS configuration for standalone client (adapted from original xts-properties.xml)

<entry key="org.jboss.jbossts.xts11.bind.address">localhost</entry>

<entry key="org.jboss.jbossts.xts11.wsat.UserTransaction">com.arjuna.mwlabs.wst11.at.remote.UserTransactionStandaloneImple</entry>

<entry key="org.jboss.jbossts.xts11.wsat.TransactionManager">com.arjuna.mwlabs.wst11.at.remote.TransactionManagerImple</entry>

<entry key="org.jboss.jbossts.xts.initialisation.xtsInitialisation_3">org.jboss.jbossts.xts.initialisation.ClientSideStandaloneInitialisation</entry>

<entry key="org.jboss.jbossts.xts11.coordinatorURL">http://localhost:8181/xts/ActivationService</entry>

line 534 of MAPAggregatorImpl is where this is failing:

if (null == theMaps

&& !ContextUtils.isOutbound(message)

&& ContextUtils.isRequestor(message)

&& getWSAddressingFeature(message) != null

&& getWSAddressingFeature(message).isAddressingRequired()) {

boolean missingWsaHeader = false;

AssertionInfoMap aim = message.get(AssertionInfoMap.class);

if (aim == null || aim.size() == 0) {

missingWsaHeader = true;

}

this is the SOAP request:

INFO: Outbound Message

---------------------------

ID: 4

Address: http://localhost:8181/xts/CompletionCoordinatorRPCService

Encoding: UTF-8

Http-Method: POST

Content-Type: text/xml

Headers: {Accept=[*/*], SOAPAction=["http://docs.oasis-open.org/ws-tx/wsat/2006/06/Commit"]}

Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Header><Action xmlns="http://www.w3.org/2005/08/addressing">http://docs.oasis-open.org/ws-tx/wsat/2006/06/Commit</Action><MessageID xmlns="http://www.w3.org/2005/08/addressing">urn:83bea858b44500f2:671aae97:14566eff089:-7ffe</MessageID><To xmlns="http://www.w3.org/2005/08/addressing">http://localhost:8181/xts/CompletionCoordinatorRPCService</To><ReplyTo xmlns="http://www.w3.org/2005/08/addressing"><Address>http://www.w3.org/2005/08/addressing/none</Address></ReplyTo><wsarj:InstanceIdentifier xmlns="http://docs.oasis-open.org/ws-tx/wscoor/2006/06" xmlns:ns2="http://www.w3.org/2005/08/addressing" xmlns:wsa="http://www.w3.org/2005/08/addressing" xmlns:wsarj="http://schemas.arjuna.com/ws/2005/10/wsarj" ns2:IsReferenceParameter="1">0:ffffc0a80102:fa4d:534d8d13:3a</wsarj:InstanceIdentifier></soap:Header><soap:Body><Commit xmlns="http://docs.oasis-open.org/ws-tx/wsat/2006/06"/></soap:Body></soap:Envelope>

this is the SOAP response:

Apr 15, 2014 12:49:37 PM org.apache.cxf.services.CompletionCoordinatorRPCService.CompletionCoordinatorRPCPortType.CompletionCoordinatorRPCPortType

INFO: Inbound Message

----------------------------

ID: 4

Response-Code: 200

Encoding: UTF-8

Content-Type: text/xml;charset=UTF-8

Headers: {content-type=[text/xml;charset=UTF-8], Date=[Tue, 15 Apr 2014 19:49:36 GMT], Server=[Apache-Coyote/1.1], transfer-encoding=[chunked]}

Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><SOAP-ENV:Header xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"/><soap:Body><Result xmlns="http://docs.oasis-open.org/ws-tx/wsat/2006/06">true</Result></soap:Body></soap:Envelope>

1. Re: Distributed transactions using spring/tomcat

paul.robinson Apr 17, 2014 10:27 AM (in response to slybuford)

Reggie,

The RPC client approach hasn't received a lot of attention recently (partly due to it being non-standard), and I'm not sure how well it is tested. I don't believe we have an integration test for this and we certainly don't have a quickstart. However, I do see it as a useful option, that we need to make sure works.

My understanding is that you have your spring+tomcat solution working for the standardised async WS-AT protocol. That's great as it suggests you have it generally working. When you switch to the synchronous client approach, you hit issues. As I'm not confident that this still works in the more familiar JBoss->JBoss setup, I think we need to test that first, so as to understand if the problem lies in your setup our the Narayana code.

I think a quickstart would be appropriate here. Maybe a simplified version of quickstart/XTS/wsat-jta-multi_service at master · jbosstm/quickstart · GitHub that invokes a single service (rather than two in that example) and the client uses the synchronous protocol. If this is something you'd like to contribute, then we'd love to have it. Otherwise, we could create this, but it may take a little longer due to time pressures at our end.

Paul.
Actions
2. Re: Distributed transactions using spring/tomcat

slybuford Apr 17, 2014 6:19 PM (in response to paul.robinson)

Hi Paul. Thanks a lot for getting back. I have pushed the test application up to github and verified with a few co-workers that the expected error is seen on their end: sleepymonkey/spring-distributed-txn · GitHub

i've tried to list out the necessary steps and make this as simple and repeatable for you as possible. Please let me know if you run into any issues and i'll address them ASAP. The last thing i want is to burden you when you're attempting to help. the error I reference above is seen when running the RpcSoapTxnClient junit test.

thanks again,
reggie
Actions
3. Re: Distributed transactions using spring/tomcat

gytis Apr 22, 2014 10:13 AM (in response to slybuford)

Hi Reggie, thanks for the code example. I have managed to reproduce your failure and will take a look at it, while Paul is away. Hopefully, we will have a solution for you soon.

Thanks,
Gytis
Actions
4. Re: Distributed transactions using spring/tomcat

slybuford Apr 22, 2014 8:44 PM (in response to gytis)

Thanks Gytis. I really appreciate the responsiveness from both you and Paul. If it's not too much trouble, could you verify that what we've cobbled together is a reasonable approach to incorporating these libraries in an application running outside JBOSS? In particular, we tried several different configurations around the JTA->WSAT transaction bridging and never managed to get it working in a simpler fashion than in our example project. To me it feels like we are doing it wrong (client ->WS-AT service -> internal JTA service -> remote WS-AT service). Things would be much cleaner if we could avoid the initial client -> WS-AT service steps and instead directly invoke the business layer/internal JTA service (HlcServiceImpl in our example project). The only reason we make those first 2 calls (client -> WS-AT web service) is to setup the Activation/Coordination services and (I think) the explicit wst UserTransaction. All downstream transaction bridging fails if we do not make calls to those services first. In other words, WS-AT -> JTA -> WS-AT bridging works great, however JTA -> WS-AT does not work for us. In our eventual production application, 8 times out of 10 we'll only have local JTA transactions. We won't know this at compile time however so currently we have structured things with the initial WS-AT web service in the event that we need to make a remote call from within a local JTA txn (thus making it a distributed txn).

Hopefully I'm not confusing matters. My original question would more than likely become pointless if we could figure out how to make the simpler JTA -> WS-AT txn bridging work. All of this is relatively new to me and sadly, I haven't mastered all the moving parts--just curious if it's possible and if you could provide any insight into where we might look if so.

thanks again for your time...
reggie
Actions
5. Re: Distributed transactions using spring/tomcat

paul.robinson Apr 26, 2014 1:56 PM (in response to slybuford)

Reggie,

I think we should try to clarify your ideal architecture first, as a solution for that is going to be better for you than one for your workaround scenario. I think this is what you want:

Essentially:

1) The our client is deployed to a spring container
2) The client begins a JTA transaction and invokes two Web Services
3) The TXBridge handlers (TXBridge) deal with the mapping of the JTA transaction to a WS-AT transaction and ensure the WS-AT context is propagated to the web services
4) Each web service is deployed to a separate Tomcat instance
5) Each web service updates a table in a MySQL DB instance within a JTA transaction
6) The mapping of the propagated WSAT transaction to the JTA transaction in the web service is handled by the TXBridge handlers (TXB).

Gytis is away until the 6th May anyway, so I suggest we use this time to figure out what you would ideally like and what would be an acceptable workaround in the event that that is not possible.

Paul.
Actions
6. Re: Distributed transactions using spring/tomcat

slybuford Apr 26, 2014 10:55 PM (in response to paul.robinson)
That's great Paul. Almost exactly what we are trying to achieve. My 1000 words succinctly depicted... The 2 things I would change:

replace "invoke WS2" in the spring box with "update db"
treat the spring box as another tomcat instance (it's really a service in our world).

The client in our architecture can be any number of different systems (web pages, cmd line scripts, other services, etc). While we may need to support more than one external webservice call from within the initial JTA transaction, I want to make sure that our typical case is accurately reflected in your picture. From my earlier post, approximately 80% of the time our spring box picture will look like: REST/SOAP webservice -> JTA txn begin -> do some db work -> JTA txn commit. It's the situation where the JTA db work was coupled with an external call to WS1 or WS2 that things broke down for us. I can't remember the exact error, but is was something along the lines of "transaction not known". I believe i have the client/service calls commented out in the example project. It would be very easy to set that up if it would help you understand precisely what we're trying to achieve and where things are failing.

Again, my apologies on confusing the issue here. While I think there might be a small problem with the RPC client, I would much prefer to achieve the flow you have listed above (with my 2 tweaks...).

thanks again for your help.
reggie
Actions
7. Re: Distributed transactions using spring/tomcat

paul.robinson May 2, 2014 6:58 AM (in response to slybuford)

Reggie,

Ok, so is this closer:

The normal case:
The case when you also need to call another WS:
Where, the service in tomcat one is using Spring. Is the service in Tomcat 2 also using Spring?

Paul.
Actions
8. Re: Distributed transactions using spring/tomcat

slybuford May 2, 2014 3:14 PM (in response to paul.robinson)

Paul, you're rockin the images. Nice work. You have captured our architecture perfectly. To answer your question, yes, both tomcat instances are utilizing spring. In fact, they are the same application, configured using separate property files.

Just 2 days ago I managed to get our ideal situation to work. I replaced the inbound handler at the webservice (e.g. WS1) with the OptionalJaxWSTxInboundBridgeHandler and added a @Transactional annotation to the webmethod/webservice endpoint. In the 'normal case' above, spring initiates and manages the JTA transaction based on the annotation at the webservice entry point. If a distributed call is necessary (2nd case above), the JBOSS outbound handler is used in our webservice client, setting up the activation/coordination services and propagating the txn context to the remote service. As the remote service is configured with the optional inbound handler, it detects the ws-t and addressing headers and performs the ws-t -> JTA translation. This is my understanding at least. Feel free to clarify if that is incorrect or off in any way. Irrespective, it works, and is much cleaner and simpler than what i originally posted up to github.

I need a few days to clean up my 'real work', but I'll return to the example project once I hear back from you and ensure things there are simplified should anyone else need a working example. While it took me a few weeks to get all of this sorted, that pales in comparison to the amount of time it would have taken us to implement from the ground up. So... thanks for making your work available to the public. It was a great help!

reggie
Actions
9. Re: Distributed transactions using spring/tomcat

paul.robinson May 6, 2014 4:00 PM (in response to slybuford)

Reggie,

Paul, you're rockin the images. Nice work. You have captured our architecture perfectly.

Thanks! I like diagrams ;-)

Just 2 days ago I managed to get our ideal situation to work. I replaced the inbound handler at the webservice (e.g. WS1) with the OptionalJaxWSTxInboundBridgeHandler and added a @Transactional annotation to the webmethod/webservice endpoint. In the 'normal case' above, spring initiates and manages the JTA transaction based on the annotation at the webservice entry point. If a distributed call is necessary (2nd case above), the JBOSS outbound handler is used in our webservice client, setting up the activation/coordination services and propagating the txn context to the remote service. As the remote service is configured with the optional inbound handler, it detects the ws-t and addressing headers and performs the ws-t -> JTA translation. This is my understanding at least. Feel free to clarify if that is incorrect or off in any way. Irrespective, it works, and is much cleaner and simpler than what i originally posted up to github.

This sounds sensible. However, it's not something we have done before with Spring and Tomcat, so I can't say for sure. We'd be happy to review your example, at which point we should be able to say more confidently wether we think you have it right.

It would be great to have this as a contribution to our quickstart repo. It's certainly something that I think others would benefit from. I think it would be great for you too. Providing the pattern is solid, we would maintain the example going forwards. We test our quickstarts with every new commit to Narayana and we ensure that they work with every release. So you would get some level of assurance going forward that your scenario will still work with future WildFly and Narayana releases.

While it took me a few weeks to get all of this sorted, that pales in comparison to the amount of time it would have taken us to implement from the ground up. So... thanks for making your work available to the public. It was a great help!

You're welcome! I'm glad you found it useful.

Paul.
Actions

Go to original post