Using WildFly instance as an external transaction coordinator service

[ike3]

I struggle with configuring an external transaction coordinator. According to https://developer.jboss.org/message/809802, all I need is the xts-environment property in my standalone.xml.

My testing environment is:

• callee - WildFly instance running callee.ear with a very simple web-service which writes a row in oracle table
• caller - WildFly instance running caller.ear with another web-service which calls the first service then writes another row in the same table

All instances are WidlfFly 15 on Java11 with standalone.xml copied from standalone-xts.xml example. @Transactional annonations are in place, transactions work fine.

The idea is to load-balance the callee service using ngnix or other balancer across multiple instances of WildFly. (I've already done it - it works fine too). Generally, there will be many other callers and callees so it may be a good idea to have an external transaction coordinator to have all transaction information in one place.

Can I just add another WildFly instance with XTS enabled and change in all my WildFly the xts-environment url pointing to this new WildFly instance?

    <subsystem xmlns="urn:jboss:domain:xts:3.0">

        <host name="default-host"/>

        <xts-environment url="http://<some_host>:<some_port>/ws-c11/ActivationService"/>

        <default-context-propagation enabled="true"/>

    </subsystem>

It seems the setting is ignored completely, no matter what URL I use, the caller instance is always using the local ActivationService. There are no requests to the coordinator host at all.

Am I doing this completely wrong? Can anybody point me in the right direction?

The source code for this example is available at https://github.com/ike3/lkp_transactions. I am using docker-compose so the environment is in the docker directory (unfortunately, the oracle database is not yet there).

[ochaloup]

Hi ike3 ,

at the first sight it seems there could be an issue in the WFLY/Narayana codebase. I will need to check that out. I will need some time to look at it. I will report back to you.

[tomjenkinson]

Ondra will be better placed to answer the XTS specifics, however what I can say is with your current approach (multiple transaction managers) you need to make sure all the servers have a unique node identifier.

[ike3]

Thanks. I changed the identifiers to be unique but nothing changes .

[tomjenkinson]

It makes the configuration more correct at least but that point was for the case where you have multiple coordinators not the new configuration you are looking to move to.

[tomjenkinson]

I think ike3 should debug into narayana/UserTransactionImple.java at master · jbosstm/narayana · GitHub  in the caller code as that is the code that connects to the remote coordinator.

[ike3]

Sorry for late response.

The breakpoint in UserTransactionImple did not trigger at all (not at caller side nor at callee). I tried other UserTransaction implementations but no luck either. This is definetely not a debugger issue as it triggers breakpoints in any other classes, i.e. JBossWSInvoker.

This is very strange as the transaction works fine (I changed in callee WS @TransactionAttribute to be MANDATORY and the transaction to be sure is active).

I've put my stack frames here:

• caller - https://pastebin.com/MmDgnEzc
• callee - https://pastebin.com/56ZFvHuR

I see CMTTxInterceptor at the caller side which basically creates the transaction but I don't know which interceptor is supporting the transaction at the callee side.

Do I need some more configuration other than replacing standalone.xml with standalone-xts.xml?

[tomjenkinson]

ochaloup I am not seeing any XTS code at all under GitHub - ike3/lkp_transactions: Исследование работы распределенных транзакций в WildFly веб-сервисах  - are you?

ike3 I think you need to update your code to be more like: quickstart/XTS/wsat-jta-multi_service at master · jbosstm/quickstart · GitHub  if you want to use XTS and JTA bridging.

[ochaloup]

tomjenkinson I assume if the standalone-xts.xml is used then the WS-AT bridging should be working without any special XTS code being involved in the user project. I quickly checked the GitHub - ike3/lkp_transactions I think there is not some big difference to what we provide in the quickstart/XTS/wsat-jta-multi_service. But I didn't check it in detail.

ike3 sorry for not taking look into this but I'm a bit busy with other duties. I can try to look at your reproducer in deail during this week but I can't promise.

Regarding the debug experience. As Tom mentioned our quickstart is a good starting point to check if the debugger hits the expected place.

I'm don't remember about the exact usage of the UserTransaction code which Tom mentioned. What I would says is that you use the container managed transactions then the ejb interceptors start the JTA transactions and that why you can see the interceptor being hit.

The JTA transaction is bridged to WS-AT (and vice versa) in the Narayana module txbridge (narayana/txbridge at 5.9.3.Final · jbosstm/narayana · GitHub). Bridging should work automatically, as you mentioned, when the standalone-xts.xml is used (aka. the server configuration defines the xts subsystem). You could debug the bridge handlers

• narayana/JaxWSTxOutboundBridgeHandler.java at master · jbosstm/narayana · GitHub
• narayana/JaxWSTxOutboundBridgeHandler.java at master · jbosstm/narayana · GitHub

They have to be registered for the transaction being transferred between WS-AT and JTA worlds.
Detailed trace logs would be helpful for the investigation as well. You can setup the TRACE logging category for the `org.jboss.jbossts.txbridge` and for lot more information about txn processing then for the 'com.arjuna` too.

[ike3]

Thanks!

I think I found the issue causing coordinator URL to be ignored.

During XTS subsystem startup in com.arjuna.webservices11.wscoor.server.ActivationCoordinatorInitialisation.startup() I can see wscEnvironmentBean initialized with the following properties:

asyncRequestWait 10000

bindAddress10 localhost

bindAddress11 172.18.0.9

bindPort10 8080

bindPort11 8080

bindPortSecure10 8080

bindPortSecure11 8443

coordinatorAddress10 null

coordinatorAddress11 null

coordinatorPath10 ws-c10/soap/ActivationCoordinator

coordinatorPath11 ws-c11/ActivationService

coordinatorPort10 8080

coordinatorPort11 8080

coordinatorScheme10 http

coordinatorScheme11 http

coordinatorURL10 null

coordinatorURL11 http://coordinator:8080/ws-c11/ActivationService

initialTransportPeriod 5000

maximumTransportPeriod 300000

serviceURLPath null

transportTimeout 30000

useAsynchronousRequest NO

The xts-environment property sets the coordinatorURL11 field but it is not used in the URL generation code:

   public static void startup()

  {

        final ServiceRegistry serviceRegistry = PrivilegedServiceRegistryFactory.getInstance().getServiceRegistry();

        WSCEnvironmentBean wscEnvironmentBean = XTSPropertyManager.getWSCEnvironmentBean();

        String bindAddress = wscEnvironmentBean.getBindAddress11();

        int bindPort = wscEnvironmentBean.getBindPort11();

        int secureBindPort = wscEnvironmentBean.getBindPortSecure11();

        String serviceURLPath = wscEnvironmentBean.getServiceURLPath();

        if (serviceURLPath == null) {

            serviceURLPath = "/ws-c11";

        }

        if (bindAddress == null) {

            bindAddress = "localhost";

        }

        if (bindPort == 0) {

            bindPort = 8080;

        }

        if (secureBindPort == 0) {

            secureBindPort = 8443;

        }

        final String baseUri = "http://" +  bindAddress + ":" + bindPort + serviceURLPath;

        final String uri = baseUri + "/ActivationService";

        final String secureBaseUri = "https://" + bindAddress + ":" + secureBindPort + serviceURLPath;

        final String secureUri = secureBaseUri + "/ActivationService";

        serviceRegistry.registerServiceProvider(CoordinationConstants.ACTIVATION_SERVICE_NAME, uri) ;

        serviceRegistry.registerSecureServiceProvider(CoordinationConstants.ACTIVATION_SERVICE_NAME, secureUri) ;

  }

Is it possible to configure the remaining properties in standalone.xml?

I've attached traces from the `org.jboss.jbossts.txbridge` category and did some debugging (JaxWSTxOutboundBridgeHandler is invoked successfully indeed). Unfortunately the traces do not contain tx-coordinator URL information.

[ochaloup]

Hi ike3 , I'm sorry for the delay of answering here.

You can check the configuration options of the XTS subsystem in the xsd here (wildfly/jboss-as-xts_3_0.xsd at master · wildfly/wildfly · GitHub ) or you can start the WFLY and check with the jboss-cli.sh with command like (/subsystem=xts:read-resource-description(recursive=true)).

How the configuration of the XTS subsystem is considered in WildFly could be overviewed here wildfly/XTSManagerService.java at master · wildfly/wildfly · GitHub

As you wrote there could be the issue that the coordinatorURL11 is not used in the code creating the URL. If you have suggestion about the fix, you will be more than wellcome to create the PR (ideally with the testcase) which adjusts the behaviour.

Thanks

Ondra