JBossDeveloper

I think that it uses the JDBC store - or certainly it should be using that and if not I suggest to raise a bug!

I haven't looked at the config you reference directly but the way WFLY works is that each WFLY has a single transaction manager and recovery manager pair. It is a requirement that a WFLY server for a particular node identifier is online in order for it to complete recovery.

For further investigation I already referenced the shell script which defines the node identifier - it's this one

https://ce-gitlab.usersys.redhat.com/ce/jboss-dockerfiles/blob/develop/scripts/os-eap7-launch/added/launch/ha.sh#L40

Wondering now about this same problem, as the Infinispan Server template for Openshift logs the WFLYTX0013 warning on each node - and obviously it is expected to have multiple nodes running. I think the controller (Openshift) should assign this explicitly as it is the only actor having all necessary knowledge of the desired cluster state. I heard it already makes a sequential number available to each pod but I'm not sure if that would be a fitting solution. What are the required properties that this "id" should have? For example I'm wondering if it would be legal for a new node which is joining to reuse an id which has been used - but is no longer in use. I expect not.. but then this will depend on how you expect to distinguish a "new node" from a "node which was temporarily offline". Would love to know more about how this id is being used. Thanks

Hi Sanne,

The design we have for EAP is that we can safely use the podname for the node identifier and so in fact the Infinispan Server template can be updated with this latest approach alongside the manual recovery procedure (for scale down). mmusgrov and ochaloup are working on ensuring the automated recovery (for scale down) in the EAP base image against [CLOUD-2261] [EAP][XA][Recovery][NFS] split lock is broken after a minute of network partition - JBoss Issue Tracker  (design doc is linked there).

Hope that helps,

Tom

Many thanks Tom! I'll suggest this link to the Infinispan team.

OpenShift aside, does this node id need to survive restarts in a stateful environment ?

If so, Infinispan can generate and persist a unique UUID on init and I was wondering whether that would be acceptable.

Hi Tristan,

It does need to survive restarts and must be connected to the same objectstore (state). Typically in for example WildFly this ID is persisted in the standalone.xml and for standalone Narayana it would be in the jbossts-properties.xml.

When considering a suitable allocation of the ID, the main thing to be aware of is it has to be globally unique with guarantees that no other instance is running at the same time with the same ID. In this case "global" refers to objectstore and any possible third party resource manager who may be contacted by the various instances of the transaction manager.

Thanks,

Tom

Hi tomjenkinson,

based on this discussion, how the standalone-openshift.xml is setup and my talk with rhusar - what about considering to define the value of node-identifier for ${jboss.node.name} as value in WFLY standalone*.xml configuration (by default).

<subsystem xmlns="urn:jboss:domain:transactions:3.0">
    <core-environment node-identifier="${jboss.node.name}">
    ...

Currently the default value for the node identifier is 1 (wildfly/TransactionSubsystemRootResourceDefinition.java at master · wildfly/wildfly · GitHub ) and in case of running the EAP container in the cluster (ha setup, ejb setup) it's expected to get the jboss.node.name being unique over the cluster (Chapter 8. Clustered Enterprise JavaBeans - Red Hat Customer Portal ). Up to that this value is used as identifier in Undertow subsystem. Thus this value is known to users and is used in WFLY in general.

Using it, instead of default value 1, we can prevent omission of changing the value which can cause issues during transaction recovery processing.

What do you think?

The advantage of "1" is we can alert the user they have not changed the Narayana node identifier to something globally unique. Could we do something similar if we used jboss.node.name instead of 1?

What is jboss.node.name set to per default? Is it just the hostname? Because then per default you could have 2xWFLY on one machine and not really know you have made a config error.

tomjenkinson and ochaloup, it seems that this has been discussed in the [WFLY-1119] Assign an unique NodeID automatically - JBoss Issue Tracker

tomjenkinson yes, you are right the jboss.node.name is set by default to the hostname (wildfly-core/ServerEnvironment.java at master · wildfly/wildfly-core · GitHub ). There is the same issue for ejb remoting, clustering and undertow when the jboss.node.name is not set then the server could behave pretty wrong (e.g. clustering won't work etc.)

We can still check for the default value and print warnings even in the case we will be using the jboss.node.name configured in the xml (we could change the default value at wildfly/TransactionSubsystemRootResourceDefinition.java at master · wildfly/wildfly · GitHub  to be like .setDefaultValue(new ModelNode().set(new ValueExpression("${jboss.node.name}"))) but I think it's not a good option based on the of the backward compatibility)

zhfeng I was checking the comments at WFLY-1119 but by my ignorance I haven't found some discussion about usage jboss.node.name. Please, what do you refer to? Thanks.

ochaloup WFLY-1119 suggests an algorithm to generate an UUID as the node-identifier when  the default value is "1". It looks like the "${jboss.node.name}" is used as the unique UUID in your proposal. So we are facing the similar issues

1. we have no way to know if the "jboss.node.name" or the other UUID is unique

2. we can not warn the user as this is the default value.

I think this is why we reject because it carries risk.

Also it can not set the default value of node-identifier as the expression ${jboss.node.name} directly. The possible way could be introduce an attribute with enableAutoGenerateNodeId.

if (defaultNodeIdentifier.equals(nodeIdentifier) && enableAutoGenerateNodeId == true) {
    // use the ${jboss.node.name} or generate the uuid as the node id
    // update the nodeIdentifier with the new value and log the warn message
    // that the node-identifier has been set to the value of ${jboss.node.name}
    // and this could be risk if it is not unique.
}