Hi Miroslav,

I think we had the similar issue before [WFLY-1119] Assign an unique NodeID automatically - JBoss Issue Tracker and we rejected it as the fixed length of node is 28 and we have no way to check if it is unique. Also the user should be forced to do this because it carries risk.

Thanks,

Amos

+1 - although it might be possible to utilize this if all node shared access to the same JDBC store? That might be a useful facility for OpenShift for example.

The node identifier defines as the following.

public static final SimpleAttributeDefinition NODE_IDENTIFIER = new SimpleAttributeDefinitionBuilder(CommonAttributes.NODE_IDENTIFIER, ModelType.STRING, true)
            .setDefaultValue(new ModelNode().set("1"))
            .setFlags(AttributeAccess.Flag.RESTART_ALL_SERVICES)
            .setAllowExpression(true)
            .setValidator(new StringBytesLengthValidator(0,23,true,true))
            .build();

So it is a string with the max length is 24.  We use it in the Xid (which imposes a 64 byte limit). The "UUID" convert to string looks like 32 bytes.

UUID.randomUUID().toString.replace("-", "");

It is possible to substring the first 24 bytes as the node name. It could be a useful feature for working in the openshift. So tomjenkinson, we might consider reopen the WFLY-1119

Sure - let's reopen but it shouldn't be the default which therefore won't help Mirek with his warning.

So long as it shows a WARN that is OK. The issue with storing in the object store though is that someone can delete it accidentally whereas normally the node id is in the config file.

I think this could be a nice feature. It would be good to ease the life of the user. I think there is a good chance that node id will not be changed by administrator even in bigger environment, it could happen when WFLY/Swarm is used in microservice world where multiple containers are started, it happens for sure in case user does not understand the meaning of the warning (that's whoever comes the first time to use Narayana doesn't) etc.

Regarding of the uniqueness, having 28 characters taken from a random generator is a quite good chance not hitting the same one on two nodes. What do you think?

mnovak would you have a link to Artemis code where the id is generated? I think it could be a good to get inspiration.

zhfeng I'm sorry for my ignorance but why the maximal length of 24 utf characters could be problematic?

tomjenkinson I think so. It sounds beneficial. Do not Openshift set this already somehow? Using the same node id for Narayana could already bring troubles. It could be fine for recovery process too. If node id is saved in object store then if WFLY node crashes it is ok to start another node just pointing to the existing object store and recovery would grab the node id to finish in-doubt txns (I see there's the limitation of having the same resources configured).

I was searching what's the way how the Openshift configures and I can see that it set the node id to {{jboss.node.name}}

https://ce-gitlab.usersys.redhat.com/ce/jboss-dockerfiles/blob/develop/scripts/os-eap7-openshift/added/standalone-openshift.xml#L417

and that's set by sh script

https://ce-gitlab.usersys.redhat.com/ce/jboss-dockerfiles/blob/develop/scripts/os-eap7-launch/added/launch/ha.sh#L40

Still, I wonder if it's possible to reconnect to an existing object store when a node crashes and having the new node having set the node id of the old node.

If the server has no configuration to distinguish it from the other servers and it connected to a shared JDBC store it would read the same node identifier as other servers which would not be good.

What you are describing could be done (entirely in WildFly code as it is responsible to set the node identifier: wildfly/CoreEnvironmentService.java at 6b61a6003f704221f66dcd9f418bcb7af88fb9ab · wildfly/wildfly · GitHub ) but I would not recommend it. Node identifiers are better set by an administrator to be certain that they are configured in a known manner. It is entirely possible that the administrator follows a process as you defined but putting that into the library seems dangerous as Narayana does not know its operating environment (e.g. does it share an object store).

Yeah, that would be good but I would still think it might fail the "openshift use case" - potentially those servers are provisioned using docker (?) which might reset to the SHA just after WFLY is unzipped and so lose the step where a node id is allocated?

What might work (based on your suggestions) is:

1. WFLY starts up with no node id

2. WFLY calculates a node id

3. WFLY checks JDBC store to see if this node id is known (under JDBC lock) and then writes node id to JDBC store to a table of known node ids and commits

4. WFLY writes node id to local configuration.xml

The issue is that there is no obvious way to garbage collect the table of known node ids for nodes that die (after step 3) and never come back.

Also, from a usability point of view if the user reverts to a config that doesn't have a node ID in the config those TX would be orphaned.