I suppose by cluster you mean domain; as the JavaDoc says "all servers in a domain must have unique names" you should really keep them unique.
But the situation you describe looks like a bug to me, it should either be checked (or also check different hostnames as you say).
(In cluster (regardless domain/standalone), the node names must be unique too; jboss.node.name property).
From a ejb-client perspective the node names need to be unique as this name is used to identify the node and store the different servers in the list which are available for an application.
If you have several servers with the same name only one of them will be used.