Problem:

Looks like one of the machines has a bad view of the cluster:

63.210.100.66:1099, 63.210.100.64:1099, 63.210.100.65:1099

The explanation is as follows:

1. Each JGroups node has an address consisting of IP address and port, e.g. hostA:12345.

2. A JGroups address is mapped to a JBoss address, which looks like hostA:1099

3. When a JGroups node bounces, it'll get a new address, e.g. hostA:23456, however the JBoss address is still hostA:1099.

4. So when hostA bounces, and the bounce is not detected, e.g. because your timeouts in FD are high, then we might have a JGroups membership of , which is translated to a JBoss membership of . This is the case until the bounced former member is removed (hostA:12345).

5. If you set the timeouts in FD to a lower value, or use FD_SOCK, this will go away