-
1. Re: Issue with HA Singleton Service
sreeja_e_j May 13, 2014 6:23 AM (in response to sreeja_e_j)This turned out to be a network issue between the master and the slave node.
When the split occurred both the Master and Slave started the singleton service (Split-Brain Issue) .
However, when the Merge occurred, the "Service Stop" request is not being received on none of the Nodes.
As a result, the singleton service continues to run on both the nodes after merge.
We do see an issue called out in https://issues.jboss.org/browse/AS7-5218
But need to know if this can be handled in JBOSS 7.1.1
-
2. Re: Issue with HA Singleton Service
wdfink May 13, 2014 8:13 AM (in response to sreeja_e_j)If you look to the JIRA the issue is resolved in 7.1.3 or EAP6.1.0.Alpha.
AS7.1.1 will behave the same as 7.1.0.
You need to upgrade to a newer version. EAP6.1.0.Alpha is available for free. AS7.1.3 or 7.2 can be build from the sources at github.
Also you can download WildFly 8.0.0.Final
-
3. Re: Issue with HA Singleton Service
sreeja_e_j May 14, 2014 9:58 AM (in response to wdfink)Thanks for the reply.
Unfortunately, we wont be able to upgrade to 7.1.3. We are looking at alternate options.
We are trying to implement GroupMembershipListener and wait for membershipChangedDuringMerge() to be invoked..
We are observing that this method is being invoked during a Merge ( after a SplitBrain situation )
Can we force a re-election at this point ? Is there a way to achieve this ?
-
4. Re: Issue with HA Singleton Service
sreeja_e_j May 14, 2014 1:05 PM (in response to sreeja_e_j)We have implemented the GroupMembershipListener and in the
public void membershipChangedDuringMerge(List<ClusterNode> arg0,
List<ClusterNode> arg1, List<ClusterNode> arg2,
List<List<ClusterNode>> arg3) {
{
<if the first node in the viewList is not the same as the nodename >
{
ServiceController<?> controller = CurrentServiceContainer.getServiceContainer().getRequiredService(<service name>);
SingletonService<String> ss = (SingletonService<String>) controller.getService();
Set set = new HashSet(arg2);
ss.serviceProvidersChanged(set, true);
}
}
This seems to be helping our error scenario - 2 Nodes Running as Singleton Provider on Merge
Do let us know if you have any comments on this
-
5. Re: Issue with HA Singleton Service
wdfink May 15, 2014 3:49 PM (in response to sreeja_e_j)If you test your workaround and it works for you you might stay with it.
But I would prefer to upgrade or integrate the fix in the version you use as I know that such workarounds live a long time as noone remember why it was installed