1 Reply Latest reply on May 30, 2013 11:53 AM by pferraro

Multiple masters when using clustering Singleton Service

kristjan273 May 29, 2013 10:47 AM

Hi,

I have noticed some problem on 7.2.0-final when using Singleton Service and whole domain group is started simultaneously. I can see that for some short time there are multiple masters on the domain. And this is quite problematic for me since the master status achieved fires some further events.

Here's a snippet from two server instances, started simultaneously:

server13 log snippet

20130528,16:43:05,485 INFO  [org.jboss.as.clustering.singleton] (ServerService Thread Pool -- 53) JBAS010342: server13:server-10.10.10.13/singleton elected as the singleton provider of the jboss.mine.ha.singleton.manager service
20130528,16:43:05,490 INFO  [org.jboss.as.clustering.singleton] (ServerService Thread Pool -- 53) JBAS010340: This node will now operate as the singleton provider of the jboss.mine.ha.singleton.manager service
20130528,16:43:05,748 INFO  [org.jboss.as.clustering.singleton] (notification-thread-0) JBAS010342: server14:server-10.10.10.14/singleton elected as the singleton provider of the jboss.mine.ha.singleton.manager service
20130528,16:43:05,749 INFO  [org.jboss.as.clustering.singleton] (notification-thread-0) JBAS010341: This node will no longer operate as the singleton provider of the jboss.mine.ha.singleton.manager service
...
20130528,16:43:07,798 INFO  [org.jboss.as.clustering.singleton] (notification-thread-0) JBAS010342: server14:server-10.10.10.14/singleton elected as the singleton provider of the jboss.mine.ha.singleton.manager service


server14 log snippet

20130528,16:43:05,757 INFO  [org.jboss.as.clustering.singleton] (ServerService Thread Pool -- 56) JBAS010342: server14:server-10.10.10.14/singleton elected as the singleton provider of the jboss.mine.ha.singleton.manager service
20130528,16:43:05,757 INFO  [org.jboss.as.clustering.singleton] (ServerService Thread Pool -- 56) JBAS010340: This node will now operate as the singleton provider of the jboss.mine.ha.singleton.manager service
20130528,16:43:07,801 INFO  [org.jboss.as.clustering.singleton] (notification-thread-0) JBAS010342: server14:server-10.10.10.14/singleton elected as the singleton provider of the jboss.mine.ha.singleton.manager service

Election policy is default, meaning that 0 and oldest node should be the only master.

I was looking into https://issues.jboss.org/browse/WFLY-68 but seems unrelated to me.

I would really appreciate if someone clarifies if this is a bug or some other misconfigured behavior?

Thanks in advance

1. Re: Multiple masters when using clustering Singleton Service

pferraro May 30, 2013 11:53 AM (in response to kristjan273)

This is a consequence of nodes starting concurrently. Each node will form it own cluster, then, after some short period of time, the clusters will get merged.
The workaround for this (which was introduced in the fix for WFLY-68) is to configure your SingletonService to require a quorum.

e.g.

new SingletonService(service, serviceName, 2);

In the above example, the service will not start unless the cluster contains at least 2 members.
Actions