JBAS-2499 Configurable Policy to Elect HASingleton Master

[brian.stansberry]

Discussion forum for http://jira.jboss.com/jira/browse/JBAS-2499 .

See also http://www.jboss.com/index.html?module=bb&op=viewtopic&t=78336.

Some very brief and rough thoughts on configurable policies for determining an HASingleton master.

The policy would be dependency injected into the target HA singleton. Would need to expose an interface; let's say "org.jboss.ha.singleton.HASingletonElectionPolicy". Not picky about names, that's one I'm tossing out.

Things I imagine in the interface, again names are just rough ideas:

/**
 * Called by the HASingleton to provide the election policy a reference to
 * itself. A policy that was designed to elect a particular kind of singleton
 * could downcast this object to a particular type and then access the
 * singleton for state information needed for the election decision.
 */
public void setManagedSingleton(Object singleton);

public Object getManagedSingleton();

/**
 * Called by the HASingleton to set the name with which the singleton
 * service is registered with the HAPartition.
 */
public void setServiceName(String serviceName);

public String getServiceName();

/**
 * Sets the ClusterPartition; from this the election policy can gain
 * access to the DistributedReplicantManager for tracking the
 * deployment topology for the singleton service and to the HAPartition
 * for making group RPC calls.
 */
public void setClusterPartition(ClusterPartition partition);

/**
 * Conducts an election and returns whether the managed singleton
 * is the master.
 */
public boolean isElectedMaster();

Any other configuration options would be exposed by the policy implementation and would be set as part of configuring the policy before injecting it into the HASingleton.

HASingletonSupport would include the necessary code to call setManagedSingleton() and setServiceName() after the policy is injected.

We'd want a default implementation that works like the current election process -- i.e. call DistributedReplicantManager.isMasterReplica(). In HASingletonSupport.createService() we should check if an election policy has been configured, and if not, create an instance of this default policy.

[belaban]

where's the list of nodes of the cluster from which you pick a singleton, e.g. ?

Address pickSingleton(Collection nodes) ?

[brian.stansberry]

I was thingking the policy has access to the HAPartition and DRM and knows the service name, so it can find out who the nodes are. So, reduce the coupling between the singleton and the policy by not making the singleton responsible for providing the topology.

But, the singleton is going to ask the policy who's been elected because it got a notification of a topology change, so it makes sense to pass the topology.

If the singleton provides the topology, then I'd say:

Address pickSingleton(List nodes)

List because order is important to most policies.

I like getting Address back instead of boolean -- that way the singleton knows who the master is. I'd keep a boolean version as well though, as a convenience, i.e.



public boolean isElectedSingleton(List node) {
return pickSingleton(nodes).equals(partition.getLocalAddress());
}

[alexfu.novell]

OK.
I will have an abstract class
org.jboss.ha.singleton.HASingletonElectionPolicyBase
and one implementation class
org.jboss.ha.singleton.HASingletonElectionPolicySimple which elects the master node based on attribute "position" defined in the MBean descriptor. Here is how position is defined:

 * The value will be divided by partition size and only remainder will be used.
 *
 * Let's say partition size is n:
 * 0 means the first oldest node.
 * 1 means the 2nd oldest node.
 * ...
 * n-1 means the nth oldest node.
 *
 * -1 means the youngest node.
 * -2 means the 2nd youngest node.
 * ...
 * -n means the nth youngest node.

The interfaces will change slightly to be consistent with ClusterPartition:

InetAddress pickSingleton(Vector nodes);

Also this one without parameter may remain by calling ClusterPartition.getCurrentView() to get the Vector of nodes.

/**
 * Conducts an election and returns whether the managed singleton
 * is the master.
 */
public boolean isElectedMaster()
{
 return pickSingleton(this.mPartition.getCurrentView()).
 equals(this.mPartition.getNodeAddress());
}

I haven't figured out how to do unit test on this but am working on it.

[brian.stansberry]

"AlexFu.Novell" wrote:
one implementation class
org.jboss.ha.singleton.HASingletonElectionPolicySimple which elects the master node based on attribute "position" defined in the MBean descriptor. Here is how position is defined

Good. If "position" is undefined, it should default to 0, which is the current behavior.

The interfaces will change slightly to be consistent with ClusterPartition:

InetAddress pickSingleton(Vector nodes);

The return type should be org.jboss.cluster.ha.framework.interfaces.ClusterNode -- the contents of the Vector are actually ClusterNode objects.

I haven't figured out how to do unit test on this but am working on it.

Take a look at org.jboss.test.cluster.test.DRMTestCase.testIsMasterReplica() and others in that class -- it uses a mock HAPartition that doesn't actually communicate over the network; the test driver then programatically changes the view.

Another approach is to implement http://jira.jboss.com/jira/browse/JBAS-2560

[alexfu.novell]

The code change has been checked in. The sample configuration can be found at: testsuite/src/resources/ha/electionpolicy/META-INF/jboss-service.xml.

A HASingleton mbean will look like this:

 <mbean code="org.jboss.ha.singleton.examples.HASingletonMBeanExample"
 name="jboss.examples:service=HASingletonMBeanExample_1">
 </mbean>

 <mbean code="org.jboss.ha.singleton.HASingletonElectionPolicySimple"
 name="jboss.examples:service=HASingletonMBeanExample-HASingletonElectionPolicySimple_1">
 <attribute name="Position">0</attribute>
 </mbean>

 <mbean code="org.jboss.ha.singleton.HASingletonController"
 name="jboss.examples:service=HASingletonMBeanExample-HASingletonController_1">
 <depends>jboss:service=${jboss.partition.name:DefaultPartition}</depends>
 <depends>jboss.examples:service=HASingletonMBeanExample_1</depends>
 <depends optional-attribute-name="HASingletonElectionPolicyMBean"
 proxy-type="attribute">jboss.examples:service=HASingletonMBeanExample-HASingletonElectionPolicySimple_1</depends>
 <attribute name="PartitionName">${jboss.partition.name:DefaultPartition}</attribute>
 <attribute name="TargetName">jboss.examples:service=HASingletonMBeanExample_1</attribute>
 <attribute name="TargetStartMethod">startSingleton</attribute>
 <attribute name="TargetStopMethod">stopSingleton</attribute>
 <attribute name="TargetStopMethodArgument">true</attribute>
 </mbean>

[brian.stansberry]

Great. Thanks :-)

If the user doesn't inject an HASingletonElectionPolicy, will HASingletonController default to creating an instance of HASingletonPolicySimple with a position of 0?

[alexfu.novell]

No. It doesn't create a default HASingletonElectionPolicySimple. But it will use the old logic so the behavior should be equivalent to HASingletonElectionPolicySimple with position of 0.

This situation is tested in the unit test.

If the user doesn't inject an HASingletonElectionPolicy, will HASingletonController default to creating an instance of HASingletonPolicySimple with a position of 0?