Clustering Management Abstractions

[brian.stansberry]

I'm opening this thread to start discussion of what the key clustering constructs are that are of interest to the clustering and JBoss ON teams. My primary goal is to gain clarity around the key concepts so we can use them in discussions of the Profile Service and the MetaDataRepository. As a side benefit I'd like to derive some well-understood terms that we can use to name these constructs in docs, training materials, etc.

Basic concepts I see are:

1) Domain -- all of the servers relevant to the management tools. (I'd love to see a more precise definition of this if there is one somewhere).

2) Node -- an individual JBoss instance.

3) Cluster -- a set of nodes able to recognize and communicate with each other for the purposes of providing HA and scalability. E.g. a set of nodes with a JGroups channel that has the same configuration. We also often call this a Partition.

4) ??? (call it a Cell for now) -- a set of nodes within a Cluster that are meant to communicate with each other to support HA and/or scalability for one or more services. Goes beyond the requirements of a Cluster in that members of a Cell are expected to be able to interoperate in a more complex way than mere exchange of messages initiated by the JGroups protocols. E.g. Cell members may need to replicate state, and therefore the classes needed to unmarshal the state would need to be available on all nodes in the cell.

Many Clusters will probably only have one Cell. Reasons for multiple Cells include:

a) Heterogeneous deployments (See some of the discussion in the JBossCache Buddy Replication thread (http://www.jboss.com/index.html?module=bb&op=viewtopic&t=78308 related to ReplicationGroups and marshalling).
b) Topology issues, e.g. the Cluster spans a WAN but we want to keep as much traffic local as possible, or we want state replication to be done to nodes on another power supply.


Nodes can belong to more than one Cell and more than one Cluster.

Another concept is "all nodes discoverable by the management tool". Perhaps this is the "Domain", although my feeling is "Domain" is meant to be more abstract. Perhaps "Cluster" is the correct term for "all nodes discoverable by the management tool", and "Partition" should be used for the Cluster concept I defined above.

[dimitris]

I would interpret Domain as an arbitrary grouping of Nodes, or "Stuff" in the more general sense (e.g. an httpd server, a jboss instance and a db).

This can be for practical reasons (e.g. devide departments, assign access/management rights, perform group operations). Sub-domains maybe possible too. The JBossON people will probably have a better definition for this.

I'm not sure about the usefullness of a "Cell", in that you get complex ways of communicating within a cluster, for most services, anyways.

[brian.stansberry]

I'll gladly bow to the definition the ON guys give to "domain" :) I think that need for sub-domains is highly likely, where again the subdomain is an arbitrary set of nodes.

A subdomain is different from the clustering abstractions in that a cluster is not completely arbitrary -- the nodes in a cluster need to share a communication channel.

It seems the clustering abstractions are more a set of services that can be applied to a set of servers. The services have something of a hierarchical nature. The basic service is a communication channel and associated group management functions (e.g. a JGroups JChannel). The next level is provides state replication and group rpc services to a subset of cluster nodes who can exchange classes. The third layer are the various familiar services that want to take advantage of state replication/group rpc layer. E.g. a distributable webapp that uses the state replication layer.

How our management tools will allow those services to be mapped to a domain, to one or more subdomains, or to arbitrary nodes is an open question.

So, I think what you're saying is the problem with the 'Cell' concept is that you can't really group the 2nd level services very well, so 'Cell' doesn't meaningfully encapsulate anything more than a particular service.

E.g., a particular set of nodes has:

JBossCache for session repl, with buddy replication to ReplicationGroup 'A'
JBossCache for EJB3 entity repl; not buddy replication based

Another set of nodes using the same channel has:

JBossCache for session repl, with buddy replication to ReplicationGroup 'B'
JBossCache for EJB3 entity repl; not buddy replication based

There is no logical 'Cell' here, both sets use the same entity replication service. You're better off doing something like creating 2 subdomains, applying the 'A' session cache to 1 subdomain, the 'B' to the other, and the entity cache to both.

[ghinkle]

From a management perspective I think people want to see all the service configurations together. The important part of a domain is its use as a logical model and the ability to organize the configurations of its parts. Its fine to target certain services to a subset of the nodes in a domain (a sub-domain), but I think the most convenient definition would be that every node in domain shares all configuration data.

I can target session replication to ReplicationGroupA and entities to ReplicationGroupB, but both should be part of the overall domain config. The only difference between nodes should be their identity and membership in these sub-domain/replication groups. I want to be able to define that replication group and then add any node in my domain to it in the tool.