1 2 3 Previous Next 30 Replies Latest reply on Apr 9, 2010 8:17 PM by jason.greene Go to original post
      • 15. Re: To scope or not to scope (domain.xml)
        emuckenhuber

        Brian Stansberry wrote:


        Since we're in the mode of defining things, what is a "multi-domain notion?"  I see it as a an ability to manage within the same tool multiple sets of servers where each set has it's own management policy. The sets of servers are conceptually distinct (hence the separate management policy / domain) but, generally for operational ease of use reasons, users wish to manage multiple domains from the same tool.

        Just to clearify what i meant with "multi-domain" - is that in this case a node can be part of more management domains.

         

        The difference i see is that in a multi-domain notion you have multiple domains - where the "managable unit" would be the domain itself.

        In case of server-groups you have a single domain - however the "manageable unit" is more or less a server-group.

         

        In case a server should be the "managable unit" then we don't need a domain model - right?

        • 16. Re: To scope or not to scope (domain.xml)
          jason.greene

          Scott Stark wrote:

           

          Brian Stansberry wrote:

          Since we're in the mode of defining things, what is a "multi-domain notion?"  I see it as a an ability to manage within the same tool multiple sets of servers where each set has it's own management policy. The sets of servers are conceptually distinct (hence the separate management policy / domain) but, generally for operational ease of use reasons, users wish to manage multiple domains from the same tool.
          Right, which is why I don't see this as impacting what the domain.xml looks like. This notion exists above the servers and profileservice. We are told what domain we are operating in.

           

          In the discussion we had at the OA&M meeting with the JON folks, we thought that the AS perspective of management ends with the domain . "Multi-domain" management was something that JON would offer, as a compelling difference. So our admin console, cli, and domain.xml interfaces would be limited to a domain. I still think this is the "right" approach.

          • 17. Re: To scope or not to scope (domain.xml)
            jason.greene

            Brian Stansberry wrote:

             

            Jason Greene wrote:

             

            When you think about it, a server-group, and a cluster are really the same thing, the only difference is that the cluster has an additional set of services that a basic group does not.

             

            Yes, but isn't the set of capabilities that a server has defined by its profile? We could say that defining a server as part of a cluster means that additional capabilities are added beyond what's in the profile. But then we are dividing the definition of a profile into two places. Also, exactly what capabilities should be added if a server is in a cluster is unclear

             

            I think there is a definite overlap between the definition of a profile and the definition of a grouping. The only way I can see having both in domain.xml is to have the profile function as a template like you mention earlier (or to pull all common config stuff out of a group).

             

            Jason Greene wrote:

             

            When you think about it, a server-group, and a cluster are really the same thing, the only difference is that the cluster has an additional set of services that a basic group does not.

             

            Yes, but isn't the set of capabilities that a server has defined by its profile? We could say that defining a server as part of a cluster means that additional capabilities are added beyond what's in the profile. But then we are dividing the definition of a profile into two places. Also, exactly what capabilities should be added if a server is in a cluster is unclear.

             

            I think this depends on the definition of a profile. To me our previous definition of profile focused much more on internal implementation details (how to build a server with ejb3 (which jars etc)). When I look at the domain model I am looking at it from a configuration standpoint. Specifying things like "jndi should be clustered on port blah blah blah". There should be a thread pool with X threads, etc

             

            A very simple concept of a cluster is that it is a server-group that also exposes some predefined configuration properties that need to be kept consistent throughout the group. (A server-group would not have such properties by default, although users could add custom ones.) Things such as:

             

            • Group name (current -g)
            • Type of intra-group communication to use by default (udp, tcp)
            • Default interface to use for intra-group communication
            • Default multicast address to use
            • other defaults less commonly changed


            Right this is exactly what I was getting at. It also would represent what resources are available (datasources etc).

             

             

            A cluster might also have some management capabilities besides what is available to a server-group, although I'm not sure what. It should be possible for example to perform a (rolling) redeployment to a server-group.

             

            A separate discussion is how communication will occur between servers who are not members of a cluster. There needs to be a group communication mechanism that spans the entire domain.

             

            I think the best way to address this is that everything in a domain will have two communication services that are distinct from everything used in clustering:

            1. Configuration syncing
            2. Deployment

             

            These services could be implemented in a number of ways, but will likely need to be a simple/specialized protocol. It might even necessitate a node agent of some sort. However I think thats a conversation for later, the important thing is the notion that synching must be possible to non-clustered nodes.

            This was what I meant by the cluster-service tag, a way to specify how a group would cluster a particular service (maybe not all services should be clustered). Although it was just something I through out there as an example. A better way likely exists

             

            Currently how we cluster a particular service is defined in that service's configuration, it's not something applied externally. And the details of what needs to be configured are often disparate between different services, so I'm not sure those configs can profitably be externalized into some common configuration element.

             

            Ah, or are you talking about "clustered-service" as a wrapper to describe how to manage the service, e.g. how to (re)deploy across cluster?

             

            Don't read too much into it, it was just  a generalization. Substitute clustered-service with "jgroups-group" or "ha-jndi".

            • 18. Re: To scope or not to scope (domain.xml)
              brian.stansberry

              Jason Greene wrote:

               

              Jason Greene wrote:

               

              When you think about it, a server-group, and a cluster are really the same thing, the only difference is that the cluster has an additional set of services that a basic group does not.

               

              Yes, but isn't the set of capabilities that a server has defined by its profile? We could say that defining a server as part of a cluster means that additional capabilities are added beyond what's in the profile. But then we are dividing the definition of a profile into two places. Also, exactly what capabilities should be added if a server is in a cluster is unclear.

               

              I think this depends on the definition of a profile. To me our previous definition of profile focused much more on internal implementation details (how to build a server with ejb3 (which jars etc)). When I look at the domain model I am looking at it from a configuration standpoint. Specifying things like "jndi should be clustered on port blah blah blah". There should be a thread pool with X threads, etc

               

              The part of our existing profile discussion I was thinking of was what's shown here: http://community.jboss.org/docs/DOC-14742. Basically a fairly compact listing of required capabilities, an indication of where a deploy directory is (which could perhaps be defaulted). Then add stuff like jdbc-resources, jms-resources, threads, etc. That's pretty end-user-oriented; the heavy implementation detail stuff is in http://community.jboss.org/docs/DOC-14743.

               

              If we didn't need the required capabilities bit (e.g. could infer them from the rest of domain.xml content), that's nice. But I don't know if we can do that, not if we allow a deploy dir with contents not fully specified in domain.xml.

              • 19. Re: To scope or not to scope (domain.xml)
                brian.stansberry

                Jason Greene wrote:

                A separate discussion is how communication will occur between servers who are not members of a cluster. There needs to be a group communication mechanism that spans the entire domain.

                 

                I think the best way to address this is that everything in a domain will have two communication services that are distinct from everything used in clustering:

                1. Configuration syncing
                2. Deployment

                 

                These services could be implemented in a number of ways, but will likely need to be a simple/specialized protocol. It might even necessitate a node agent of some sort. However I think thats a conversation for later, the important thing is the notion that synching must be possible to non-clustered nodes.

                 

                Agreed, both with the details and the discussion for later part.

                • 20. Re: To scope or not to scope (domain.xml)
                  emuckenhuber

                  Brian Stansberry wrote:


                  The part of our existing profile discussion I was thinking of was what's shown here: http://community.jboss.org/docs/DOC-14742. Basically a fairly compact listing of required capabilities, an indication of where a deploy directory is (which could perhaps be defaulted). Then add stuff like jdbc-resources, jms-resources, threads, etc. That's pretty end-user-oriented; the heavy implementation detail stuff is in http://community.jboss.org/docs/DOC-14743.

                   

                  If we didn't need the required capabilities bit (e.g. could infer them from the rest of domain.xml content), that's nice. But I don't know if we can do that, not if we allow a deploy dir with contents not fully specified in domain.xml.

                   

                  Having this part of the domain model was basically the reason why i tried to separate "user configuration" from the server side module configuration. We can obviously also change the names in the configuration files, to make things a more obvious.

                   

                  In the end - I think if you configure clustering it would make sense to also deploy the needed service. Where you definitely should not care about which deployments are needed for this. Since we are going to have a domain specific model - each fragment could export a requirement on a module.

                   

                  But, yes - in case we have don't have this part of a separate "module" and deployments are still in deploy dirs, then this would not make much sense.

                  • 21. Re: To scope or not to scope (domain.xml)
                    emuckenhuber

                    Jason Greene wrote:


                    In the discussion we had at the OA&M meeting with the JON folks, we thought that the AS perspective of management ends with the domain . "Multi-domain" management was something that JON would offer, as a compelling difference. So our admin console, cli, and domain.xml interfaces would be limited to a domain. I still think this is the "right" approach.

                    Yes, limiting to a domain seems the right approach. But are we sure that a domain should manage a heterogenous group?

                     

                    Let's put it another way - should embedded jopr manage a heterogenous group? Where is the value addon from JON? How likely is it that you have multiple heterogenous configurations within a company? Isn't that just only a small percentage where you would need that?

                     

                    Grouping obviously seems to be a important. Where having a heterogenous configuration within a domain, makes this quickly quite big and complex. Splitting that up in smaller "manageable units", which have more in common than "just" a domain name - with clearer boundaries/responsibilities could help to make it easier to understand and reduce overall complexity.

                    • 22. Re: To scope or not to scope (domain.xml)
                      brian.stansberry

                      Good input. While I see multiple groups in a homogeneous domain as a "must-have" to support rolling updates, a heterogeneous domain is just a "nice-to-have".  I think this is a topic to clarify with PM input the week of the 19th.

                       

                      We need to be careful though about declaring stuff out of scope that's just going to end up back in scope later. E.g. saying a domain is homogeneous and anything heterogeneous is a JON concern. And then there is a business requirement for JON to support a more expansive view of a domain, a la the multi-tier example I posted above. And then to make that work it ends up having to be shoved back into the domain model with duct tape and baling wire.

                       

                      I'm not saying that will happen, just saying we should be aware of it.

                      • 23. Re: To scope or not to scope (domain.xml)
                        brian.stansberry

                        Emanuel Muckenhuber wrote:

                        In the end - I think if you configure clustering it would make sense to also deploy the needed service. Where you definitely should not care about which deployments are needed for this. Since we are going to have a domain specific model - each fragment could export a requirement on a module.

                         

                        The tricky part about this is the  "not care about which deployments are needed for this" bit. Well, thinking about it maybe not all that tricky. The clustering services come in 4 general "categories"

                         

                        1. Core capabilites (jgroups integration, infinispan integration)
                        2. Remote access stuff (HAPartition for cluster-aware proxies, HA-JNDI). You don't know if this is needed as the requirement comes from
                          • an annotation on an unknown EJB in deploy/
                          • a completely unknown remote client that wants to use HA-JNDI
                        3. HASingleton (which is just a different use case for HAPartition), no added services
                        4. Misc odds and ends

                         

                        So, a <cluster/> element itself implies a need for 1. A couple attributes on <cluster/>, default = true, could control 2 and 3. I'd have to look more closely at the 4 stuff, but I suspect it's either so lightweight we just include it if 2 or 3 are needed, or we can detect the need for it from what other capabilities are required.

                         

                        So, yeah, using <cluster/> as a trigger to add some extra capabilities sounds good.

                        • 24. Re: To scope or not to scope (domain.xml)
                          emuckenhuber

                          One thing which would be interesting to look at is how much configuration can actually be shared between a DataGrid and JEE server-group.

                           

                          I mean if it's just the domain name, then it's rather a grouping feature which is most probably easier to do on a aggregation level (JON) than in the domain model itself. Where you would also be able to share a domain across multiple groups.

                          Also based on this we could also see if multi-tier should be done in one domain, where you cannot really share a server-group between different domains. Or if a hierarchical domain would be a better approach trying to solve this common configuration.

                          • 25. Re: To scope or not to scope (domain.xml)
                            brian.stansberry

                            Emanuel Muckenhuber wrote:

                             

                            One thing which would be interesting to look at is how much configuration can actually be shared between a DataGrid and JEE server-group.

                             

                            Probably not a lot; just the info (e.g. URLs or some info needed for discovery) needed for the JEE servers to contact the data grid. For that particular example, it is just aggregation.

                             

                            I like your way of thinking about this for when we discuss in Westford. We need a use case for why heterogeneous servers need to be in the same domain as opposed to in an aggregate view.

                            • 26. Re: To scope or not to scope (domain.xml)
                              emuckenhuber

                              Brian Stansberry wrote:

                               

                              So, yeah, using <cluster/> as a trigger to add some extra capabilities sounds good.

                              Yeah, that's the goal or at least the direction i would like to go. I would like to look at some more use cases to see what's really needed and if this can be handled in a better way. In the end ProfileService is limited in what it can/should do - so it's the combination of module requirements (deploy only what's needed), on-demand beans (start only what's needed) and something like your on demand profile activation which needs to make sense altogether.

                              • 27. Re: To scope or not to scope (domain.xml)
                                starksm64

                                Brian Stansberry wrote:

                                 


                                This was what I meant by the cluster-service tag, a way to specify how a group would cluster a particular service (maybe not all services should be clustered). Although it was just something I through out there as an example. A better way likely exists

                                 

                                Currently how we cluster a particular service is defined in that service's configuration, it's not something applied externally. And the details of what needs to be configured are often disparate between different services, so I'm not sure those configs can profitably be externalized into some common configuration element.

                                 

                                Ah, or are you talking about "clustered-service" as a wrapper to describe how to manage the service, e.g. how to (re)deploy across cluster?

                                But this is something that needs to change in the sense that basic things like what stack the ejb/web/jms/cluster aware container is using has to be a domain level setting that admins control as well as these stacks in terms of an admin clustering DSL. That config is then mapped onto the container metadata that controls the clustering behavior.

                                • 28. Re: To scope or not to scope (domain.xml)
                                  starksm64

                                  That is really something that is outside of the domain model as I see it, at least in terms of the level of the cluster element in some bean configuration. At the domain level it would show up as a cluster-config-ref or some such in the associated bean container, but doubtful that admins are going to be tweaking bean specific containers.

                                  • 29. Re: To scope or not to scope (domain.xml)
                                    brian.stansberry

                                    Getting to what you describe shouldn't be a huge hurdle; things already largely work this way in AS 5, it's just that the core JGroups and JBC related services, like everything else, are in their own config files.  People don't configure a JGroups stack in container configuration files or annotations; at most they indicate what named cache config to use from a centralized set managed by the CacheManager service; that cache config specifies the name of the JGroups stack to use, which again comes from a centralized set from the ChannelFactory service. Those centralized sets are logically under the control of an admin.

                                     

                                    The stuff I was talking about when I said "the details of what needs to be configured are often disparate  between different services, so I'm not sure those configs can profitably  be externalized into some common configuration element" was things like load balance policies for remote EJBs, eviction configurations for entity caches, etc.