1 2 Previous Next 17 Replies Latest reply on Nov 15, 2002 11:30 AM by farooqui

    Federation support, clustering, fail over, etc.

    nphelps

      This comment was posted by Hiram Chirino on jboss-group mailing list. I'm reposting it here to facilitate discussion.

      Let me also add to you list of requirements:
      - the ability to have a network or federation of JMS servers.

        • 1. Re: Federation support, clustering, fail over, etc.
          nphelps

          This comment was posted by Marc Fleury on jboss-group mailing list. I'm reposting it here to facilitate discussion.

          > 'clustering' is one of those terms that can be applied
          > to many things. if you load balance a queue, but
          > don't provide failover, is it 'clustered'??

          We need cache replication on the server side to replicate queues smartly, that is Bela ban's Cache on JBoss effort.

          We need client interceptors ala EJB to provide fail-over

          > JavaGroups will definitely play a KEY role in the
          > intra-cluster communication. It might also fit the
          > bill for inter-federation communication, but that will > need to be explored.

          Talked to Bela while in SF. We agree that he is going to write a quick and dirty implementation of Caches/JBoss and make it fancy down the road. Separate email on that.

          marc f

          • 2. Re: Federation support, clustering, fail over, etc.
            nphelps

            This comment was posted by Bela Ban on jboss-group mailing list. I'm reposting it here to facilitate discussion.


            > This is high on my own list as well. Some people I've talked to see
            > federation and clustering as analogous.


            Most people I've talked to want 'clustering' and 'replication' for JMS,
            but when you ask them what they actually mean they cannot give any clear
            specs. I guess the buzzwords just sound good.


            > However, I've always thought it
            > would be idea to provide clustering between several JMS servers (in
            > other words, a queue defined on one server is defined on all in the
            > cluster,
            > etc.)
            > as well as the support for federating the clusters into a network. In
            > other
            > words, you'd have a federated network of JMS clusters that could
            > communicate
            > via multicast or direct sockets.


            I can give you some input wrt requirements. Sacha, Roman and myself had
            a discussion on reqs about half a year ago. I guess you should just go
            ahead and implement what you think are good semantics, then let people
            complain after the fact (like clustering :-)).

            > Incidentally, don't you think JavaGroups might fit the bill to provide
            > federation services much like it currently provides clustering
            > services for JBoss?


            I do ! :-) Don't hesitate to ask, I can give you all the support you
            need from my side.
            Cheers,

            --
            Bela Ban

            • 3. Re: Federation support, clustering, fail over, etc.
              nphelps

              This is high on my own list as well. Some people I've talked to see federation and clustering as analogous. However, I've always thought it would be idea to provide clustering between several JMS servers (in other words, a queue defined on one server is defined on all in the cluster, etc.) as well as the support for federating the clusters into a network. In other words, you'd have a federated network of JMS clusters that could communicate via multicast or direct sockets.

              Incidentally, don't you think JavaGroups might fit the bill to provide federation services much like it currently provides clustering services for JBoss?

              Thanks,

              Nathan

              • 4. Re: Federation support, clustering, fail over, etc.
                nphelps

                This comment was posted by Hiram Chirino on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                > This is high on my own list as well. Some people I've talked to see
                > federation and clustering as analogous. However, I've always thought
                > it

                'clustering' is one of those terms that can be applied to many things. if you load balance a queue, but don't provide failover, is it 'clustered'??

                > would be idea to provide clustering between several JMS servers (in
                > other words, a queue defined on one server is defined on all in the
                > cluster, etc.)

                Agreed, this is the most natural definition.

                > as well as the support for federating the clusters into a network. In
                > other words, you'd have a federated network of JMS clusters that could
                > communicate
                > via multicast or direct sockets.
                >

                Super important when you have different business entities that want to
                communicate via a reliable transport. Example: If Company A sends a PO to
                Company B via JMS message, if Company B's JMS server is down it will not
                affect Company A's JMS server.

                > Incidentally, don't you think JavaGroups might fit the bill to provide
                > federation services much like it currently provides clustering
                > services for
                > JBoss?

                JavaGroups will definitely play a KEY role in the intra-cluster
                communication. It might also fit the bill for inter-federation
                communication, but that will need to be explored.

                Regards,
                Hiram

                • 5. Re: Federation support, clustering, fail over, etc.
                  nphelps

                  This comment was posted by Bill Burke on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                  > 'clustering' is one of those terms that can be applied to many
                  > things.
                  > if you load balance a queue, but don't provide failover, is it
                  > 'clustered'??
                  >

                  The most important thing is that if a user of JBoss wants to scale by adding another machine JMS should be able to handle this. This is not the case right now and needs to be addressed. I was forced to recommend a commercial JMS implementation because of this.

                  > JavaGroups will definitely play a KEY role in the intra-cluster
                  > communication. It might also fit the bill for inter-federation
                  > communication, but that will need to be explored.
                  >

                  For JBoss clustering, we've been trying to avoid having the remote client use JavaGroups. This is because the client must be a member of the Group/Channel to do anything. Maybe this has changed in the current version of JG. Sacha? Bela? comments?

                  Bill

                  • 6. Re: Federation support, clustering, fail over, etc.
                    nphelps

                    This comment was posted by Sacha Labourey on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                    > For JBoss clustering, we've been trying to avoid having the remote
                    > client use JavaGroups. This is because the client must be a member
                    > of the Group/Channel to do anything. Maybe this has changed in the
                    > current version of JG. Sacha? Bela? comments?

                    No, it hasn't changed.

                    • 7. Re: Federation support, clustering, fail over, etc.
                      nphelps

                      This comment was posted by Bela Ban on jboss-group mailing list. I'm reposting it here to facilitate discussion.


                      > This is high on my own list as well. Some people I've talked to see
                      > federation and clustering as analogous.


                      Most people I've talked to want 'clustering' and 'replication' for JMS,
                      but when you ask them what they actually mean they cannot give any clear
                      specs. I guess the buzzwords just sound good.


                      > However, I've always thought it
                      > would be idea to provide clustering between several JMS servers (in
                      > other words, a queue defined on one server is defined on all in the
                      > cluster,
                      > etc.)
                      > as well as the support for federating the clusters into a network. In
                      > other
                      > words, you'd have a federated network of JMS clusters that could
                      > communicate
                      > via multicast or direct sockets.


                      I can give you some input wrt requirements. Sacha, Roman and myself had
                      a discussion on reqs about half a year ago. I guess you should just go
                      ahead and implement what you think are good semantics, then let people
                      complain after the fact (like clustering :-)).

                      > Incidentally, don't you think JavaGroups might fit the bill to provide
                      > federation services much like it currently provides clustering
                      > services for JBoss?


                      I do ! :-) Don't hesitate to ask, I can give you all the support you
                      need from my side.
                      Cheers,

                      --
                      Bela Ban

                      • 8. Re: Federation support, clustering, fail over, etc.
                        nphelps

                        This comment was posted by Bela Ban on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                        Unless you have few clients and the clients need to talk to each other or know about each other, I would not use JavaGroups for client-server communication, but rather suggest something along the lines of what Sacha and Bill did with Clustering. For server-server communication I obviously recommend JavaGroups.

                        --
                        Bela Ban

                        • 9. Re: Federation support, clustering, fail over, etc.
                          nphelps

                          This comment was posted by Bela Ban on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                          > For JBoss clustering, we've been trying to avoid
                          > having the remote client use JavaGroups. This is
                          > because the client must be a member of the
                          > Group/Channel to do anything. Maybe this has
                          > changed in the current version of JG. Sacha? Bela? >comments?

                          JavaGroups can be configured to use a 'services' architecture where there are 2 groups: a lightweight client-group, in which all clients and one or more servers are member, and a server-group in which only the
                          servers are members. Clients use the client-group to talk to a server, which in turn uses the server-group, e.g. to replicate.

                          However, as mentioned in my prev email, there are issues, e.g.:

                          * Great number of clients, and/or clients not necessarily in the same multicast enabled network
                          * Clients don't need to talk to each other, don't need to know about each other (e.g. no need to get notified when a new client joins/leaves/crashes).

                          I'd suggest using JavaGroups for server-server communication, not client-server comm (although the latter is supported as well).

                          --
                          Bela Ban

                          • 10. Re: Federation support, clustering, fail over, etc.
                            nphelps

                            This comment was posted by Bill Burke on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                            > Most people I've talked to want 'clustering'
                            > and 'replication' for JMS, but when you ask them what
                            > they actually mean they cannot give any clear specs. I
                            > guess the buzzwords just sound good.

                            This is more than buzzwords and checkmarks. You cannot "fake" clustering with JMS as you could with EJBs, you cannot add another machine to your "cluster" to scale if you're using JMS. This is a serious problem. The paying customers I've actually talked to have definate requirements.

                            - If a node goes down, subscriber remains connected seemlessly..

                            - publishers seemlessly failover (as well as any other api you're calling) This is there right now but I don't think anybody has used it. Generalizing proxy creation with the aspect framework will help with this functionality.

                            - Any node can take over on a failure. Meaning, durable subscription, any message cache must be replicated.

                            Bill

                            • 11. Re: Federation support, clustering, fail over, etc.
                              nphelps

                              This comment was posted by Nathan Phelps on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                              >- If a node goes down, subscriber remains connected seemlessly..

                              Sounds like the duty of client interceptors.

                              >- publishers seemlessly failover (as well as any other
                              > api you're calling) This is there right now but I
                              > don't think anybody has used it. Generalizing proxy
                              > creation with the aspect framework will help with this
                              > functionality.

                              Frankly, I'm struggling to apply the dynamic client proxies to JMS since the JMS API is static in nature. I fail to see how proxy creation is necessary at all. Can someone explain this to me? Additionally, I'm getting confused with the mixed use of the terminology of "aspect" and "interceptor." Do you consider these terms interchangeable?

                              >- Any node can take over on a failure. Meaning,
                              >durable subscription, any message cache must be
                              > replicated.

                              Yes, I prefer this architecture to the master/slave relationship that many organizations implement for clustering and failover. Any node in a cluster should be able to take over for any other node in a cluster.

                              • 12. Re: Federation support, clustering, fail over, etc.
                                nphelps

                                This comment was posted by Bill Burke on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                                > Sounds like the duty of client interceptors.
                                >

                                IMHO, its the duty of the server. The server should replicate the subscribers across the cluster. What I mean by this is that the server replicates the client proxies of the subscribers across the cluster (replicated RMI proxies for instance).


                                > Frankly, I'm struggling to apply the dynamic client proxies to JMS
                                > since the JMS API is static in nature. I fail to see how proxy
                                > creation is necessary at all. Can someone explain this to me?
                                > Additionally, I'm getting confused
                                > with the mixed use of the terminology of "aspect" and
                                > "interceptor." Do you
                                > consider these terms interchangeable?
                                >

                                Proxy is important because you can attach interceptor behavior to your clients and re-use the Invoker MBean architecture. For instance, you could re-use the client Tx and Security interceptors to automagically propagate transactional and security information across the wire. ISVs can add proprietary APIs (attaching interfaces via new interceptors) to JMS. Also, encapsulating Transactional and Security propagation in one interceptor object helps with maintainability as this integration/code changes. Proxies give us power and flexibility and managability and reusability even with a static api.

                                The way I see it is, Topics and Queues are MBeans and receive everything through a generic MBean invocation. If you do it this way, communication is unified/generalized across EJB, MBeans, and JMS and you can leverage existing code.

                                BTW, interceptor is a subset of aspect. Aspects have interceptors but also the ability to attach additional interfaces to the object. Go to the aspect forum to get more information.

                                > Yes, I prefer this architecture to the master/slave relationship that
                                > many organizations implement for clustering and failover. Any node
                                > in a cluster should be able to take over for any other node in a
                                > cluster.

                                Yes, this is the way EJBs and the Web tier are working.

                                • 13. Re: Federation support, clustering, fail over, etc.
                                  nphelps

                                  This comment was posted by Dain Sundstrom on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                                  > Sounds like the duty of client interceptors.

                                  I agree, but I'm not the expert. I really see client to server JMS as a simple RPC class that puts a message in the server queue and returns. So it should be able to reuse our client failover code.

                                  > Frankly, I'm struggling to apply the dynamic client
                                  > proxies to JMS since the JMS API is static in nature.
                                  > I fail to see how proxy creation is necessary at all.
                                  > Can someone explain this to me?

                                  I would agree that it is not necessary, but it allows easier reuse. Also Bill (I think it was Bill that came up with this) has a plan to allow clients to cast a proxy to an addin management interface. The only example I can think of is for entities (surprise surprise). For a cmp entity you have a read-ahead configuration that is basically set on the server. The problem is clients need to change it for application
                                  tuned loading, so you add another interface to the proxy when you generate it on the server. Then the client casts the proxy to it and can configure the readahead attributes. For example:

                                  ((ReadAheadConfiguration)myCMPEntity).setLoadGroup("AccountingView");

                                  Follow?

                                  > Additionally, I'm getting confused with the mixed use
                                  > of the terminology of "aspect" and "interceptor." Do
                                  > you consider these terms interchangeable?

                                  In Jboss the terms are fairly interchangeable. Interceptors are really the manifestation of an aspect.

                                  -dain

                                  • 14. Re: Federation support, clustering, fail over, etc.
                                    nphelps

                                    This comment was posted by Marc Fleury on jboss-group mailing list. I'm reposting it here to facilitate discussion.

                                    > IMHO, its the duty of the server. The server should
                                    > replicate the subscribers across the cluster.

                                    Yep

                                    >
                                    > Proxy is important because you can attach interceptor
                                    > behavior to your clients and re-use the Invoker MBean
                                    > architecture. For instance, you could re-use the
                                    > client Tx and Security interceptors to automagically
                                    > propagate transactional and security information
                                    > across the wire. ISVs can add proprietary APIs
                                    > (attaching interfaces via new interceptors) to JMS.
                                    > Also, encapsulating Transactional and Security
                                    > propagation in one interceptor object helps with
                                    > maintainability as this integration/code changes.
                                    > Proxies give us power and flexibility and managability
                                    > and reusability even with a static api.
                                    >
                                    > The way I see it is, Topics and Queues are MBeans and
                                    > receive everything through a generic MBean
                                    > invocation. If you do it this way, communication is
                                    > unified/generalized across EJB, MBeans, and JMS and
                                    > you can leverage existing code.

                                    Nathan, you didn't pay attention to my presentation at the jug did you? DP allows us to transform a typed signature into a detyped 'invoke' and then we can string together some behavior. Replicating failover logic, using detyped invoker etc. I expect your effort to yield an interesting one way invoker that we are missing today.

                                    > BTW, interceptor is a subset of aspect. Aspects have
                                    > interceptors but also the ability to attach additional
                                    > interfaces to the object. Go to the aspect forum to
                                    > get more
                                    > information.

                                    I think this is a good example of how I want people to use the forums in the future. WE can track the discussions in the particular forums and the guy responsible can give the skinny on what is going on. I will try to work on that soon.

                                    marcf

                                    1 2 Previous Next