10 Replies Latest reply on Jul 18, 2013 8:23 AM by dlin12

    Jboss 7 in a whole instance level clustering and failover?

    dlin12

      Hi all,

       

      Is it possible to configure a Jboss 7 in a whole instance level clustering and failover?

      The application I am going to build is a web service (jax-ws) calling a  remote statleless sesssion bean which performs persistency using hibernate/jpa. There is no web tier required.

      The stateless session bean can be configured clustering by using the @Clustered annotation.

      What about the web service and hibernate clustering configuration?

       

      Thanks

      David Lin

        • 1. Re: Jboss 7 in a whole instance level clustering and failover?
          rhusar

          Not sure I understand what you mean by "whole instance level".

           

          WS are exposed via HTTP, so similar access patterns apply. If the service is stateless, just deploy more AS instances. Then you typically deploy a load balancer in front of your AS instances.

           

          For Hibernate, if you are using 2nd leve cache you will need to cluster that as well.

          1 of 1 people found this helpful
          • 2. Re: Jboss 7 in a whole instance level clustering and failover?
            dlin12

            Thanks Rado. Does that means I put the  <distributable/> element inside the web.xml will solve web service clustering issue?

            The applicaiton is going to deploy in aws ec2 using aws loader balancer. I assume the modcluster and httpd will not be used in this case? Just a little bit confused about the modadvertiser and proxylist setting.


            • 3. Re: Jboss 7 in a whole instance level clustering and failover?
              rhusar

              Thanks Rado. Does that means I put the  <distributable/> element inside the web.xml will solve web service clustering issue?

              Well, sure you can, but I am not sure how will that help since that app seems to be stateless.

               

              The applicaiton is going to deploy in aws ec2 using aws loader balancer. I assume the modcluster and httpd will not be used in this case? Just a little bit confused about the modadvertiser and proxylist setting.

              I have no personal experience with ELB but I guess you could also deploy your own LB if you need. ELB does not integrate with mod_cluster so you can remove this subsystem from your server profile and just use ELB.

              • 4. Re: Jboss 7 in a whole instance level clustering and failover?
                dlin12

                That is cool. The purpose of using clustering here is for failover. When one application server is shut down for maintence, the load balancer will forward all the new request to the live server, but for the requests being served during the server shut down process, should be able to continue even it is stateless.

                • 5. Re: Jboss 7 in a whole instance level clustering and failover?
                  rhusar

                  Sure, so yeah, you should add the <distributable/> tag.

                   

                  When it comes to ELB and graceful shutdown for maintenance, the graceful shutdown is not 100% graceful. I would recommend to remove the node you are about to shutdown from the LB and only then when there are no more requests on that node, shut it down (not sure if possible and how to configure that). This is one of the problems that mod_cluster solves, by notifing the LB about the application lifecycle.

                  1 of 1 people found this helpful
                  • 6. Re: Jboss 7 in a whole instance level clustering and failover?
                    dlin12

                    Thanks Rado for the quick reponse. I have created a clustering remote stateless session bean with a client repeatly calling it at every 3 seconds and server will be shut down during the call and bring to live again. The jboss version tested is 7.2 final built from source https://github.com/wildfly/wildfly/tree/7.2.0.Final

                    There is not mod_cluster and httpd configure for this test, just clustering ejb in a local pc running two nodes. The remote clustering lookup will fail a few times, I have commentd the issue in https://issues.jboss.org/browse/AS7-5592

                    The command to shutdown server is ./jboss-cli.sh --connect :shutdown, which sometimes hangs with message:

                    14:42:08,310 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000082: Stopping the RpcDispatcher

                    I have to kill it using -9 option.

                    If the remote clustering lookup is working, the client can failover to another server during the server shutdown.

                    I have not tested the web service clustering yet. It seems like the jboss community version does not have a stable version at this moment.

                    • 7. Re: Jboss 7 in a whole instance level clustering and failover?
                      rhusar

                      The issue you linked is not related to

                       

                      14:42:08,310 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000082: Stopping the RpcDispatcher

                      This is "normal" and could happen on shutdown, just wait 30 seconds for the timeout to kick in and shut the server down. Not exactly sure what is the problem, but something in lines that the (session replication) transaction in the process and the server cannot handle it any more since its shutting down.

                      • 8. Re: Jboss 7 in a whole instance level clustering and failover?
                        dlin12

                        Finally the ejb clustering lookup issue resolved by upgrading the jboss-remoting module to 3.2.16GA and jboss-ejb-client module to 1.0.22 as jboss guys suggested in the issue.

                        Very impressed by the jboss community. The ejb clustering works well without the mod_clustering, it can failover to other server if one is down.

                        Does that means the mod_clustering and apache httpd  works only with the web clustering?

                         

                        The second test I am going to write is for web service clustering with mod_clustering (AWS ELB will not be used at this moment)

                        When the <distributable/> tag is used, the console display warning message

                        10:54:52,634 WARN  [org.jgroups.protocols.MPING] (MPING) jboss2/web: discarding discovery request for cluster 'ejb' from jboss2/ejb; our cluster name is 'web'. Please separate your clusters cleanly.

                         

                        I saw you post comment in https://community.jboss.org/thread/199078

                        Make sure to use different multicast group or port for each cache container.

                         

                        How can I make it work?

                        I start jboss in two standardalone modes using the tcp as default stack

                        server1/standalone.sh -c standalone-ha.xml -Djboss.node.name=jboss1

                        server2/standalone.sh -c standalone-ha.xml -Djboss.node.name=jboss2 -Djboss.socket.binding.port-offset=100

                         

                        Thanks


                        • 9. Re: Jboss 7 in a whole instance level clustering and failover?
                          wdfink

                          EJB clustering is a different protocol and handled by JBoss classes. Therefore the client side knows about the cluster topology and can handle the request load-balanced and with failover.

                          The http request can not do similar here you need a loadbalancer instance to handle this.

                           

                          For the WARN message I'm not sure what you try to achieve and what is configured. You might change the configuration and for that the container assume you have different clusters.

                          From what I understand you want to have both instances in the same cluster.

                           

                          Maybe a look to the documentation cluster howto helps.

                          • 10. Re: Jboss 7 in a whole instance level clustering and failover?
                            dlin12

                            Thanks Wolf. I have a much clearer picture about jboss clustering now.

                            For the WARN message, I am still confused. The configure file is using the standalone-ha.xml default setting except using the tcp stack.

                            The application has web service and ejb. The web service clustering is using the <distributable/> in web.xml file. Does that means the web clustering is try to communicate with the ejb clustering?

                            What is the best practice here? Should I use same cluster or different clusters?