11 Replies Latest reply on Jun 15, 2017 2:57 PM by Preeta Kuruvilla

    Wildfly 10.1-Final with load-balancing feature

    Preeta Kuruvilla Newbie

      I have managed to configure a sample cluster setup with wildfly-10.1-Final. The undertow was configured in domain.xml for the load-balancer feature. Attached is the domain.xml and host.xml.

       

      I have this question: Our application would need the wildfly load-balancer to manage two host that has the JMS. This communication with the two hosts will happen over http-remoting.

       

      Hence in domain.xml, I am planning to have the below changes highlighted in "RED". Let me know if this is okay and it will work. Or do I have to do some other change to make it working with http upgrade instead of ajp.

       

      <subsystem xmlns="urn:jboss:domain:modcluster:2.0">

                      <mod-cluster-config advertise-socket="modcluster" connector="http">

                          <dynamic-load-provider>

                              <load-metric type="requests"/>

                          </dynamic-load-provider>

                      </mod-cluster-config>

                  </subsystem>

                  <subsystem xmlns="urn:jboss:domain:undertow:3.1">

                      <buffer-cache name="default"/>

                      <server name="default-server">

                          <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true"/>

                          <http-listener name="management" socket-binding="mcmp-management" enable-http2="true"/>

                          <host name="default-host" alias="10.76.80.29">

                              <location name="/app" handler="my-handler"/>

                              <filter-ref name="load-balancer"/>

                          </host>

                      </server>

                      <servlet-container name="default"/>

                      <handlers>

                          <reverse-proxy name="my-handler">

                              <host name="host1" outbound-socket-binding="remote-host1" scheme="http-remoting" path="/sample" instance-id="myroute"/>

                              <host name="host2" outbound-socket-binding="remote-host2" scheme="http-remoting" path="/sample" instance-id="myroute"/>

                          </reverse-proxy>

                      </handlers>

                      <filters>

                          <mod-cluster name="load-balancer" management-socket-binding="mcmp-management" advertise-socket-binding="modcluster" enable-http2="true"/>

                      </filters>

                  </subsystem>

       

       

       

      <socket-binding-group name="load-balancer-sockets" default-interface="public">

                  <socket-binding name="http" port="${jboss.http.port:8080}"/>

                  <socket-binding name="https" port="${jboss.https.port:8443}"/>

                  <socket-binding name="mcmp-management" interface="public" port="${jboss.mcmp.port:8090}"/>

                  <socket-binding name="modcluster" interface="public" multicast-address="224.0.1.105" multicast-port="23364"/>

                  <outbound-socket-binding name="remote-host1">

                      <remote-destination host="10.76.82.38" port="<port for http-remoting=8181>"/>

                  </outbound-socket-binding>

                  <outbound-socket-binding name="remote-host2">

                      <remote-destination host="10.106.3.215" port="<port for http-remoting=8181>"/>

                  </outbound-socket-binding>

              </socket-binding-group>

          </socket-binding-groups>

        • 1. Re: Wildfly 10.1-Final with load-balancing feature
          Justin Bertram Master

          What exactly are you trying to load-balance with regards to JMS?  Are you just load-balancing the JNDI requests?  Assuming you've configured Artemis to be clustered I'm not sure why this would be necessary as Artemis already has load-balancing functionality built-in to the cluster.

          • 2. Re: Wildfly 10.1-Final with load-balancing feature
            Preeta Kuruvilla Newbie

            Hi Justin,

             

            Thanks for responding. Well I was not aware that Artemis already has load-balancing functionality built in to cluster. I actually am yet to try out JMS clustering with wildfly 10.1.

             

            Earlier I tried the JMS clustering (hornetq) with wildfly 8.2 and let me attach the design with this so you know how I managed to use nginx as load-balancer.

             

            Please take a look at the attached pic. So basically I used a global load-balancer as nginx and then there were local load-balancers managing the live and backup servers.

             

            Please let me know if there is a better way I can follow with wildfly 10.1 that uses Activemq

             

            Thanks,

            Preeta

            • 3. Re: Wildfly 10.1-Final with load-balancing feature
              Justin Bertram Master

              Well I was not aware that Artemis already has load-balancing functionality built in to cluster. I actually am yet to try out JMS clustering with wildfly 10.1.

              HornetQ in Wildfly 8.2 also has the same load-balancing functionality.

               

              Please take a look at the attached pic. So basically I used a global load-balancer as nginx and then there were local load-balancers managing the live and backup servers.

              The picture doesn't make clear what exactly is going through the load-balancer.  Is it just balancing JNDI lookups?  Please clarify.

              • 4. Re: Wildfly 10.1-Final with load-balancing feature
                Preeta Kuruvilla Newbie

                I am pasting below the ngnix.conf for your reference:

                 

                 

                events {  }

                 

                http {

                map $http_upgrade $connection_upgrade {

                        default upgrade;

                        ''      close;

                    }

                    include       mime.types;

                    default_type  application/octet-stream;

                 

                 

                 

                    server {

                        listen       8181;

                        server_name  10.76.82.38; // nginx load-balancer

                 

                 

                 

                     

                     

                        location / {

                            proxy_pass        http://jboss;

                            proxy_http_version 1.1;

                            proxy_set_header Upgrade $http_upgrade;

                            proxy_set_header Connection $connection_upgrade;

                            proxy_next_upstream  error timeout invalid_header http_500;

                            proxy_connect_timeout 2;

                            proxy_set_header  Host $host;

                            proxy_set_header  X-Real-IP $remote_addr;

                            proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;

                         

                        }

                 

                 

                   }

                 

                   upstream jboss {

                      # Sticky session

                      ip_hash;

                 

                      server 10.76.82.120:8080; // wildfly node1 (JMS)

                      server 10.76.82.121:8080; // wildfly node2 (JMS)

                   }

                 

                }

                 

                 

                Also the domain.xml of PSC application that's on a seperate VM. This PSC application needs to interact with the JMS cluster using the JMS-JNDI url.

                 

                Below please find the section of domain.xml

                 

                <socket-binding-group name="ha-sockets" default-interface="public">

                            <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/>

                            <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/>

                            <socket-binding name="ajp" port="${jboss.ajp.port:8009}"/>

                            <socket-binding name="http" port="${jboss.http.port:8080}"/>

                            <socket-binding name="https" port="${jboss.https.port:8443}"/>

                            <socket-binding name="jgroups-mping" port="0" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/>

                            <socket-binding name="jgroups-tcp" port="7600"/>

                            <socket-binding name="jgroups-tcp-fd" port="57600"/>

                            <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>

                            <socket-binding name="jgroups-udp-fd" port="54200"/>

                            <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>

                            <socket-binding name="txn-recovery-environment" port="4712"/>

                            <socket-binding name="txn-status-manager" port="4713"/>

                            <outbound-socket-binding name="remote-http">

                                <remote-destination host="10.76.82.38" port="${jboss.http.port:8181}"/>

                            </outbound-socket-binding>

                            <outbound-socket-binding name="mail-smtp">

                                <remote-destination host="localhost" port="25"/>

                            </outbound-socket-binding>

                        </socket-binding-group>

                • 5. Re: Wildfly 10.1-Final with load-balancing feature
                  Justin Bertram Master

                  I'm not really concerned with the nginx.conf.  I just want to know what's going through the load-balancer from an application perspective.  Are you able to answer this question?

                  • 6. Re: Wildfly 10.1-Final with load-balancing feature
                    Preeta Kuruvilla Newbie

                    I have added a section of domain.xml of our application  in my previous message :

                     

                    <outbound-socket-binding name="remote-http">

                                    <remote-destination host="10.76.82.38" port="${jboss.http.port:8181}"/>

                                </outbound-socket-binding>

                     

                     

                    So its the JMS jndi url that connects to the JMS through the load balancer. I hope I made it clear this time.

                     

                    So, actually yes its just balancing JNDI lookups.

                    • 7. Re: Wildfly 10.1-Final with load-balancing feature
                      Justin Bertram Master

                      I think you may be confused about how JMS and JNDI are related.  They are both 100% independent.  When you look something up in JNDI it could be any kind of resource.  There's no such thing as "JMS jndi," per se.  The JMS specifications sets forth the convention that admin objects (e.g. connection factories and destinations) can be looked up via JNDI.

                       

                      Furthermore, when a HornetQ or Artemis connection factory is looked up via JNDI the client simply gets back a stub which is used internally to create a connection to the broker.  Depending on the configuration that connection may or may not use the same host name and port as the JNDI lookup.  Again, the two things are completely independent.

                       

                      Couple of points...

                      • As noted previously, both HornetQ and Artemis have load-balancing functionality built into the cluster so if the only thing you're pushing through your load-balancer is JMS-related JNDI lookups then you can probably just ditch it completely and simplify your architecture.  When clients use a connection factory then the broker they connect to will be determined by the load-balancing-policy configured on the connection factory (round-robin by default).
                      • If you're connecting to the JMS broker from another instance of Wildfly then you should leverage the power of Java EE on the client's application server.  For example, if you're consuming messages then you should use MDB, and if you're sending messages then you should use a local <pooled-connection-factory>.
                      • It's not clear to me that you actually need load-balancing at this point.  A single instance of HornetQ or Artemis can handle thousands of messages per second.  You could potentially simplify your architecture further by having just a single live and backup pair.
                      • 8. Re: Wildfly 10.1-Final with load-balancing feature
                        Preeta Kuruvilla Newbie

                        That was a great info. Thanks a lot. I just managed to get our application work with Wildfly 10.1-Final (standalone-setup). Next will be setting up our application on VM1 with JMS on VM2 and VM3. So I will try configuring the load-balancing as you mentioned without using the external load-balancer like nginx.

                         

                        I will have VM2 (JMS) with live1 and backup2 servers and VM3(JMS) with live2 and backup1 servers.

                         

                        Once I get those up and working properly, I will discuss with our team on what you mentioned that we can do with just one live server and one backup server without the need of load-balancing.

                         

                        Thanks,

                        Preeta

                        • 9. Re: Wildfly 10.1-Final with load-balancing feature
                          Preeta Kuruvilla Newbie

                          Hi Justin,

                           

                          Could you please take a look at this issue Wildfly 10.1-Cluster deployment failure -Services with missing/unavailable dependencies" => undefined ?

                           

                          Once I am done with setting up a cluster with wildfly 10.1, I will be able to get back to this JMS cluster.

                           

                          Thanks,

                          Preeta

                          • 10. Re: Wildfly 10.1-Final with load-balancing feature
                            Justin Bertram Master

                            I really only work with messaging technology so I don't have the expertise to help you with the issue you linked.

                            • 11. Re: Wildfly 10.1-Final with load-balancing feature
                              Preeta Kuruvilla Newbie

                              oh I see. Thanks. I will be coming back to this one soon once my cluster is setup.

                               

                              -preeta