12 Replies Latest reply on Mar 14, 2013 7:25 AM by Radoslav Husar

    mod cluster performance

    Gnanaguru Sattanathan Novice

      Hello,

       

      I am doing performance testing with a mod cluster setup with multiple application servers.

       

      I have one modcluster instance which is connected to 6 AS7 nodes. I sending lakhs of requests to the mod cluster url and I could notice the request is being handled by all the 6 AS7 nodes. But when I do the same test by reducing the number of AS7 nodes, say 4 AS7 nodes. I couldnt find any difference in the performance graph.

       

      The performance graph was plotted against time and number of requests. The pattern remains the same for any number of AS7 nodes connected to mod cluster.

       

      Any ideas and suggestions ? I wanted to know whether my approach is right or wrong.

       

      Note: I am using a Async use case to test this, http -> JMS

       

      Regards

      Guru

      @gnanagurus

        • 1. Re: mod cluster performance
          Jean-Frederic Clere Master

          without a better description of what you are testing it is hard to help...

          • 2. Re: mod cluster performance
            Gnanaguru Sattanathan Novice

            Hi Jean,

             

            I have a camel application running on 6 independent JBoss AS7 nodes. Camel application exposes a http endpoint which can receive requests and the response will be sent to a JMS queue.

             

            Now I have connected all the 6 JBoss application server nodes to mod_cluster. I was able to send request to mod cluster which distributes the request to all 6 JBoss AS nodes.

             

            I am sending one lakh requests to mod cluster,  which in turn load balances across all the nodes. But when I decrease the number of JBoss AS7 nodes, I couldnt find any noticable difference.

             

            The performance results of mod cluster with 3 AS nodes and 6 AS nodes are same.

             

            Hope I am clear. Please help.

             

            Regards

            Guru

            @gnanagurus

            • 3. Re: mod cluster performance
              Jean-Frederic Clere Master

              Ok you mean the limit seems to be httpd (mod_cluster), Is the box busy? and what have you configured there?

              • 4. Re: mod cluster performance
                Gnanaguru Sattanathan Novice

                Hi Jean,

                 

                Please find the configuration file..

                 

                #

                # This is the main Apache HTTP server configuration file.  It contains the

                # configuration directives that give the server its instructions.

                # See <URL:http://httpd.apache.org/docs/2.2> for detailed information.

                # In particular, see

                # <URL:http://httpd.apache.org/docs/2.2/mod/directives.html>

                # for a discussion of each configuration directive.

                #

                # Do NOT simply read the instructions in here without understanding

                # what they do.  They're here only as hints or reminders.  If you are unsure

                # consult the online docs. You have been warned. 

                #

                # Configuration and logfile names: If the filenames you specify for many

                # of the server's control files begin with "/" (or "drive:/" for Win32), the

                # server will use that explicit path.  If the filenames do *not* begin

                # with "/", the value of ServerRoot is prepended -- so "logs/foo.log"

                # with ServerRoot set to "C:/Users/Administrator/isova/backup/Apache2.2" will be interpreted by the

                # server as "C:/Users/Administrator/isova/backup/Apache2.2/logs/foo.log".

                #

                # NOTE: Where filenames are specified, you must use forward slashes

                # instead of backslashes (e.g., "c:/apache" instead of "c:\apache").

                # If a drive letter is omitted, the drive on which httpd.exe is located

                # will be used by default.  It is recommended that you always supply

                # an explicit drive letter in absolute paths to avoid confusion.

                 

                 

                #

                # ServerRoot: The top of the directory tree under which the server's

                # configuration, error, and log files are kept.

                #

                # Do not add a slash at the end of the directory path.  If you point

                # ServerRoot at a non-local disk, be sure to point the LockFile directive

                # at a local disk.  If you wish to share the same ServerRoot for multiple

                # httpd daemons, you will need to change at least LockFile and PidFile.

                #

                ServerRoot "C:/Users/Administrator/isova/backup/Apache2.2"

                 

                 

                #

                # Listen: Allows you to bind Apache to specific IP addresses and/or

                # ports, instead of the default. See also the <VirtualHost>

                # directive.

                #

                # Change this to Listen on specific IP addresses as shown below to

                # prevent Apache from glomming onto all bound IP addresses.

                #

                #Listen 12.34.56.78:80

                Maxcontext 10

                Maxnode 40

                Maxhost 40

                Listen 10.112.38.78:6666

                <VirtualHost 10.112.38.78:6666>

                <Location />

                Order deny,allow

                Deny from all

                Allow from all

                </Location>

                KeepAliveTimeout 60

                MaxKeepAliveRequests 0

                ManagerBalancerName mycluster

                ServerAdvertise off

                AdvertiseFrequency 5

                </VirtualHost>

                 

                 

                #

                # Dynamic Shared Object (DSO) Support

                #

                # To be able to use the functionality of a module which was built as a DSO you

                # have to place corresponding `LoadModule' lines at this location so the

                # directives contained in it are actually available _before_ they are used.

                # Statically compiled modules (those listed by `httpd -l') do not need

                # to be loaded here.

                #

                # Example:

                # LoadModule foo_module modules/mod_foo.so

                #

                LoadModule actions_module modules/mod_actions.so

                LoadModule alias_module modules/mod_alias.so

                LoadModule asis_module modules/mod_asis.so

                LoadModule auth_basic_module modules/mod_auth_basic.so

                #LoadModule auth_digest_module modules/mod_auth_digest.so

                #LoadModule authn_alias_module modules/mod_authn_alias.so

                #LoadModule authn_anon_module modules/mod_authn_anon.so

                #LoadModule authn_dbd_module modules/mod_authn_dbd.so

                #LoadModule authn_dbm_module modules/mod_authn_dbm.so

                LoadModule authn_default_module modules/mod_authn_default.so

                LoadModule authn_file_module modules/mod_authn_file.so

                #LoadModule authnz_ldap_module modules/mod_authnz_ldap.so

                #LoadModule authz_dbm_module modules/mod_authz_dbm.so

                LoadModule authz_default_module modules/mod_authz_default.so

                LoadModule authz_groupfile_module modules/mod_authz_groupfile.so

                LoadModule authz_host_module modules/mod_authz_host.so

                #LoadModule authz_owner_module modules/mod_authz_owner.so

                LoadModule authz_user_module modules/mod_authz_user.so

                LoadModule autoindex_module modules/mod_autoindex.so

                #LoadModule cache_module modules/mod_cache.so

                #LoadModule cern_meta_module modules/mod_cern_meta.so

                LoadModule cgi_module modules/mod_cgi.so

                #LoadModule charset_lite_module modules/mod_charset_lite.so

                #LoadModule dav_module modules/mod_dav.so

                #LoadModule dav_fs_module modules/mod_dav_fs.so

                #LoadModule dav_lock_module modules/mod_dav_lock.so

                #LoadModule dbd_module modules/mod_dbd.so

                #LoadModule deflate_module modules/mod_deflate.so

                LoadModule dir_module modules/mod_dir.so

                #LoadModule disk_cache_module modules/mod_disk_cache.so

                #LoadModule dumpio_module modules/mod_dumpio.so

                LoadModule env_module modules/mod_env.so

                #LoadModule expires_module modules/mod_expires.so

                #LoadModule ext_filter_module modules/mod_ext_filter.so

                #LoadModule file_cache_module modules/mod_file_cache.so

                #LoadModule filter_module modules/mod_filter.so

                #LoadModule headers_module modules/mod_headers.so

                #LoadModule ident_module modules/mod_ident.so

                #LoadModule imagemap_module modules/mod_imagemap.so

                LoadModule include_module modules/mod_include.so

                #LoadModule info_module modules/mod_info.so

                LoadModule isapi_module modules/mod_isapi.so

                #LoadModule ldap_module modules/mod_ldap.so

                #LoadModule logio_module modules/mod_logio.so

                LoadModule log_config_module modules/mod_log_config.so

                #LoadModule log_forensic_module modules/mod_log_forensic.so

                #LoadModule mem_cache_module modules/mod_mem_cache.so

                LoadModule mime_module modules/mod_mime.so

                #LoadModule mime_magic_module modules/mod_mime_magic.so

                LoadModule negotiation_module modules/mod_negotiation.so

                LoadModule proxy_module modules/mod_proxy.so

                LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

                #LoadModule proxy_balancer_module modules/mod_proxy_balancer.so

                #LoadModule proxy_connect_module modules/mod_proxy_connect.so

                #LoadModule proxy_ftp_module modules/mod_proxy_ftp.so

                #LoadModule proxy_http_module modules/mod_proxy_http.so

                #LoadModule proxy_scgi_module modules/mod_proxy_scgi.so

                #LoadModule reqtimeout_module modules/mod_reqtimeout.so

                #LoadModule rewrite_module modules/mod_rewrite.so

                LoadModule setenvif_module modules/mod_setenvif.so

                #LoadModule speling_module modules/mod_speling.so

                #LoadModule ssl_module modules/mod_ssl.so

                #LoadModule status_module modules/mod_status.so

                #LoadModule substitute_module modules/mod_substitute.so

                #LoadModule unique_id_module modules/mod_unique_id.so

                #LoadModule userdir_module modules/mod_userdir.so

                #LoadModule usertrack_module modules/mod_usertrack.so

                #LoadModule version_module modules/mod_version.so

                #LoadModule vhost_alias_module modules/mod_vhost_alias.so

                LoadModule slotmem_module modules/mod_slotmem.so

                LoadModule manager_module modules/mod_manager.so

                LoadModule proxy_cluster_module modules/mod_proxy_cluster.so

                LoadModule advertise_module modules/mod_advertise.so

                 

                 

                <Location /mod_cluster_manager>

                SetHandler mod_cluster-manager

                Order deny,allow

                Deny from all

                Allow from 10.112.38.78

                </Location>

                <IfModule !mpm_netware_module>

                <IfModule !mpm_winnt_module>

                #

                # If you wish httpd to run as a different user or group, you must run

                # httpd as root initially and it will switch. 

                #

                # User/Group: The name (or #number) of the user/group to run httpd as.

                # It is usually good practice to create a dedicated user and group for

                # running httpd, as with most system services.

                #

                User daemon

                Group daemon

                 

                 

                </IfModule>

                </IfModule>

                 

                 

                # 'Main' server configuration

                #

                # The directives in this section set up the values used by the 'main'

                # server, which responds to any requests that aren't handled by a

                # <VirtualHost> definition.  These values also provide defaults for

                # any <VirtualHost> containers you may define later in the file.

                #

                # All of these directives may appear inside <VirtualHost> containers,

                # in which case these default settings will be overridden for the

                # virtual host being defined.

                #

                 

                 

                #

                # ServerAdmin: Your address, where problems with the server should be

                # e-mailed.  This address appears on some server-generated pages, such

                # as error documents.  e.g. admin@your-domain.com

                #

                ServerAdmin admin@wipro.com

                 

                 

                #

                # ServerName gives the name and port that the server uses to identify itself.

                # This can often be determined automatically, but we recommend you specify

                # it explicitly to prevent problems during startup.

                #

                # If your host doesn't have a registered DNS name, enter its IP address here.

                #

                #ServerName L-156018275.wipro.com:80

                 

                 

                #

                # DocumentRoot: The directory out of which you will serve your

                # documents. By default, all requests are taken from this directory, but

                # symbolic links and aliases may be used to point to other locations.

                #

                DocumentRoot "C:/Users/Administrator/isova/backup/Apache2.2/htdocs"

                 

                 

                #

                # Each directory to which Apache has access can be configured with respect

                # to which services and features are allowed and/or disabled in that

                # directory (and its subdirectories).

                #

                # First, we configure the "default" to be a very restrictive set of

                # features. 

                #

                <Directory />

                    Options FollowSymLinks

                    AllowOverride None

                    Order deny,allow

                    Deny from all

                </Directory>

                 

                 

                #

                # Note that from this point forward you must specifically allow

                # particular features to be enabled - so if something's not working as

                # you might expect, make sure that you have specifically enabled it

                # below.

                #

                 

                 

                #

                # This should be changed to whatever you set DocumentRoot to.

                #

                <Directory "C:/Users/Administrator/isova/backup/Apache2.2/htdocs">

                    #

                    # Possible values for the Options directive are "None", "All",

                    # or any combination of:

                    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews

                    #

                    # Note that "MultiViews" must be named *explicitly* --- "Options All"

                    # doesn't give it to you.

                    #

                    # The Options directive is both complicated and important.  Please see

                    # http://httpd.apache.org/docs/2.2/mod/core.html#options

                    # for more information.

                    #

                    Options Indexes FollowSymLinks

                 

                 

                    #

                    # AllowOverride controls what directives may be placed in .htaccess files.

                    # It can be "All", "None", or any combination of the keywords:

                    #   Options FileInfo AuthConfig Limit

                    #

                    AllowOverride None

                 

                 

                    #

                    # Controls who can get stuff from this server.

                    #

                    Order allow,deny

                    Allow from all

                 

                 

                </Directory>

                 

                 

                #

                # DirectoryIndex: sets the file that Apache will serve if a directory

                # is requested.

                #

                <IfModule dir_module>

                    DirectoryIndex index.html

                </IfModule>

                 

                 

                #

                # The following lines prevent .htaccess and .htpasswd files from being

                # viewed by Web clients.

                #

                <FilesMatch "^\.ht">

                    Order allow,deny

                    Deny from all

                    Satisfy All

                </FilesMatch>

                 

                 

                #

                # ErrorLog: The location of the error log file.

                # If you do not specify an ErrorLog directive within a <VirtualHost>

                # container, error messages relating to that virtual host will be

                # logged here.  If you *do* define an error logfile for a <VirtualHost>

                # container, that host's errors will be logged there and not here.

                #

                ErrorLog "logs/error.log"

                 

                 

                #

                # LogLevel: Control the number of messages logged to the error_log.

                # Possible values include: debug, info, notice, warn, error, crit,

                # alert, emerg.

                #

                LogLevel warn

                 

                 

                <IfModule log_config_module>

                    #

                    # The following directives define some format nicknames for use with

                    # a CustomLog directive (see below).

                    #

                    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

                    LogFormat "%h %l %u %t \"%r\" %>s %b" common

                 

                 

                    <IfModule logio_module>

                      # You need to enable mod_logio.c to use %I and %O

                      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio

                    </IfModule>

                 

                 

                    #

                    # The location and format of the access logfile (Common Logfile Format).

                    # If you do not define any access logfiles within a <VirtualHost>

                    # container, they will be logged here.  Contrariwise, if you *do*

                    # define per-<VirtualHost> access logfiles, transactions will be

                    # logged therein and *not* in this file.

                    #

                    CustomLog "logs/access.log" common

                 

                 

                    #

                    # If you prefer a logfile with access, agent, and referer information

                    # (Combined Logfile Format) you can use the following directive.

                    #

                    #CustomLog "logs/access.log" combined

                </IfModule>

                 

                 

                <IfModule alias_module>

                    #

                    # Redirect: Allows you to tell clients about documents that used to

                    # exist in your server's namespace, but do not anymore. The client

                    # will make a new request for the document at its new location.

                    # Example:

                    # Redirect permanent /foo http://L-156018275.wipro.com/bar

                 

                 

                    #

                    # Alias: Maps web paths into filesystem paths and is used to

                    # access content that does not live under the DocumentRoot.

                    # Example:

                    # Alias /webpath /full/filesystem/path

                    #

                    # If you include a trailing / on /webpath then the server will

                    # require it to be present in the URL.  You will also likely

                    # need to provide a <Directory> section to allow access to

                    # the filesystem path.

                 

                 

                    #

                    # ScriptAlias: This controls which directories contain server scripts.

                    # ScriptAliases are essentially the same as Aliases, except that

                    # documents in the target directory are treated as applications and

                    # run by the server when requested rather than as documents sent to the

                    # client.  The same rules about trailing "/" apply to ScriptAlias

                    # directives as to Alias.

                    #

                    ScriptAlias /cgi-bin/ "C:/Users/Administrator/isova/backup/Apache2.2/cgi-bin/"

                 

                 

                </IfModule>

                 

                 

                <IfModule cgid_module>

                    #

                    # ScriptSock: On threaded servers, designate the path to the UNIX

                    # socket used to communicate with the CGI daemon of mod_cgid.

                    #

                    #Scriptsock logs/cgisock

                </IfModule>

                 

                 

                #

                # "C:/Users/Administrator/isova/backup/Apache2.2/cgi-bin" should be changed to whatever your ScriptAliased

                # CGI directory exists, if you have that configured.

                #

                <Directory "C:/Users/Administrator/isova/backup/Apache2.2/cgi-bin">

                    AllowOverride None

                    Options None

                    Order allow,deny

                    Allow from all

                </Directory>

                 

                 

                #

                # DefaultType: the default MIME type the server will use for a document

                # if it cannot otherwise determine one, such as from filename extensions.

                # If your server contains mostly text or HTML documents, "text/plain" is

                # a good value.  If most of your content is binary, such as applications

                # or images, you may want to use "application/octet-stream" instead to

                # keep browsers from trying to display binary files as though they are

                # text.

                #

                DefaultType text/plain

                 

                 

                <IfModule mime_module>

                    #

                    # TypesConfig points to the file containing the list of mappings from

                    # filename extension to MIME-type.

                    #

                    TypesConfig conf/mime.types

                 

                 

                    #

                    # AddType allows you to add to or override the MIME configuration

                    # file specified in TypesConfig for specific file types.

                    #

                    #AddType application/x-gzip .tgz

                    #

                    # AddEncoding allows you to have certain browsers uncompress

                    # information on the fly. Note: Not all browsers support this.

                    #

                    #AddEncoding x-compress .Z

                    #AddEncoding x-gzip .gz .tgz

                    #

                    # If the AddEncoding directives above are commented-out, then you

                    # probably should define those extensions to indicate media types:

                    #

                    AddType application/x-compress .Z

                    AddType application/x-gzip .gz .tgz

                 

                 

                    #

                    # AddHandler allows you to map certain file extensions to "handlers":

                    # actions unrelated to filetype. These can be either built into the server

                    # or added with the Action directive (see below)

                    #

                    # To use CGI scripts outside of ScriptAliased directories:

                    # (You will also need to add "ExecCGI" to the "Options" directive.)

                    #

                    #AddHandler cgi-script .cgi

                 

                 

                    # For type maps (negotiated resources):

                    #AddHandler type-map var

                 

                 

                    #

                    # Filters allow you to process content before it is sent to the client.

                    #

                    # To parse .shtml files for server-side includes (SSI):

                    # (You will also need to add "Includes" to the "Options" directive.)

                    #

                    #AddType text/html .shtml

                    #AddOutputFilter INCLUDES .shtml

                </IfModule>

                 

                 

                #

                # The mod_mime_magic module allows the server to use various hints from the

                # contents of the file itself to determine its type.  The MIMEMagicFile

                # directive tells the module where the hint definitions are located.

                #

                #MIMEMagicFile conf/magic

                 

                 

                #

                # Customizable error responses come in three flavors:

                # 1) plain text 2) local redirects 3) external redirects

                #

                # Some examples:

                #ErrorDocument 500 "The server made a boo boo."

                #ErrorDocument 404 /missing.html

                #ErrorDocument 404 "/cgi-bin/missing_handler.pl"

                #ErrorDocument 402 http://L-156018275.wipro.com/subscription_info.html

                #

                 

                 

                #

                # MaxRanges: Maximum number of Ranges in a request before

                # returning the entire resource, or one of the special

                # values 'default', 'none' or 'unlimited'.

                # Default setting is to accept 200 Ranges.

                #MaxRanges unlimited

                 

                 

                #

                # EnableMMAP and EnableSendfile: On systems that support it,

                # memory-mapping or the sendfile syscall is used to deliver

                # files.  This usually improves server performance, but must

                # be turned off when serving from networked-mounted

                # filesystems or if support for these functions is otherwise

                # broken on your system.

                #

                #EnableMMAP off

                #EnableSendfile off

                 

                 

                # Supplemental configuration

                #

                # The configuration files in the conf/extra/ directory can be

                # included to add extra features or to modify the default configuration of

                # the server, or you may simply copy their contents here and change as

                # necessary.

                 

                 

                # Server-pool management (MPM specific)

                #Include conf/extra/httpd-mpm.conf

                 

                 

                # Multi-language error messages

                #Include conf/extra/httpd-multilang-errordoc.conf

                 

                 

                # Fancy directory listings

                #Include conf/extra/httpd-autoindex.conf

                 

                 

                # Language settings

                #Include conf/extra/httpd-languages.conf

                 

                 

                # User home directories

                #Include conf/extra/httpd-userdir.conf

                 

                 

                # Real-time info on requests and configuration

                #Include conf/extra/httpd-info.conf

                 

                 

                # Virtual hosts

                #Include conf/extra/httpd-vhosts.conf

                 

                 

                # Local access to the Apache HTTP Server Manual

                #Include conf/extra/httpd-manual.conf

                 

                 

                # Distributed authoring and versioning (WebDAV)

                #Include conf/extra/httpd-dav.conf

                 

                 

                # Various default settings

                #Include conf/extra/httpd-default.conf

                 

                 

                # Secure (SSL/TLS) connections

                #Include conf/extra/httpd-ssl.conf

                #

                # Note: The following must must be present to support

                #       starting without SSL on platforms with no /dev/random equivalent

                #       but a statically compiled-in mod_ssl.

                #

                <IfModule ssl_module>

                SSLRandomSeed startup builtin

                SSLRandomSeed connect builtin

                </IfModule>

                 

                regards

                Guru

                • 5. Re: mod cluster performance
                  Gnanaguru Sattanathan Novice

                  Hi Jean,

                   

                  I tested with multiple scenarios with high loads. I couldnt find any difference.

                   

                  Is there any provision for changing the load balancing algorithm ?

                   

                  Can you suggest any other approach ?

                   

                  Regards

                  Guru

                  • 6. Re: mod cluster performance
                    Jean-Frederic Clere Master

                    You should read http://httpd.apache.org/docs/2.2/misc/perf-tuning.html

                    check is you use worker:

                    httpd -V

                    it should have "Server MPM: Worker" in output.

                    then check http://httpd.apache.org/docs/2.2/mod/worker.html otherwise recompile httpd (or install a worker version) to use worker.

                     

                    • 7. Re: mod cluster performance
                      Gnanaguru Sattanathan Novice

                      Great Jean,

                       

                      I hope I am redirected to a right approach. Am going throught the docs. By the way, I did httpd-V, this is what is shown...

                       

                      Server MPM: WinNT

                       

                      I could see the mpm-worker-module present in httpd-mpm.conf, Do you have any idea where I can configure to choose mpm as 'worker' instead of 'winnt'

                       

                      Regards

                      Guru

                       


                      • 9. Re: mod cluster performance
                        Radoslav Husar Master

                        I have a camel application running on 6 independent JBoss AS7 nodes. Camel application exposes a http endpoint which can receive requests and the response will be sent to a JMS queue.

                        That seems like a wrong test scenario if you want to benchmark a load-balancer since the JMS queue might be the bottleneck and no the load-balancer itself. You need to make sure that what you are testing does not imply more work to the cluster if the nodes are added. Make sure you are testing really just the LB, by accessing a static page for example or any other page that is not affected by the cluster.

                         

                        E.g. it would be the same like testing session replication using replicated (Infinispan) cache. With each extra node you would also increase the redundancy so in the end you would be comparing apples to oranges.

                        1 of 1 people found this helpful
                        • 10. Re: mod cluster performance
                          Gnanaguru Sattanathan Novice

                          I have one more doubt. When I connect multiple JBoss AS 7 instances with mod_cluster, do I need to cluster the JBoss AS7 instances also ? ( using standalone-ha.xml).

                           

                          I am basically using this setup for middleware use cases, I dont really use it for web applications or sessions.

                          • 11. Re: mod cluster performance
                            Jean-Frederic Clere Master

                            To use mod_cluster you don't need to cluster the JBoss AS7 instances.