HTTPD using a lot of CPU

[nichele]

Hi all,

i was experiencing strange behaviour using tomcat 6.0.13 + mod-cluster 1.1.3 + apache 2.2.21.

I have 1 apache and 3 tomcat instances.

Using this configuration tomcat side:

        advertise="false"

        maxAttempts="3"

        nodeTimeout="600"

        workerTimeout="900"

        ping="60"

        stickySession="true"

        stickySessionRemove="true"

        stickySessionForce="false"

I saw that stopping one on the tomcat instances, httpd starts asap to have a very high CPU usage. Also restarting tomcat the CPU usage stays high.

Strange thing...i found that setting "stickySessionForce" to true, this behaviour doesn't happen.

I continued my investigation and I fixed the issue compiling mod_manager and mod_proxy_cluster on the host with httpd (not sure if the issue was in the mod_manager or in mod_proxy_cluster but if needed i can try).

So now i'm pretty confused :-) since using .so modules from this package http://downloads.jboss.org/mod_cluster//1.1.3.Final/mod_cluster-1.1.3.Final-linux2-x64-so.tar.gz i have the issue aforementioned, but compiling mod-cluster source i don't have it.

For completeness:

[root]# httpd -V

Server version: Apache/2.2.21 (Unix)

Server built:   Jan 12 2012 15:28:45

Server's Module Magic Number: 20051115:30

Server loaded:  APR 1.2.7, APR-Util 1.2.7

Compiled using: APR 1.2.7, APR-Util 1.2.7

Architecture:   64-bit

Server MPM:     Worker

  threaded:     yes (fixed thread count)

    forked:     yes (variable process count)

Server compiled with....

-D APACHE_MPM_DIR="server/mpm/worker"

-D APR_HAS_SENDFILE

-D APR_HAS_MMAP

-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)

-D APR_USE_SYSVSEM_SERIALIZE

-D APR_USE_PTHREAD_SERIALIZE

-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT

-D APR_HAS_OTHER_CHILD

-D AP_HAVE_RELIABLE_PIPED_LOGS

-D DYNAMIC_MODULE_LIMIT=128

-D HTTPD_ROOT="/usr/local/httpd-2.2.21"

-D SUEXEC_BIN="/usr/local/httpd-2.2.21/bin/suexec"

-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"

-D DEFAULT_ERRORLOG="logs/error_log"

-D AP_TYPES_CONFIG_FILE="conf/mime.types"

-D SERVER_CONFIG_FILE="conf/httpd.conf"

To note that from mod_cluster i'm using just these .so:

- mod_advertise.so

- mod_manager.so

- mod_proxy_cluster.so

- mod_slotmem.so

About mod_proxy, mod_proxy_ajp, mod_proxy_http i use tha ones coming from httpd

since otherwise i have issues (stickysession doesn't work).

At this point i don't know if i'm installing mod-cluster in the right way or if i'm missing something.

Could you help me in understanding what is happening and where i'm doing wrong ?

Thanks

ste

[nichele]

Hi all,

i compared the .so files genearted by compiling the code with the one released in the official 1.1.3 package using nm (nm - list symbols from object files) and i found something strange. For instance i saw that in the compiled version there is this symbol:

0000000000002c19 t add_balancer_node

that is missing in the released version (but it's not the only one).

I'm attaching both versions of the so file hoping someone can help me in shedding light on this issue.

thanks in advance

Ste

[nichele]

Hi all,

i found that the differences in the compiled version are caused by the 'O2' option of the gcc optimazer. This explain why in my version i have the symbol 'add_balancer_node' that in yours is missing (you are using O2, i don't). So basically my ignorance :-).

But the original issue is still there. Compiling mod_proxy_cluster with 'O2' option the .so file doens't work as expected with the mentioned configuration  (HTTPD goes crazy in case of tomcat failure).

Compilining the code without 'O2' option, all works fine.

As said i'm not an expert of gcc/native code, but since also your .so file doesn't work as expected (it is compiled with O2) i think that the issue should be taken in consideration by you.

Ideas?

thanks

ste