6 Replies Latest reply on Jul 11, 2013 1:22 AM by jfclere

    Error [MEM: MEM: Can't read node: {4}]

    fanreds
    fanreds May 8, 2013 11:07 AM

      Welcome

      I have problem with mod_cluster. I use jboss 7.1.1 in domain mode and mod_cluster 1.2.0.

       

      I get this error:

      [Server:server-three] 16:36:07,288 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy mint/192.168.1.17:10001, configuration will be reset

      [Server:server-one] 16:36:11,380 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy mint/192.168.1.17:10001, configuration will be reset

      [Server:server-two] 16:36:14,118 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy mint/192.168.1.17:10001, configuration will be reset

       

       

      In domain.xml - profile full and full-ha contain:

                  <subsystem xmlns="urn:jboss:domain:web:1.1" default-virtual-server="default-host" instance-id="${jboss.node.name}" native="false">

                      <connector name="http" protocol="HTTP/1.1" scheme="http" socket-binding="http"/>

                      <connector name="ajp" protocol="AJP/1.3" scheme="http" socket-binding="ajp"/>

                      <virtual-server name="default-host" enable-welcome-root="true">

                          <alias name="localhost"/>

                          <alias name="example.com"/>

                      </virtual-server>

                  </subsystem>

       

      <subsystem xmlns="urn:jboss:domain:modcluster:1.0">

                      <mod-cluster-config advertise-socket="modcluster" proxy-list="192.168.1.17:10001">

                          <dynamic-load-provider>

                              <load-metric type="busyness"/>

                          </dynamic-load-provider>

                      </mod-cluster-config>

                  </subsystem>

       

       

      When i go to 192.168.1.17:10001/mod_cluster-manager i get The requested URL /mod_cluster-manager was not found on this server.

       

      mods-enabled/mod_cluster.conf

      CreateBalancers 1

        <IfModule manager_module>

          Listen 192.168.1.17:10001

          ManagerBalancerName mycluster

       

          <VirtualHost 192.168.1.17:10001>

              KeepAliveTimeout 300

              MaxKeepAliveRequests 0

              AdvertiseFrequency 5

              ServerAdvertise On

       

                EnableMCPMReceive

              <Location />

                  Order deny,allow

                  Allow from 192.168.1.

              </Location>

       

           </VirtualHost>

       

      </IfModule>

       

      mods-enabled/mod_cluster.load

      LoadModule proxy_module /usr/lib/apache2/modules/mod_proxy.so

      LoadModule proxy_http_module /usr/lib/apache2/modules/mod_proxy_http.so

      LoadModule proxy_ajp_module /usr/lib/apache2/modules/mod_proxy_ajp.so

      LoadModule slotmem_module /usr/lib/apache2/modules/mod_slotmem.so

      LoadModule manager_module /usr/lib/apache2/modules/mod_manager.so

      LoadModule proxy_cluster_module /usr/lib/apache2/modules/mod_proxy_cluster.so

      LoadModule advertise_module /usr/lib/apache2/modules/mod_advertise.so

       

      sites-enabled/jboss

      NameVirtualHost *:80

      <VirtualHost *:80>

          ServerName mint

          ServerAlias mint

       

          ProxyPass / balancer://mycluster stickysession=JSESSIONID|jsessionid nofailover=On

          ProxyPassReverse / balancer://mycluster

          ProxyPreserveHost On

       

          <Location />

              Order deny,allow

              Allow from All

          </Location>

       

         <Location /mod_cluster-manager>

            SetHandler mod_cluster-manager

            Order deny,allow

            #Deny from all

            Allow from 192.168.1

         </Location>

       

      </VirtualHost>

       


       

      What could be wrong?

        • 1. Re: Error [MEM: MEM: Can't read node: {4}]
          rhusar
          rhusar May 9, 2013 10:40 AM (in response to fanreds)

          Could be anything really.

           

          Lets look at the loadbalancer log: turn it on by (IIRC) LogLevel debug and see what will it say.

            • Actions
          • 2. Re: Error [MEM: MEM: Can't read node: {4}]
            fanreds
            fanreds May 9, 2013 12:37 PM (in response to rhusar)

            My log:

             

            [Thu May 09 18:29:10 2013] [notice] Advertise initialized for process 21207

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21212 for worker proxy:reverse

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21212 for (*)

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21215 for worker proxy:reverse

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21215 for (*)

            [Thu May 09 18:29:10 2013] [notice] Apache/2.2.22 (Ubuntu) mod_cluster/1.2.0.Final PHP/5.4.6-1ubuntu1.2 configured -- resuming normal operations

            [Thu May 09 18:29:10 2013] [info] Server built: Mar  8 2013 15:53:14

            [Thu May 09 18:29:10 2013] [debug] prefork.c(1023): AcceptMutex: sysvsem (default: sysvsem)

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21216 for worker proxy:reverse

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21216 for (*)

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21213 for worker proxy:reverse

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21213 for (*)

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21218 for worker proxy:reverse

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:10 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21218 for (*)

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:11 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:14 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:14 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:29:14 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:14 2013] [warn] manager_handler STATUS error: MEM: Can't read node

            [Thu May 09 18:29:15 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:15 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:29:15 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:15 2013] [warn] manager_handler STATUS error: MEM: Can't read node

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(1667): manager_trans INFO (/)

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2323): manager_handler INFO (/) processing: ""

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2366): manager_handler INFO  OK

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(1667): manager_trans CONFIG (/)

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2323): manager_handler CONFIG (/) processing: "JVMRoute=master%3Aserver-three&Host=mint&Maxattempts=1&Port=8259&StickySessionForce=No&Type=ajp&ping=10"

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2366): manager_handler CONFIG  OK

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(1667): manager_trans ENABLE-APP (/)

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2323): manager_handler ENABLE-APP (/) processing: "JVMRoute=master%3Aserver-three&Alias=default-host%2Clocalhost%2Cexample.com&Context=%2Facademy"

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2366): manager_handler ENABLE-APP  OK

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:19 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:19 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21213 for (mint)

            [Thu May 09 18:29:19 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:19 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:19 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:19 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:19 2013] [debug] proxy_util.c(2444): proxy: ajp: fam 2 socket created to connect to mint

            [Thu May 09 18:29:19 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:19 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:19 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21215 for (mint)

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21216 for (mint)

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21218 for (mint)

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21212 for (mint)

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:20 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:20 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21227 for worker proxy:reverse

            [Thu May 09 18:29:20 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:20 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21227 for (*)

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21227 for (mint)

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:21 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(1667): manager_trans INFO (/)

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(2323): manager_handler INFO (/) processing: ""

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(2366): manager_handler INFO  OK

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(1667): manager_trans CONFIG (/)

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(2323): manager_handler CONFIG (/) processing: "JVMRoute=master%3Aserver-one&Host=mint&Maxattempts=1&Port=8009&StickySessionForce=No&Type=ajp&ping=10"

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(2366): manager_handler CONFIG  OK

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:24 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:24 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21212 for (mint)

            [Thu May 09 18:29:24 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:24 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:24 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:24 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:24 2013] [debug] proxy_util.c(2444): proxy: ajp: fam 2 socket created to connect to mint

            [Thu May 09 18:29:24 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:24 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:24 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(1667): manager_trans INFO (/)

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(2323): manager_handler INFO (/) processing: ""

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(2366): manager_handler INFO  OK

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(1667): manager_trans CONFIG (/)

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(2323): manager_handler CONFIG (/) processing: "JVMRoute=master%3Aserver-two&Host=mint&Maxattempts=1&Port=8159&StickySessionForce=No&Type=ajp&ping=10"

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(2366): manager_handler CONFIG  OK

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21216 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(2444): proxy: ajp: fam 2 socket created to connect to mint

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21215 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21215 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21212 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21213 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21213 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21216 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21218 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21218 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21227 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21227 for (mint)

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:25 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21232 for worker proxy:reverse

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:25 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21232 for (*)

            [Thu May 09 18:29:26 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21234 for worker proxy:reverse

            [Thu May 09 18:29:26 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:26 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21234 for (*)

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21232 for (mint)

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21232 for (mint)

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:26 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21235 for worker proxy:reverse

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:26 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21232 for (mint)

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:26 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21235 for (*)

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:26 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21234 for (mint)

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21235 for (mint)

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21234 for (mint)

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21235 for (mint)

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21234 for (mint)

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21235 for (mint)

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:29:27 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:29:29 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:29 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:29:29 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:29 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:29 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:29 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:29 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:29 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:29 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:34 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:34 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:29:34 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:34 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:34 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:34 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:34 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:34 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:34 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:35 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:35 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:29:35 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:35 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:35 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:35 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:35 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:35 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:35 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:39 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:39 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:29:39 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:39 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:39 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:39 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:39 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:39 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:39 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:44 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:44 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:29:44 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:44 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:44 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:44 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:44 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:44 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:44 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:45 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:45 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:29:45 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:45 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:45 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:45 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:45 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:45 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:45 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:49 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:49 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:29:49 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:49 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:49 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:49 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:49 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:49 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:49 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:54 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:54 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:29:54 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:54 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:54 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:54 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8009/ to mint:8009

            [Thu May 09 18:29:54 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:54 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:54 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:55 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:55 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:29:55 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:55 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:55 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:55 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8159/ to mint:8159

            [Thu May 09 18:29:55 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:55 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:55 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:29:59 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:29:59 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:29:59 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:29:59 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:29:59 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:59 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:29:59 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:29:59 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:29:59 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:30:04 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:30:04 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:30:04 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:30:04 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:30:04 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8009/ to mint:8009

            [Thu May 09 18:30:04 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8009/ to mint:8009

            [Thu May 09 18:30:04 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:30:04 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:30:04 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:30:05 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:30:05 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:30:05 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:30:05 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:30:05 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8159/ to mint:8159

            [Thu May 09 18:30:05 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8159/ to mint:8159

            [Thu May 09 18:30:05 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:30:05 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:30:05 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:30:09 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:30:09 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:30:09 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:30:09 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:30:09 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:30:09 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:30:09 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:30:09 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:30:09 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:30:14 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:30:14 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-one&Load=100"

            [Thu May 09 18:30:14 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:30:14 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:30:14 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8009/ to mint:8009

            [Thu May 09 18:30:14 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8009/ to mint:8009

            [Thu May 09 18:30:14 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:30:14 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:30:14 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:30:15 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:30:15 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-two&Load=100"

            [Thu May 09 18:30:15 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:30:15 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:30:15 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8159/ to mint:8159

            [Thu May 09 18:30:15 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8159/ to mint:8159

            [Thu May 09 18:30:15 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:30:15 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:30:15 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

            [Thu May 09 18:30:17 2013] [error] [client 192.168.1.17] File does not exist: /var/www/mod_cluster-manager

            [Thu May 09 18:30:17 2013] [debug] mod_deflate.c(615): [client 192.168.1.17] Zlib: Compressed 298 to 229 : URL /mod_cluster-manager

            [Thu May 09 18:30:18 2013] [debug] proxy_util.c(1818): proxy: grabbed scoreboard slot 0 in child 21419 for worker proxy:reverse

            [Thu May 09 18:30:18 2013] [debug] proxy_util.c(1837): proxy: worker proxy:reverse already initialized

            [Thu May 09 18:30:18 2013] [debug] proxy_util.c(1934): proxy: initialized single connection worker 0 in child 21419 for (*)

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8259

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 1 in child 21419 for (mint)

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8259 1 (status): 1

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8009

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 2 in child 21419 for (mint)

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8009 2 (status): 1

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(254): Created: worker for ajp://mint:8159

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(377): proxy: initialized single connection worker 3 in child 21419 for (mint)

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(418): Created: worker for ajp://mint:8159 3 (status): 1

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(678): update_workers_node starting

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(693): update_workers_node done

            [Thu May 09 18:30:19 2013] [debug] mod_manager.c(1667): manager_trans STATUS (/)

            [Thu May 09 18:30:19 2013] [debug] mod_manager.c(2323): manager_handler STATUS (/) processing: "JVMRoute=master%3Aserver-three&Load=100"

            [Thu May 09 18:30:19 2013] [debug] mod_manager.c(1404): Processing STATUS

            [Thu May 09 18:30:19 2013] [debug] proxy_util.c(2011): proxy: ajp: has acquired connection for (mint)

            [Thu May 09 18:30:19 2013] [debug] proxy_util.c(2067): proxy: connecting ajp://mint:8259/ to mint:8259

            [Thu May 09 18:30:19 2013] [debug] proxy_util.c(2193): proxy: connected ajp://mint:8259/ to mint:8259

            [Thu May 09 18:30:19 2013] [debug] mod_proxy_cluster.c(773): ajp_cping_cpong: Done

            [Thu May 09 18:30:19 2013] [debug] proxy_util.c(2029): proxy: ajp: has released connection for (mint)

            [Thu May 09 18:30:19 2013] [debug] mod_manager.c(2366): manager_handler STATUS  OK

              • Actions
            • 3. Re: Error [MEM: MEM: Can't read node: {4}]
              mustaq.pradhan
              mustaq.pradhan Jul 8, 2013 8:40 PM (in response to fanreds)

              I am also experiencing the issue.

               

              ERROR [org.jboss.modcluster] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) MODCLUSTER000042: Error MEM sending STATUS command to domain.controller.host.name/domain.controller.host.ip:10001, configuration will be reset: MEM: Can't read node

              ERROR [org.jboss.modcluster] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) MODCLUSTER000042: Error MEM sending STATUS command to host1.controller.host.name/host1.controller.host.ip:10001, configuration will be reset: MEM: Can't read node

              ERRPR [org.jboss.modcluster] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) MODCLUSTER000042: Error MEM sending STATUS command to host2.controller.host.name/host2.controller.host.ip:10001, configuration will be reset: MEM: Can't read node

              ERROR [org.jboss.modcluster] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) MODCLUSTER000042: Error MEM sending STATUS command to host3.controller.host.name/host3.controller.host.ip:10001, configuration will be reset: MEM: Can't read node

               

               

              My mod_cluster.conf is:

               

               

              LoadModule slotmem_module       modules/mod_slotmem.so

              LoadModule proxy_cluster_module modules/mod_proxy_cluster.so

              LoadModule advertise_module     modules/mod_advertise.so

              LoadModule manager_module       modules/mod_manager.so

               

               

              <Location /mod_cluster_manager>

                  SetHandler mod_cluster-manager

              </Location>

               

               

              Listen ud-j10-00.appu.dev.det.nsw.edu.au:10001

              <VirtualHost ud-j10-00.appu.dev.det.nsw.edu.au:10001>

                 KeepAliveTimeout 60

                 MaxKeepAliveRequests 0

                 ManagerBalancerName mycluster

               

               

                 ServerAdvertise On

                 AdvertiseFrequency 5

                 AdvertiseGroup 228.1.10.1:23364

               

               

                 EnableMCPMReceive

               

               

              </VirtualHost>

               

               

              Maxhost 1000

              Maxnode 100

               

               

              I have increased Maxhost and Maxnode, but still getting these errors.
              Please help.

                • Actions
              • 4. Re: Error [MEM: MEM: Can't read node: {4}]
                jfclere
                jfclere Jul 10, 2013 1:46 AM (in response to mustaq.pradhan)

                what are the error messages in error_log of htttpd?

                Note that like in the error_log of the previous case there might a few STATUS causing errors when httpd is restarted, is that the case?

                  • Actions
                • 5. Re: Error [MEM: MEM: Can't read node: {4}]
                  mustaq.pradhan
                  mustaq.pradhan Jul 10, 2013 2:51 AM (in response to jfclere)

                  We get STATUS warning messages on error_log:

                  [warn] manager_handler STATUS error: MEM: Can't read node

                   

                  But they do not correspond to the errors we are getting to the server.log, where this is happening more frequently.

                    • Actions
                  • 6. Re: Error [MEM: MEM: Can't read node: {4}]
                    jfclere
                    jfclere Jul 11, 2013 1:22 AM (in response to mustaq.pradhan)

                    The frequency is the same the message is the answer from httpd to the STATUS command.

                    As it is a warning probably mod_cluster works correctly, is that the case?

                      • Actions