1 Reply Latest reply on Mar 27, 2014 5:21 PM by vikrammurugesan

    mod_cluster - Error [MEM: MEM: Can't read node: {4}]

    vikrammurugesan
    vikrammurugesan Oct 15, 2013 4:27 PM

      Hello,

       

      I have a 2 node standalone cluster that I am trying to configure with mod_cluster. I read this page before starting - https://docs.jboss.org/author/display/AS72/AS7+Cluster+Howto

       

      I grabbed the mod_cluster-1.2.0.Final-windows-x86-ssl.zip file from Mod_cluster 1.2.0.Final - JBoss Community  and grabbed the 4 files mod_advertise.so, mod_manager.so, mod_proxy_cluster.so, mod_slotmem.so and dropped it into my already existing Apache HTTP server's modules directory.

       

      I made sure I commented out the line that loads mod_proxy module. I appended the below lines to my httpd.conf file, (xxx.xxx.xxx.xx is the IP of my machine and -b property of both my server startups)

       

      ########### mod_cluster to JBossAS7 ###########
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule manager_module modules/mod_manager.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule advertise_module modules/mod_advertise.so

Listen xxx.xxx.xxx.xx:3001

<VirtualHost xxx.xxx.xxx.xx:3001>
  <Directory />
       Order deny,allow
       Deny from all
       Allow from all
  </Directory>

  <Location /mod_cluster-manager>
       SetHandler mod_cluster-manager
       Order deny,allow
       Deny from all
       Allow from all
  </Location>

  KeepAliveTimeout 60
  MaxKeepAliveRequests 0
  AdvertiseFrequency 5
  AdvertiseSecurityKey secret
  EnableMCPMReceive
</VirtualHost>
######### End mod_cluster to JBossAS7 #########

       

      I modified my standalone-full-ha.xml in three places,

       

      I added the node name to my server, (on the second node the name is 'nodeB')

      <server name="nodeA" xmlns="urn:jboss:domain:1.3">

       

      I added instance-id to the web subsystem, (I also pass jboss.node.name as system property with the same value 'nodeA' and 'nodeB')

      <subsystem xmlns="urn:jboss:domain:web:1.2" default-virtual-server="default-host" instance-id="${jboss.node.name}" native="false">

       

      I added adverties=on, proxy-list and advertise-security-key to modcluster subsystem, (where xxx.xxx.xxx.xx is the same IP as in my httpd.conf file. It is also my -b bind address)

      <mod-cluster-config advertise-socket="modcluster" connector="ajp" advertise="true" proxy-list="xxx.xxx.xxx.xx:3001" advertise-security-key="secret">

       

      I bounced Apache server and it started up clean.

       

      After I start my first jboss node 'nodeA', I see this repeating in my logs every 10 seconds,

       

      15:03:19,204 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy my.machine.com/xxx.xxx.xxx.xx:3001, configuration will be reset
15:03:29,419 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy my.machine.com/xxx.xxx.xxx.xx:3001, configuration will be reset
15:03:39,632 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy my.machine.com/xxx.xxx.xxx.xx:3001, configuration will be reset
15:03:49,846 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy my.machine.com/xxx.xxx.xxx.xx:3001, configuration will be reset
15:04:00,060 ERROR [org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler] Error [MEM: MEM: Can't read node: {4}] sending command STATUS to proxy my.machine.com/xxx.xxx.xxx.xx:3001, configuration will be reset

       

      Also here is the apache error log, (LogLevel was debug) Sorry fot the big log file. I can attach it

       

      [Tue Oct 15 15:07:53 2013] [notice] Advertise initialized for process 14544
[Tue Oct 15 15:07:53 2013] [notice] Apache/2.2.22 (Win32) mod_cluster/1.2.0.Final configured -- resuming normal operations
[Tue Oct 15 15:07:53 2013] [notice] Server built: Jan 28 2012 11:16:39
[Tue Oct 15 15:07:53 2013] [notice] Parent: Created child process 14184
[Tue Oct 15 15:07:53 2013] [debug] mpm_winnt.c(477): Parent: Sent the scoreboard to the child
[Tue Oct 15 15:07:53 2013] [debug] mod_advertise.c(577): [14184 - 14544] in child post config hook
[Tue Oct 15 15:07:53 2013] [notice] Child 14184: Child process is running
[Tue Oct 15 15:07:53 2013] [info] Parent: Duplicating socket 280 and sending it to child process 14184
[Tue Oct 15 15:07:53 2013] [debug] mpm_winnt.c(398): Child 14184: Retrieved our scoreboard from the parent.
[Tue Oct 15 15:07:53 2013] [info] Parent: Duplicating socket 272 and sending it to child process 14184
[Tue Oct 15 15:07:53 2013] [debug] mpm_winnt.c(595): Parent: Sent 2 listeners to child 14184
[Tue Oct 15 15:07:53 2013] [debug] mpm_winnt.c(554): Child 14184: retrieved 2 listeners from parent
[Tue Oct 15 15:07:53 2013] [notice] Child 14184: Acquired the start mutex.
[Tue Oct 15 15:07:53 2013] [notice] Child 14184: Starting 64 worker threads.
[Tue Oct 15 15:07:53 2013] [notice] Child 14184: Starting thread to listen on port 3001.
[Tue Oct 15 15:07:53 2013] [notice] Child 14184: Starting thread to listen on port 80.
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(1667): manager_trans INFO 
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(2323): manager_handler INFO (C:/) processing: ""
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(2366): manager_handler INFO  OK
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(1667): manager_trans CONFIG 
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(2323): manager_handler CONFIG (C:/) processing: "JVMRoute=nodeA&Host=localhost&Maxattempts=1&Port=8009&StickySessionForce=No&Type=ajp&ping=10"
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(2366): manager_handler CONFIG  OK
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(1667): manager_trans STATUS 
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(2323): manager_handler STATUS (C:/) processing: "JVMRoute=nodeA&Load=100"
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(1404): Processing STATUS
[Tue Oct 15 15:08:20 2013] [notice] Parent: child process exited with status 3221225477 -- Restarting.
[Tue Oct 15 15:08:20 2013] [notice] Advertise initialized for process 14544
[Tue Oct 15 15:08:20 2013] [notice] Apache/2.2.22 (Win32) mod_cluster/1.2.0.Final configured -- resuming normal operations
[Tue Oct 15 15:08:20 2013] [notice] Server built: Jan 28 2012 11:16:39
[Tue Oct 15 15:08:20 2013] [notice] Parent: Created child process 7836
[Tue Oct 15 15:08:20 2013] [debug] mpm_winnt.c(477): Parent: Sent the scoreboard to the child
[Tue Oct 15 15:08:20 2013] [debug] mod_advertise.c(577): [7836 - 14544] in child post config hook
[Tue Oct 15 15:08:20 2013] [notice] Child 7836: Child process is running
[Tue Oct 15 15:08:20 2013] [debug] mpm_winnt.c(398): Child 7836: Retrieved our scoreboard from the parent.
[Tue Oct 15 15:08:20 2013] [info] Parent: Duplicating socket 280 and sending it to child process 7836
[Tue Oct 15 15:08:20 2013] [info] Parent: Duplicating socket 272 and sending it to child process 7836
[Tue Oct 15 15:08:20 2013] [debug] mpm_winnt.c(595): Parent: Sent 2 listeners to child 7836
[Tue Oct 15 15:08:20 2013] [debug] mpm_winnt.c(554): Child 7836: retrieved 2 listeners from parent
[Tue Oct 15 15:08:20 2013] [notice] Child 7836: Acquired the start mutex.
[Tue Oct 15 15:08:20 2013] [notice] Child 7836: Starting 64 worker threads.
[Tue Oct 15 15:08:20 2013] [notice] Child 7836: Starting thread to listen on port 3001.
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(1667): manager_trans STATUS 
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(2323): manager_handler STATUS (C:/) processing: "JVMRoute=nodeA&Load=100"
[Tue Oct 15 15:08:20 2013] [debug] mod_manager.c(1404): Processing STATUS
[Tue Oct 15 15:08:20 2013] [warn] manager_handler STATUS error: MEM: Can't read node
[Tue Oct 15 15:08:20 2013] [notice] Child 7836: Starting thread to listen on port 80.
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(1667): manager_trans INFO 
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(2323): manager_handler INFO (C:/) processing: ""
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(2366): manager_handler INFO  OK
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(1667): manager_trans CONFIG 
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(2323): manager_handler CONFIG (C:/) processing: "JVMRoute=nodeA&Host=localhost&Maxattempts=1&Port=8009&StickySessionForce=No&Type=ajp&ping=10"
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(2366): manager_handler CONFIG  OK
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(1667): manager_trans STATUS 
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(2323): manager_handler STATUS (C:/) processing: "JVMRoute=nodeA&Load=100"
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(1404): Processing STATUS
[Tue Oct 15 15:08:30 2013] [notice] Parent: child process exited with status 3221225477 -- Restarting.
[Tue Oct 15 15:08:30 2013] [notice] Advertise initialized for process 14544
[Tue Oct 15 15:08:30 2013] [notice] Apache/2.2.22 (Win32) mod_cluster/1.2.0.Final configured -- resuming normal operations
[Tue Oct 15 15:08:30 2013] [notice] Server built: Jan 28 2012 11:16:39
[Tue Oct 15 15:08:30 2013] [notice] Parent: Created child process 1060
[Tue Oct 15 15:08:30 2013] [debug] mpm_winnt.c(477): Parent: Sent the scoreboard to the child
[Tue Oct 15 15:08:30 2013] [debug] mod_advertise.c(577): [1060 - 14544] in child post config hook
[Tue Oct 15 15:08:30 2013] [notice] Child 1060: Child process is running
[Tue Oct 15 15:08:30 2013] [info] Parent: Duplicating socket 280 and sending it to child process 1060
[Tue Oct 15 15:08:30 2013] [debug] mpm_winnt.c(398): Child 1060: Retrieved our scoreboard from the parent.
[Tue Oct 15 15:08:30 2013] [info] Parent: Duplicating socket 272 and sending it to child process 1060
[Tue Oct 15 15:08:30 2013] [debug] mpm_winnt.c(595): Parent: Sent 2 listeners to child 1060
[Tue Oct 15 15:08:30 2013] [debug] mpm_winnt.c(554): Child 1060: retrieved 2 listeners from parent
[Tue Oct 15 15:08:30 2013] [notice] Child 1060: Acquired the start mutex.
[Tue Oct 15 15:08:30 2013] [notice] Child 1060: Starting 64 worker threads.
[Tue Oct 15 15:08:30 2013] [notice] Child 1060: Starting thread to listen on port 3001.
[Tue Oct 15 15:08:30 2013] [notice] Child 1060: Starting thread to listen on port 80.
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(1667): manager_trans STATUS 
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(2323): manager_handler STATUS (C:/) processing: "JVMRoute=nodeA&Load=100"
[Tue Oct 15 15:08:30 2013] [debug] mod_manager.c(1404): Processing STATUS
[Tue Oct 15 15:08:30 2013] [warn] manager_handler STATUS error: MEM: Can't read node

       

      Not sure what I am doing wrong.

       

      Here is the version detail,

      OS - Windows 7 Professional 32 bit

      JBoss EAP 6.0.1

      mod_cluster 1.2.0.Final

      Apache HTTP Server 2.2.22 (4 modules were added to an existing installation)

      JDK 1.7.0_25

        • 1. Re: mod_cluster - Error [MEM: MEM: Can't read node: {4}]
          vikrammurugesan
          vikrammurugesan Mar 27, 2014 5:21 PM (in response to vikrammurugesan)

          I was able to fix the problem. I just wanted to post it here so that people don't see an unanswered question.

           

          First thing - I did not use Apache 2.2.22. Instead I used mod_cluster 1.2.6.Final installation that I downloaded from here. I would suggest using apache httpd that you get from mod_cluster instead of trying to add mod_cluster module to an existing Apache. I might be wrong, but using httpd that comes with mod_cluster saves a lot of time.

           

          Here are my final settings.

           

          standalone-full-ha.xml

           

          <subsystem xmlns="urn:jboss:domain:modcluster:1.1">
          <mod-cluster-config advertise-socket="modcluster" proxy-list="127.0.0.1:8796" connector="ajp">
          <dynamic-load-provider>
               <load-metric type="busyness"/>
          </dynamic-load-provider>
          </mod-cluster-config>
</subsystem>

           

           

           

          httpd.conf

           

          # MOD_CLUSTER_ADDS
# Adjust to you hostname and subnet.
<IfModule manager_module>
  Listen 127.0.0.1:8796

  <VirtualHost 127.0.0.1:8796>
    <Directory />
     Order deny,allow
     Deny from all
     Allow from 127.0.0
    </Directory>

    KeepAliveTimeout 300
    MaxKeepAliveRequests 0
    #ServerAdvertise on http://127.0.0.1:8796
     AdvertiseFrequency 5
    #AdvertiseSecurityKey secret
    #AdvertiseGroup 224.0.1.105:23364
    EnableMCPMReceive
  ManagerBalancerName mycluster

  <Location /mod_cluster-manager>
       SetHandler mod_cluster-manager
       Order deny,allow
       Deny from all
       Allow from 127.0.0
       AllowDisplay On
    </Location>

  </VirtualHost>
</IfModule>

           

          Good luck and let me know if I have missed anything.

            • Actions