Clustering JBoss 4.0.5, not working, stuck

[egandt]

I am using the All profile, to which I have made the changes at http://wiki.jboss.org/wiki/UsingMod_jk1.2WithJBoss?action=e&windowstate=normal&mode=view
This includes adding the

jvmRoute="node1"

and

jvmRoute="node2"

to the 2 instances, setting

<attribute name="UseJK">true</attribute>

and ensuring that AJP connector is uncommented.

Yet when I start up the 2 cluster members, the first and second servers to be brought up are not finding each other. I allowed the first server to completely start then started the second server.


I have also setup apache 2.2 using mod_proxy_http as per the instructions on http://wiki.jboss.org/wiki/UsingMod_proxyWithJBoss

However if the 2 cluster members can not see each other, the Apache configuration is not really an issue yet.


First Cluster Member:

GMS: address is oraclelinuxoracle11clusterA:1420 (additional data: 16 bytes)
-------------------------------------------------------
21:42:32,178 INFO [DefaultPartition] Number of cluster members: 1
21:42:32,180 INFO [DefaultPartition] Other members: 0
21:42:32,190 INFO [DefaultPartition] Fetching state (will wait for 30000 milliseconds):
21:42:32,216 INFO [DefaultPartition] State could not be retrieved (we are the first member in group)
21:42:32,244 INFO [DefaultPartition] New cluster view for partition DefaultPartition (id: 0, delta: 0) : [172.19.2.10:1099]
21:42:32,462 INFO [DefaultPartition] I am (172.19.2.10:1099) received membershipChanged event:
21:42:32,466 INFO [DefaultPartition] Dead members: 0 ([])
21:42:32,510 INFO [DefaultPartition] New Members : 0 ([])
21:42:32,515 INFO [DefaultPartition] All Members : 1 ([172.19.2.10:1099])
21:42:32,658 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/172.19.2.10
21:42:32,725 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /172.19.2.10:1102, group=230.0.0.4, HA-JNDI address=172.19.2.10:1100
21:42:35,774 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436F6E746578744578743A312E3000000000000200000000000000E8000102000000000C3137322E31392E322E3130000DC80000000000114A426F73732F4E616D696E672F726F6F74000000000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100010001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C3137322E31392E322E3130000DC9004000000000000000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
21:42:36,123 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F702F5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000E8000102000000000C3137322E31392E322E3130000DC80000000000144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100010001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C3137322E31392E322E3130000DC9004000000000000000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
21:42:37,447 INFO [Embedded] Catalina naming disabled
21:42:37,659 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
21:42:37,708 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
21:42:38,558 INFO [Http11BaseProtocol] Initializing Coyote HTTP/1.1 on http-172.19.2.10-8080
21:42:38,611 INFO [Catalina] Initialization processed in 856 ms
21:42:38,614 INFO [StandardService] Starting service jboss.web
21:42:38,667 INFO [StandardEngine] Starting Servlet Engine: Apache Tomcat/5.5.20
21:42:38,768 INFO [StandardHost] XML validation disabled
21:42:38,850 INFO [Catalina] Server startup in 236 ms

Second Cluster Member:

-------------------------------------------------------
GMS: address is oraclelinuxoracle11clusterB:1427 (additional data: 16 bytes)
-------------------------------------------------------
21:43:56,137 INFO [DefaultPartition] Number of cluster members: 1
21:43:56,139 INFO [DefaultPartition] Other members: 0
21:43:56,185 INFO [DefaultPartition] Fetching state (will wait for 30000 milliseconds):
21:43:56,236 INFO [DefaultPartition] State could not be retrieved (we are the first member in group)
21:43:56,422 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/172.19.2.11
21:43:56,499 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /172.19.2.11:1102, group=230.0.0.4, HA-JNDI address=172.19.2.11:1100
21:43:59,903 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436F6E746578744578743A312E3000000000000200000000000000E8000102000000000C3137322E31392E322E3131000DC80000000000114A426F73732F4E616D696E672F726F6F74000000000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100010001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C3137322E31392E322E3130000DC9004000000000000000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
21:44:00,333 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F702F5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000E8000102000000000C3137322E31392E322E3131000DC80000000000144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100010001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C3137322E31392E322E3130000DC9004000000000000000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
21:44:01,720 INFO [Embedded] Catalina naming disabled
21:44:01,872 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
21:44:01,884 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
21:44:03,158 INFO [Http11BaseProtocol] Initializing Coyote HTTP/1.1 on http-172.19.2.11-8080
21:44:03,178 INFO [Catalina] Initialization processed in 1290 ms

Any ideas on what I'm doing wrong?
ERIC

[egandt]

I also tried adding

java.naming.provider.url=172.19.2.10:1100:172.19.2.11:1100

to the all/conf/jndi.properties file, which force the a set of nodes. This causes the first member to be brought up to throw exceptions, saying it can not contact the second member (which is understandable), but the second member (while not throwing any exceptions), also does not find the first cluster member:


Example of the second server starting up:

-------------------------------------------------------
GMS: address is oraclelinuxoracle11clusterA:1453 (additional data: 16 bytes)
-------------------------------------------------------
22:15:41,835 INFO [DefaultPartition] Number of cluster members: 1
22:15:41,846 INFO [DefaultPartition] Other members: 0
22:15:41,847 INFO [DefaultPartition] Fetching state (will wait for 30000 milliseconds):
22:15:41,872 INFO [DefaultPartition] State could not be retrieved (we are the first member in group)
22:15:42,578 INFO [DefaultPartition] New cluster view for partition DefaultPartition (id: 0, delta: 0) : [172.19.2.10:1099]
22:15:42,642 INFO [DefaultPartition] I am (172.19.2.10:1099) received membershipChanged event:
22:15:42,645 INFO [DefaultPartition] Dead members: 0 ([])
22:15:42,645 INFO [DefaultPartition] New Members : 0 ([])
22:15:42,646 INFO [DefaultPartition] All Members : 1 ([172.19.2.10:1099])
22:15:42,874 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/172.19.2.10
22:15:43,118 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /172.19.2.10:1102, group=230.0.0.4, HA-JNDI address=172.19.2.10:1100
22:15:50,536 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436F6E746578744578743A312E3000000000000200000000000000E8000102000000000C3137322E31392E322E3130000DC80000000000114A426F73732F4E616D696E672F726F6F74000000000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100010001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C3137322E31392E322E3130000DC9004000000000000000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
22:15:51,522 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F702F5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000E8000102000000000C3137322E31392E322E3130000DC80000000000144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100010001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C3137322E31392E322E3130000DC9004000000000000000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
22:15:54,043 INFO [Embedded] Catalina naming disabled
22:15:54,204 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
22:15:54,215 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
22:15:56,100 INFO [Http11BaseProtocol] Initializing Coyote HTTP/1.1 on http-172.19.2.10-8080
22:15:56,160 INFO [Catalina] Initialization processed in 1940 ms

Notice it did not find the primary member 172.19.2.11 in this case, even though it is already started.

[brian.stansberry]

Have a look at first item on http://wiki.jboss.org/wiki/ClusteringFAQ and following the link from there to the Troubleshooting section on the JGroups wiki page.

[egandt]

First off I turned org.jgroups to "TRACE", however I'm unsure how to read the results of this,but then I also started the 2 cluster members and attempted to probe the jgroup, as discussed at http://wiki.jboss.org/wiki/Probe:

root@oraclelinuxoracle11clusterA lib]# /opt/jdk1.6.0_02/bin/java -cp jgroups.jar org.jgroups.tests.Probe -timeout 500
java.lang.NullPointerException: null buffer || null address
at java.net.PlainDatagramSocketImpl.send(Native Method)
at java.net.DatagramSocket.send(DatagramSocket.java:612)
at org.jgroups.tests.Probe.start(Probe.java:33)
at org.jgroups.tests.Probe.main(Probe.java:93)

Fails so lets give it all the possible options as suggested:
[root@oraclelinuxoracle11clusterA lib]# /opt/jdk1.6.0_02/bin/java -cp jgroups.jar:commons-logging.jar org.jgroups.tests.Probe -timeout 500 -bind_addr 172.19.2.10 -port 7500 -addr 224.0.0.75

-- send probe on /224.0.0.75:7500


Find nothing, so lets change the port since the jboss log shows that it is listening at> 1099, 1100 or 1102:
JBOSS log> 23:09:04,979 INFO [DefaultPartition] I am (172.19.2.10:1099) received membershipChanged event:
JBOSS log> 23:09:04,991 INFO [DefaultPartition] Dead members: 0 ([])
JBOSS log> 23:09:05,032 INFO [DefaultPartition] New Members : 0 ([])
JBOSS log> 23:09:05,034 INFO [DefaultPartition] All Members : 1 ([172.19.2.10:1099])
JBOSS log> 23:09:05,124 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/172.19.2.10
JBOSS log> 23:09:05,187 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /172.19.2.10:1102, group=230.0.0.4, HA-JNDIaddress=172.19.2.10:1100

Ok testing on Ports 1099, 1100 or 1102 finds nothing, same as looking on port 7500, I also tried ports 1485, 1486 and 45571 since they showed up in the server.log file, this time (the 1485, 1486 and 45571 addresses change every time I restart JBoss so I'm assuming they are not what I want.



It would appear based on this information, that jgroups is not running at all, yet JBoss says its running and it shows in the server.log file when set to org.jgroup is set to TRACE. I also thought that perhaps the multicast address is wrong, but then I could not find any place in JBoss where the jgroup multicast port is set so I have no idea what it is if not the default. The only multicast set in the logs is "228.1.2.3", yet it appears in a different section and even running the probe on that address fails to find any groups.



The following chunk of the JBoss server.log when I start the second cluster member. I am not sure, but it appears that there is no connections made from the second member (to this the first member), which makes sense as probing does not even show that the jgroup is started.

2008-07-08 23:03:35,248 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:03:35,248 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:03:37,172 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:03:39,573 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:03:39,575 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:03:45,511 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Creating an empty ClusteredSession
2008-07-08 23:03:45,512 DEBUG [org.jboss.web.tomcat.tc5.session.SessionIDGenerator] getSessionId called: vTrmYfRCYsIAhMcEFPpSxA**
2008-07-08 23:03:45,512 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] createSession(): useJK is true. Will append JvmRoute: node2
2008-07-08 23:03:45,512 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Session with id=vTrmYfRCYsIAhMcEFPpSxA**.node2 added. Current active sessions 2
2008-07-08 23:03:45,512 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Created a ClusteredSession with id: vTrmYfRCYsIAhMcEFPpSxA**.node2
2008-07-08 23:03:45,559 DEBUG [org.jboss.web.tomcat.tc5.session.ClusteredSession] The session has expired with id: vTrmYfRCYsIAhMcEFPpSxA**.node2 -- is it local? false
2008-07-08 23:03:45,561 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Removing session from store with id: vTrmYfRCYsIAhMcEFPpSxA**
2008-07-08 23:03:45,561 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheService] Remove session from distributed store. Fqn: /JSESSION/localhost/cs/vTrmYfRCYsIAhMcEFPpSxA**
2008-07-08 23:03:45,561 WARN [org.jboss.cache.TreeCache] node /JSESSION/localhost/cs/vTrmYfRCYsIAhMcEFPpSxA** not found
2008-07-08 23:03:47,176 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:03:57,181 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:03:59,779 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:03:59,786 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:04:07,190 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:04:15,522 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Creating an empty ClusteredSession
2008-07-08 23:04:15,523 DEBUG [org.jboss.web.tomcat.tc5.session.SessionIDGenerator] getSessionId called: YgTiVX5lLdOoM63lthxGkQ**
2008-07-08 23:04:15,523 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] createSession(): useJK is true. Will append JvmRoute: node2
2008-07-08 23:04:15,540 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Session with id=YgTiVX5lLdOoM63lthxGkQ**.node2 added. Current active sessions 2
2008-07-08 23:04:15,540 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Created a ClusteredSession with id: YgTiVX5lLdOoM63lthxGkQ**.node2
2008-07-08 23:04:15,567 DEBUG [org.jboss.web.tomcat.tc5.session.ClusteredSession] The session has expired with id: YgTiVX5lLdOoM63lthxGkQ**.node2 -- is it local? false
2008-07-08 23:04:15,568 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Removing session from store with id: YgTiVX5lLdOoM63lthxGkQ**
2008-07-08 23:04:15,569 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheService] Remove session from distributed store. Fqn: /JSESSION/localhost/cs/YgTiVX5lLdOoM63lthxGkQ**
2008-07-08 23:04:15,570 WARN [org.jboss.cache.TreeCache] node /JSESSION/localhost/cs/YgTiVX5lLdOoM63lthxGkQ** not found
2008-07-08 23:04:17,200 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:04:20,494 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:04:20,495 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:04:27,237 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:04:32,946 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:04:32,962 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:04:37,242 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:04:45,519 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Creating an empty ClusteredSession
2008-07-08 23:04:45,520 DEBUG [org.jboss.web.tomcat.tc5.session.SessionIDGenerator] getSessionId called: wUDdoVz3jmRKktZhk2HGJQ**
2008-07-08 23:04:45,520 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] createSession(): useJK is true. Will append JvmRoute: node2
2008-07-08 23:04:45,521 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Session with id=wUDdoVz3jmRKktZhk2HGJQ**.node2 added. Current active sessions 2
2008-07-08 23:04:45,521 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Created a ClusteredSession with id: wUDdoVz3jmRKktZhk2HGJQ**.node2
2008-07-08 23:04:45,551 DEBUG [org.jboss.web.tomcat.tc5.session.ClusteredSession] The session has expired with id: wUDdoVz3jmRKktZhk2HGJQ**.node2 -- is it local? false
2008-07-08 23:04:45,561 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Removing session from store with id: wUDdoVz3jmRKktZhk2HGJQ**
2008-07-08 23:04:45,562 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheService] Remove session from distributed store. Fqn: /JSESSION/localhost/cs/wUDdoVz3jmRKktZhk2HGJQ**
2008-07-08 23:04:45,562 WARN [org.jboss.cache.TreeCache] node /JSESSION/localhost/cs/wUDdoVz3jmRKktZhk2HGJQ** not found
2008-07-08 23:04:47,247 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:04:53,133 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:04:53,135 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:04:53,591 DEBUG [org.jgroups.protocols.UDP]
sending msgs:
230.1.2.7:45577: 1 msgs

2008-07-08 23:04:55,562 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:04:55,563 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:04:57,251 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:05:06,152 DEBUG [org.jgroups.protocols.PING] initial mbrs are []
2008-07-08 23:05:06,152 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[]
2008-07-08 23:05:07,258 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...
2008-07-08 23:05:15,508 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Creating an empty ClusteredSession
2008-07-08 23:05:15,509 DEBUG [org.jboss.web.tomcat.tc5.session.SessionIDGenerator] getSessionId called: qt6x6A+8rq5zyFVD3EFJbg**
2008-07-08 23:05:15,509 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] createSession(): useJK is true. Will append JvmRoute: node2
2008-07-08 23:05:15,509 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Session with id=qt6x6A+8rq5zyFVD3EFJbg**.node2 added. Current active sessions 2
2008-07-08 23:05:15,509 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Created a ClusteredSession with id: qt6x6A+8rq5zyFVD3EFJbg**.node2
2008-07-08 23:05:15,519 DEBUG [org.jboss.web.tomcat.tc5.session.ClusteredSession] The session has expired with id: qt6x6A+8rq5zyFVD3EFJbg**.node2 -- is it local? false
2008-07-08 23:05:15,520 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Removing session from store with id: qt6x6A+8rq5zyFVD3EFJbg**
2008-07-08 23:05:15,521 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheService] Remove session from distributed store. Fqn: /JSESSION/localhost/cs/qt6x6A+8rq5zyFVD3EFJbg**
2008-07-08 23:05:15,521 WARN [org.jboss.cache.TreeCache] node /JSESSION/localhost/cs/qt6x6A+8rq5zyFVD3EFJbg** not found
2008-07-08 23:05:17,263 DEBUG [org.jboss.web.tomcat.tc5.session.JBossCacheManager] Looking for sessions that have expired ...

I'm just at a loss here it appears that while using the "all" profile and having followed the instructions on setting up a cluster with Apache 2.2 and modproxyhttp or modproxyajp the required jgroup is not either starting (which the log says it is) or broadcasting its existance (which seems more likely) so that other nodes can find it.

Any ideas?
ERIC