Server's failing to join cluster after restart
lordsteve_666 Aug 2, 2005 11:37 AMHi folks,
I've got a serious problem at hand: My production cluster consists of 4 JBoss 4.0.2 servers (tat-01,tat-02,tat-03,tat-04). The cluster ran stable and undisturbed for the last few weeks. Due to a deployment problem, I had to kill and restart one of the cluster servers, and now it refuses to
a) join the cluster and
b) startup properly
I tried to restart one of the remaining servers without killing it, and ran into the same problem.
All I get to see in the log file are the messages I've posted below. With this, JBoss hangs and those errors and warnings are repeated over and over in the log.
Any help or insight is highly appreciated!
- Stefan
-------------------------------------------------------
GMS: address is tat01:43229
-------------------------------------------------------
2005-08-02 08:32:26,500 INFO [org.jboss.cache.TreeCache] state could not be retrieved (must be first member in group)
2005-08-02 08:32:26,501 INFO [org.jboss.cache.TreeCache] viewAccepted(): new members: [tat01:43229]
2005-08-02 08:32:26,501 INFO [org.jboss.cache.TreeCache] new cache is null (maybe first member in cluster)
2005-08-02 08:32:26,696 INFO [org.apache.catalina.startup.Embedded] Catalina naming disabled
2005-08-02 08:32:27,083 INFO [org.apache.coyote.http11.Http11Protocol] Initializing Coyote HTTP/1.1 on http-0.0.0.0-8080
2005-08-02 08:32:27,085 INFO [org.apache.catalina.startup.Catalina] Initialization processed in 325 ms
2005-08-02 08:32:27,087 INFO [org.jboss.web.tomcat.tc5.StandardService] Starting service jboss.web
2005-08-02 08:32:27,091 INFO [org.apache.catalina.core.StandardEngine] Starting Servlet Engine: Apache Tomcat/5.5.9
2005-08-02 08:32:27,154 INFO [org.apache.catalina.core.StandardHost] XML validation disabled
2005-08-02 08:32:28,258 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33038 is not a member !
2005-08-02 08:32:28,782 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33031 (additional data: 18 bytes) is not a member !
2005-08-02 08:32:28,783 INFO [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.TatPartition] Suspected member: tat-04:33031 (additional data: 18 bytes)
2005-08-02 08:32:31,504 WARN [org.jgroups.protocols.pbcast.NAKACK] [tat01:43229] discarded message from non-member tat-03:48594
2005-08-02 08:32:33,254 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33038 is not a member !
2005-08-02 08:32:33,783 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33031 (additional data: 18 bytes) is not a member !
2005-08-02 08:32:33,783 INFO [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.TatPartition] Suspected member: tat-04:33031 (additional data: 18 bytes)
2005-08-02 08:32:38,255 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33038 is not a member !
2005-08-02 08:32:38,786 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33031 (additional data: 18 bytes) is not a member !
2005-08-02 08:32:38,787 INFO [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.TatPartition] Suspected member: tat-04:33031 (additional data: 18 bytes)
2005-08-02 08:32:43,255 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33038 is not a member !
2005-08-02 08:32:43,788 ERROR [org.jgroups.protocols.pbcast.CoordGmsImpl] mbr tat-04:33031 (additional data: 18 bytes) is not a member !
2005-08-02 08:32:43,789 INFO [org.jboss.ha.framework.interfaces.HAPartition.lifecycle.TatPartition] Suspected member: tat-04:33031 (additional data: 18 bytes)
2005-08-02 08:32:44,736 WARN [org.jgroups.protocols.pbcast.NAKACK] [tat01:43229] discarded message from non-member tat-03:48594
2005-08-02 08:32:44,737 WARN [org.jgroups.protocols.pbcast.NAKACK] [tat01:43229] discarded message from non-member tat-03:48594