0 Replies Latest reply on Feb 9, 2005 6:27 AM by osandum

New node won't join partition

osandum Feb 9, 2005 6:27 AM

Setting: JBoss 3.2.6 in "all"-configuration, 2-node cluster on Mac OS X 10.3.

Cluster was started on "Node A" a few days ago. When I try starting "Node B", it fails to recognize the running cluster, and proceeds to deploy its own copy of singleton services etc. Their startup log files look nearly identical, both claiming to "must be first member in group".

I have tested that basic multicast connectivity works (as per http://www.jgroups.org/javagroupsnew/docs/newuser/node15.html).

DEBUG-level log from org.jgroups (see below) indicate, that Node A sees Node B and keeps sending MERGE events every few seconds, but Node B never joins.

Any ideas about what to look for?

Part of Node A's log:

2005-02-09 11:47:04,124 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost:59052 (additional data: 14 bytes), coord_addr=bifrost:59052 (additional data: 14 bytes)], [own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:04,124 DEBUG [org.jgroups.protocols.MERGE2] found multiple coordinators: [bifrost:59052 (additional data: 14 bytes), bifrost2:52907 (additional data: 14 bytes)]; sending up MERGE event
2005-02-09 11:47:04,124 DEBUG [org.jgroups.protocols.pbcast.CoordGmsImpl] coordinators in merge protocol are: [bifrost2:52907 (additional data: 14 bytes), bifrost:59052 (additional data: 14 bytes)]
2005-02-09 11:47:07,992 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:07,993 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:07,995 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost:59058, returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost:59058, coord_addr=bifrost:59058]]
2005-02-09 11:47:07,997 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]
2005-02-09 11:47:07,998 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost:59058, coord_addr=bifrost:59058]
2005-02-09 11:47:07,998 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1995, got 2 rsps
2005-02-09 11:47:09,993 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52913, coord_addr=bifrost2:52913], [own_addr=bifrost:59058, coord_addr=bifrost:59058]]
2005-02-09 11:47:09,994 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52913, coord_addr=bifrost2:52913], [own_addr=bifrost:59058, coord_addr=bifrost:59058]]
2005-02-09 11:47:09,994 DEBUG [org.jgroups.protocols.MERGE2] found multiple coordinators: [bifrost2:52913, bifrost:59058]; sending up MERGE event
2005-02-09 11:47:09,994 DEBUG [org.jgroups.protocols.pbcast.CoordGmsImpl] coordinators in merge protocol are: [bifrost2:52913, bifrost:59058]
2005-02-09 11:47:20,601 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:20,601 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:20,604 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost:59058, returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost:59058, coord_addr=bifrost:59058]]
2005-02-09 11:47:20,607 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost:59058, coord_addr=bifrost:59058]
2005-02-09 11:47:20,608 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]
2005-02-09 11:47:20,608 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1993, got 2 rsps
2005-02-09 11:47:20,758 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:20,758 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:20,760 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost:59052 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost:59052 (additional data: 14 bytes), coord_addr=bifrost:59052 (additional data: 14 bytes)]]
2005-02-09 11:47:20,762 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost:59052 (additional data: 14 bytes), coord_addr=bifrost:59052 (additional data: 14 bytes)]
2005-02-09 11:47:20,762 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1996, got 1 rsps
2005-02-09 11:47:20,763 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]
2005-02-09 11:47:20,764 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1990, got 2 rsps
2005-02-09 11:47:23,546 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost:59058, coord_addr=bifrost:59058], [own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:23,547 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost:59058, coord_addr=bifrost:59058], [own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:23,547 DEBUG [org.jgroups.protocols.MERGE2] found multiple coordinators: [bifrost:59058, bifrost2:52913]; sending up MERGE event

Part of Node B's log:

2005-02-09 11:47:14,013 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52907 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:14,015 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]
2005-02-09 11:47:14,016 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1995, got 1 rsps
2005-02-09 11:47:16,011 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:16,012 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:20,636 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost:59058, returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:20,791 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost:59052 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:23,729 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:23,730 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:23,731 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52913, returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:23,733 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]
2005-02-09 11:47:23,733 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1997, got 1 rsps
2005-02-09 11:47:25,731 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:25,732 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:30,853 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:30,854 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:30,855 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52907 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:30,857 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]
2005-02-09 11:47:30,857 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1997, got 1 rsps
2005-02-09 11:47:32,855 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:32,855 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:38,946 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost:59052 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:42,452 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:42,452 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:42,454 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52913, returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:42,456 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]
2005-02-09 11:47:42,456 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1996, got 1 rsps
2005-02-09 11:47:44,452 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:44,452 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:44,855 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:44,855 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:44,858 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52907 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:44,862 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]
2005-02-09 11:47:44,862 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1993, got 1 rsps
2005-02-09 11:47:46,856 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:46,856 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:47:58,700 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:47:58,701 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:47:58,702 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52913, returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:47:58,704 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]
2005-02-09 11:47:58,704 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1997, got 1 rsps
2005-02-09 11:48:00,702 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:48:00,702 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52913, coord_addr=bifrost2:52913]]
2005-02-09 11:48:05,504 DEBUG [org.jgroups.protocols.PING] FIND_INITIAL_MBRS
2005-02-09 11:48:05,504 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=2000, got 0 rsps
2005-02-09 11:48:05,506 DEBUG [org.jgroups.protocols.PING] received GET_MBRS_REQ from bifrost2:52907 (additional data: 14 bytes), returning [PING: type=GET_MBRS_RSP, arg=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:48:05,509 DEBUG [org.jgroups.protocols.PING] received FIND_INITAL_MBRS_RSP, rsp=[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]
2005-02-09 11:48:05,509 DEBUG [org.jgroups.protocols.PING] waiting for initial members: time_to_wait=1995, got 1 rsps
2005-02-09 11:48:07,504 DEBUG [org.jgroups.protocols.PING] initial mbrs are [[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]
2005-02-09 11:48:07,505 DEBUG [org.jgroups.protocols.MERGE2] initial_mbrs=[[own_addr=bifrost2:52907 (additional data: 14 bytes), coord_addr=bifrost2:52907 (additional data: 14 bytes)]]