ad infinitum: both corr and transport are null, cannot send
mikaelhg Jun 7, 2005 6:51 AMHi, we have a problem with JBossCache cluster members leaving the cluster and never coming back. Anyone seen (and hopefully fixed) similar symptoms?
Our protocol stack:
<TCP1_4 start_port="7800" bind_addr="28.14.25.20" loopback="false"/>
<TCPPING timeout="3000"
initial_hosts="28.24.25.21[7800],28.24.25.22[7800],28.24.25.25[7800],28.24.25.26[7800]"
port_range="3" num_initial_members="1"/>
<FD timeout="2000" max_tries="4"/>
<VERIFY_SUSPECT timeout="1500" down_thread="false" up_thread="false"/>
<pbcast.NAKACK gc_lag="100" retransmit_timeout="600,1200,2400,4800"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="20000" down_thread="false" max_bytes="0" up_thread="false"/>
<pbcast.GMS print_local_addr="true" join_timeout="5000" join_retry_timeout="2000" shun="true"/>
<pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>
The error log is as follows:
2005-05-25 17:59:48,842 ERROR org.jgroups.protocols.pbcast.CoordGmsImpl:262 - mbr portal-07:7800 is not a member !
2005-05-25 17:59:52,357 WARN org.jgroups.protocols.FD:220 - I was suspected, but will not remove myself from membership (waiting for EXIT message)
2005-05-25 17:59:52,358 WARN org.jgroups.protocols.FD:220 - I was suspected, but will not remove myself from membership (waiting for EXIT message)
2005-05-25 17:59:52,359 WARN org.jgroups.protocols.FD:220 - I was suspected, but will not remove myself from membership (waiting for EXIT message)
2005-05-25 17:59:52,360 WARN org.jgroups.protocols.FD:220 - I was suspected, but will not remove myself from membership (waiting for EXIT message)
2005-05-25 17:59:52,360 WARN org.jgroups.protocols.FD:220 - I was suspected, but will not remove myself from membership (waiting for EXIT message)
2005-05-25 17:59:53,862 WARN org.jgroups.protocols.pbcast.CoordGmsImpl:313 - I am the coord and I'm being am suspected -- will probably leave shortly
2005-05-25 17:59:53,923 WARN org.jgroups.protocols.pbcast.GMS:324 - checkSelfInclusion() failed, portal-06:7800 is not a member of view [portal-08:7800|4]
[portal-08:7800]; discarding view
2005-05-25 17:59:53,924 WARN org.jgroups.protocols.pbcast.GMS:333 - I (portal-06:7800) am being shunned, will leave and rejoin group (prev_members are [po
rtal-06:7800 portal-08:7800 portal-07:7800 ])
2005-05-25 17:59:53,925 INFO org.jgroups.JChannel:1251 - received an EXIT event, will leave the channel
2005-05-25 17:59:53,929 INFO org.jgroups.JChannel:1276 - closing the channel
2005-05-25 17:59:53,930 INFO org.jgroups.blocks.ConnectionTable:377 - exception is java.net.SocketException: Socket closed
2005-05-25 17:59:53,931 INFO org.jgroups.blocks.ConnectionTable:725 - exception is java.net.SocketException: Socket closed
2005-05-25 17:59:53,931 INFO org.jgroups.blocks.ConnectionTable:725 - exception is java.net.SocketException: Socket closed
2005-05-25 17:59:53,931 INFO org.jgroups.blocks.ConnectionTable:720 - exception is java.io.EOFException
2005-05-25 17:59:53,932 INFO org.jgroups.blocks.ConnectionTable:332 - addr=portal-06:7800, connections are connections (0):
2005-05-25 17:59:53,934 INFO org.jgroups.blocks.ConnectionTable:332 - addr=portal-06:7800, connections are connections (0):
2005-05-25 17:59:53,934 INFO org.jgroups.blocks.ConnectionTable:332 - addr=portal-08:7800, connections are connections (0):
2005-05-25 17:59:55,495 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,497 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,520 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,522 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,528 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,530 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,532 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,534 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,539 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,541 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,543 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,545 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
2005-05-25 17:59:55,549 ERROR org.jgroups.blocks.GroupRequest:178 - both corr and transport are null, cannot send group request
The "both corr and transport are null, cannot send group request" continues until restart.