Distributed state Replication problem in cluster nodes
Hi all,
I m running into a strange problem with distributed state replication across cluster nodes. I am using Jboss-4.0.2. (I have two servers in cluster. Data added into distributed state of 1 machine doesn't gets replicated in distributed state of another machine in cluster)
Distributed state entires used to replicate in older JBOSS version i.e. 3.2.7 and 3.2.2, though the replication happened very slowly in other cluster node under high load.
I enabled the following in log4j.xml to trace whats happening
<!-- Limit the org.jgroups category to WARN as its INFO is verbose -->
When i m adding some data in distributed state, following is the publishing server's (HOSTNAME: RAJESHNARKHED IPADDRESS: 10.21.0.48) server.log:
=================================
2006-04-03 20:58:10,360 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] callAsynchMethodOnCluster(true), objName=DistributedState, methodName=_set, members=[VAISH:3047 (additional data: 17 bytes)]
2006-04-03 20:58:10,360 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] dests=[VAISH:3047 (additional data: 17 bytes)], method_call=DistributedState._set(XSAMTransactionLog, 10A605E4068C5BD1EF23, {MT_TIMESTAMP=1144078090360, Service_ID=114389318443601021}), mode=6, timeout=60000
2006-04-03 20:58:10,376 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] real_dests=[VAISH:3047 (additional data: 17 bytes)]
2006-04-03 20:58:10,391 DEBUG [org.jgroups.blocks.GroupRequest] sending request (id=1144078090391)
2006-04-03 20:58:10,391 DEBUG [org.jgroups.protocols.FC] credit for rajeshnarkhed:3139 (additional data: 16 bytes) is 999813
2006-04-03 20:58:10,391 DEBUG [org.jgroups.protocols.FC] credit for VAISH:3047 (additional data: 17 bytes) is 999813
2006-04-03 20:58:10,391 DEBUG [org.jgroups.protocols.pbcast.STABLE] stable task started; num_gossip_runs=3, max_gossip_runs=3
2006-04-03 20:58:10,391 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #10
2006-04-03 20:58:10,391 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [1 - 9]
2006-04-03 20:58:10,391 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=rajeshnarkhed:3139 (additional data: 16 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=10, range=null], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=REQ, id=1144078090391, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)]}
2006-04-03 20:58:10,391 DEBUG [org.jgroups.blocks.GroupRequest] received all responses: [GroupRequest:
req_id=1144078090391
members: VAISH:3047 (additional data: 17 bytes)
responses: null
request_msg: [dst: 228.8.8.32:45566, src: rajeshnarkhed:3139 (additional data: 16 bytes) (3 headers), size = 371 bytes]
rsp_mode: 6
done: false
timeout: 60000
expected_mbrs: 0
]
2006-04-03 20:58:10,391 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] responses: [sender=VAISH:3047 (additional data: 17 bytes), retval=null, received=false, suspected=false]
2006-04-03 20:58:10,407 DEBUG [org.jgroups.protocols.UDP] received (mcast) 830 bytes from /10.21.0.48:3140 (size=830 bytes)
2006-04-03 20:58:10,407 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: rajeshnarkhed:3139 (additional data: 16 bytes) (3 headers), size = 371 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=10, range=null], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=REQ, id=1144078090391, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)]}
2006-04-03 20:58:10,407 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [rajeshnarkhed:3139 (additional data: 16 bytes)] received <rajeshnarkhed:3139 (additional data: 16 bytes)#10>
2006-04-03 20:58:10,407 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for rajeshnarkhed:3139 (additional data: 16 bytes) is: received_msgs: [10 - 10 (size=1, missing=0)], delivered_msgs: [2 - 9]
2006-04-03 20:58:10,407 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 371 bytes, total bytes received=822
2006-04-03 20:58:10,407 DEBUG [org.jgroups.protocols.FC] credit for rajeshnarkhed:3139 (additional data: 16 bytes) is 999813
2006-04-03 20:58:10,407 DEBUG [org.jgroups.blocks.RequestCorrelator] discarded request from rajeshnarkhed:3139 (additional data: 16 bytes) as we are not part of destination list (local_addr=rajeshnarkhed:3139 (additional data: 16 bytes), hdr=[Header: name=MessageDispatcher, type=REQ, id=1144078090391, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)])
2006-04-03 20:58:10,860 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] callAsynchMethodOnCluster(true), objName=DistributedState, methodName=_set, members=[VAISH:3047 (additional data: 17 bytes)]
2006-04-03 20:58:10,860 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] dests=[VAISH:3047 (additional data: 17 bytes)], method_call=DistributedState._set(XSAMTransactionLog, 9892161134:030406205810, {MT_MSG_IDS=[Ljava.lang.String;@100c858, MT_NO_OF_MSG=1, MT_TIMESTAMP=1144078090860, Service_ID=114389318443601021}), mode=6, timeout=60000
2006-04-03 20:58:10,876 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] real_dests=[VAISH:3047 (additional data: 17 bytes)]
2006-04-03 20:58:10,876 DEBUG [org.jgroups.blocks.GroupRequest] sending request (id=1144078090876)
2006-04-03 20:58:10,876 DEBUG [org.jgroups.blocks.GroupRequest] received all responses: [GroupRequest:
req_id=1144078090876
members: VAISH:3047 (additional data: 17 bytes)
responses: null
request_msg: [dst: , src: (1 headers), size = 497 bytes]
rsp_mode: 6
done: false
timeout: 60000
expected_mbrs: 0
]
2006-04-03 20:58:10,876 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] responses: [sender=VAISH:3047 (additional data: 17 bytes), retval=null, received=false, suspected=false]
2006-04-03 20:58:10,891 DEBUG [org.jgroups.protocols.FC] credit for rajeshnarkhed:3139 (additional data: 16 bytes) is 999442
2006-04-03 20:58:10,891 DEBUG [org.jgroups.protocols.FC] credit for VAISH:3047 (additional data: 17 bytes) is 999442
2006-04-03 20:58:10,891 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #11
2006-04-03 20:58:10,891 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [1 - 10]
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=rajeshnarkhed:3139 (additional data: 16 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=11, range=null], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=REQ, id=1144078090876, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)]}
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.UDP] received (mcast) 956 bytes from /10.21.0.48:3140 (size=956 bytes)
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: rajeshnarkhed:3139 (additional data: 16 bytes) (3 headers), size = 497 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=11, range=null], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=REQ, id=1144078090876, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)]}
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [rajeshnarkhed:3139 (additional data: 16 bytes)] received <rajeshnarkhed:3139 (additional data: 16 bytes)#11>
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for rajeshnarkhed:3139 (additional data: 16 bytes) is: received_msgs: [11 - 11 (size=1, missing=0)], delivered_msgs: [2 - 10]
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 497 bytes, total bytes received=1319
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.FC] credit for rajeshnarkhed:3139 (additional data: 16 bytes) is 999442
2006-04-03 20:58:10,923 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] callAsynchMethodOnCluster(true), objName=DistributedState, methodName=_remove, members=[VAISH:3047 (additional data: 17 bytes)]
2006-04-03 20:58:10,923 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] dests=[VAISH:3047 (additional data: 17 bytes)], method_call=DistributedState._remove(XSAMTransactionLog, 10A605E4068C5BD1EF23), mode=6, timeout=60000
2006-04-03 20:58:10,923 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] real_dests=[VAISH:3047 (additional data: 17 bytes)]
2006-04-03 20:58:10,923 DEBUG [org.jgroups.blocks.GroupRequest] sending request (id=1144078090923)
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.FC] credit for rajeshnarkhed:3139 (additional data: 16 bytes) is 998945
2006-04-03 20:58:10,923 DEBUG [org.jgroups.protocols.FC] credit for VAISH:3047 (additional data: 17 bytes) is 998945
2006-04-03 20:58:10,923 DEBUG [org.jgroups.blocks.GroupRequest] received all responses: [GroupRequest:
req_id=1144078090923
members: VAISH:3047 (additional data: 17 bytes)
responses: null
request_msg: [dst: , src: (1 headers), size = 167 bytes]
rsp_mode: 6
done: false
timeout: 60000
expected_mbrs: 0
]
2006-04-03 20:58:10,923 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] responses: [sender=VAISH:3047 (additional data: 17 bytes), retval=null, received=false, suspected=false]
2006-04-03 20:58:10,969 DEBUG [org.jgroups.blocks.RequestCorrelator] discarded request from rajeshnarkhed:3139 (additional data: 16 bytes) as we are not part of destination list (local_addr=rajeshnarkhed:3139 (additional data: 16 bytes), hdr=[Header: name=MessageDispatcher, type=REQ, id=1144078090876, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)])
2006-04-03 20:58:10,969 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #12
2006-04-03 20:58:10,969 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [1 - 11]
2006-04-03 20:58:10,969 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=rajeshnarkhed:3139 (additional data: 16 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=12, range=null], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=REQ, id=1144078090923, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)]}
2006-04-03 20:58:10,985 DEBUG [org.jgroups.protocols.UDP] received (mcast) 626 bytes from /10.21.0.48:3140 (size=626 bytes)
2006-04-03 20:58:10,985 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: rajeshnarkhed:3139 (additional data: 16 bytes) (3 headers), size = 167 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=12, range=null], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=REQ, id=1144078090923, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)]}
2006-04-03 20:58:10,985 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [rajeshnarkhed:3139 (additional data: 16 bytes)] received <rajeshnarkhed:3139 (additional data: 16 bytes)#12>
2006-04-03 20:58:10,985 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for rajeshnarkhed:3139 (additional data: 16 bytes) is: received_msgs: [12 - 12 (size=1, missing=0)], delivered_msgs: [2 - 11]
2006-04-03 20:58:10,985 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 167 bytes, total bytes received=1486
2006-04-03 20:58:10,985 DEBUG [org.jgroups.protocols.FC] credit for rajeshnarkhed:3139 (additional data: 16 bytes) is 998945
2006-04-03 20:58:10,985 DEBUG [org.jgroups.blocks.RequestCorrelator] discarded request from rajeshnarkhed:3139 (additional data: 16 bytes) as we are not part of destination list (local_addr=rajeshnarkhed:3139 (additional data: 16 bytes), hdr=[Header: name=MessageDispatcher, type=REQ, id=1144078090923, rsp_expected=false], dest_mbrs=[VAISH:3047 (additional data: 17 bytes)])
=================================
and below is the receiving server's (HOSTNAME: VAISH IP: 10.21.0.195)server.log output:
=================================
2006-04-03 20:59:07,392 DEBUG [org.jgroups.protocols.UDP] received (mcast) 358 bytes from /10.21.0.48:3140 (size=358 bytes)
2006-04-03 20:59:07,392 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: RAJESHNARKHED:3139 (additional data: 16 bytes) (3 headers), size = 0 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=14, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [2 : 13 (13)], vaish:3047 (additional data: 17 bytes): [0 : 0 (1)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:07,392 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#14>
2006-04-03 20:59:07,392 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [5 - 14 (size=10, missing=9)], delivered_msgs: [null - null]
2006-04-03 20:59:07,705 DEBUG [org.jgroups.protocols.pbcast.NAKACK] vaish:3047 (additional data: 17 bytes): sending XMIT_REQ ([5, 13]) to RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,705 DEBUG [org.jgroups.protocols.UDP] sending message to RAJESHNARKHED:3139 (additional data: 16 bytes) (src=vaish:3047 (additional data: 17 bytes)), headers are {NAKACK=[NAKACK: XMIT_REQ, seqno=-1, range=[5 : 13]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:07,736 DEBUG [org.jgroups.protocols.UDP] received (ucast) 4461 bytes from /10.21.0.48:3139
2006-04-03 20:59:07,736 DEBUG [org.jgroups.protocols.UDP] message is [dst: vaish:3047 (additional data: 17 bytes), src: RAJESHNARKHED:3139 (additional data: 16 bytes) (2 headers), size = 4287 bytes], headers are {NAKACK=[NAKACK: XMIT_RSP, seqno=-1, range=[5 : 13]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:07,736 DEBUG [org.jgroups.protocols.pbcast.NAKACK] received missing messages [5 : 13]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#5>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#5
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [5 - 14 (size=10, missing=8)], delivered_msgs: [null - null]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=427
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.STABLE] stable task started; num_gossip_runs=3, max_gossip_runs=3
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.ParticipantGmsImpl] view=MergeView::[RAJESHNARKHED:3139 (additional data: 16 bytes)|1] [RAJESHNARKHED:3139 (additional data: 16 bytes), vaish:3047 (additional data: 17 bytes)], subgroups=[[RAJESHNARKHED:3139 (additional data: 16 bytes)|0] [RAJESHNARKHED:3139 (additional data: 16 bytes)], [vaish:3047 (additional data: 17 bytes)|0] [vaish:3047 (additional data: 17 bytes)]]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.GMS] [local_addr=vaish:3047 (additional data: 17 bytes)] view is MergeView::[RAJESHNARKHED:3139 (additional data: 16 bytes)|1] [RAJESHNARKHED:3139 (additional data: 16 bytes), vaish:3047 (additional data: 17 bytes)], subgroups=[[RAJESHNARKHED:3139 (additional data: 16 bytes)|0] [RAJESHNARKHED:3139 (additional data: 16 bytes)], [vaish:3047 (additional data: 17 bytes)|0] [vaish:3047 (additional data: 17 bytes)]]
2006-04-03 20:59:07,767 ERROR [org.jgroups.protocols.pbcast.GMS] [vaish:3047 (additional data: 17 bytes)] received view <= current view; discarding it (current vid: [RAJESHNARKHED:3139 (additional data: 16 bytes)|1], new vid: [RAJESHNARKHED:3139 (additional data: 16 bytes)|1])
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#6>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#6
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [6 - 14 (size=9, missing=7)], delivered_msgs: [5 - 5]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#7>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#7
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [7 - 14 (size=8, missing=6)], delivered_msgs: [5 - 6]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#8>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#8
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [8 - 14 (size=7, missing=5)], delivered_msgs: [5 - 7]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#9>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#9
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [9 - 14 (size=6, missing=4)], delivered_msgs: [5 - 8]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#10>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#10
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [10 - 14 (size=5, missing=3)], delivered_msgs: [5 - 9]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#11>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#11
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [11 - 14 (size=4, missing=2)], delivered_msgs: [5 - 10]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#12>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#12
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [12 - 14 (size=3, missing=1)], delivered_msgs: [5 - 11]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <RAJESHNARKHED:3139 (additional data: 16 bytes)#13>
2006-04-03 20:59:07,767 DEBUG [org.jgroups.stack.NakReceiverWindow] added missing msg RAJESHNARKHED:3139 (additional data: 16 bytes)#13
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for RAJESHNARKHED:3139 (additional data: 16 bytes) is: received_msgs: [13 - 14 (size=2, missing=0)], delivered_msgs: [5 - 12]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 187 bytes, total bytes received=614
2006-04-03 20:59:07,767 DEBUG [org.jgroups.protocols.FC] credit for RAJESHNARKHED:3139 (additional data: 16 bytes) is 1000000
2006-04-03 20:59:07,767 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1144077844109, rsp_expected=true], dest_mbrs=[vaish:3047 (additional data: 17 bytes)]
2006-04-03 20:59:07,767 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jboss.ha.framework.server.HAPartitionImpl) with request 1144077844109
2006-04-03 20:59:07,767 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Partition RAJESH received msg
2006-04-03 20:59:07,783 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] pre methodName: DistributedReplicantManager.lookupLocalReplicants
2006-04-03 20:59:07,783 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] handlerName: DistributedReplicantManager methodName: lookupLocalReplicants
2006-04-03 20:59:07,783 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Handle: DistributedReplicantManager.lookupLocalReplicants
2006-04-03 20:59:07,783 DEBUG [org.jboss.ha.framework.server.DistributedReplicantManagerImpl.RAJESH] lookupLocalReplicants called (10.21.0.195:21099). Return: 2
2006-04-03 20:59:07,783 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] rpc call return value: [Ljava.lang.Object;@d78cb8
2006-04-03 20:59:07,799 DEBUG [org.jgroups.blocks.RequestCorrelator] sending rsp for 1144077844109 to RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.FC] credit for RAJESHNARKHED:3139 (additional data: 16 bytes) is 999813
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.UNICAST] [vaish:3047 (additional data: 17 bytes)] --> DATA(RAJESHNARKHED:3139 (additional data: 16 bytes): #40, first=false)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.UDP] sending message to RAJESHNARKHED:3139 (additional data: 16 bytes) (src=vaish:3047 (additional data: 17 bytes)), headers are {UNICAST=[UNICAST: DATA, seqno=40], UDP=[UDP:group_addr=RAJESH], MessageDispatcher=[Header: name=MessageDispatcher, type=RSP, id=1144077844109, rsp_expected=false]}
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=638
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#6 (6), vaish:3047 (additional data: 17 bytes)#0 (1) from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] sending stability msg RAJESHNARKHED:3139 (additional data: 16 bytes)#0 (6), vaish:3047 (additional data: 17 bytes)#0 (6)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] stability_task=null, delay is 14
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=662
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#7 (7), vaish:3047 (additional data: 17 bytes)#0 (1) from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=686
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#8 (8), vaish:3047 (additional data: 17 bytes)#0 (1) from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] already received gossip from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 371 bytes, total bytes received=1057
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 497 bytes, total bytes received=1554
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 167 bytes, total bytes received=1721
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=1745
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#12 (12), vaish:3047 (additional data: 17 bytes)#0 (1) from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] already received gossip from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=1769
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#13 (13), vaish:3047 (additional data: 17 bytes)#0 (1) from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.pbcast.STABLE] already received gossip from RAJESHNARKHED:3139 (additional data: 16 bytes)
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.FC] credit for RAJESHNARKHED:3139 (additional data: 16 bytes) is 999813
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.FC] credit for RAJESHNARKHED:3139 (additional data: 16 bytes) is 999442
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.UDP] received (ucast) 136 bytes from /10.21.0.48:3139
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.UDP] message is [dst: vaish:3047 (additional data: 17 bytes), src: RAJESHNARKHED:3139 (additional data: 16 bytes) (2 headers), size = 0 bytes], headers are {UNICAST=[UNICAST: DATA_ACK, seqno=40], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:07,799 DEBUG [org.jgroups.protocols.FC] credit for RAJESHNARKHED:3139 (additional data: 16 bytes) is 998945
2006-04-03 20:59:07,799 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1144078090391, rsp_expected=false], dest_mbrs=[vaish:3047 (additional data: 17 bytes)]
2006-04-03 20:59:07,799 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jboss.ha.framework.server.HAPartitionImpl) with request 1144078090391
2006-04-03 20:59:07,799 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Partition RAJESH received msg
2006-04-03 20:59:07,814 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] pre methodName: DistributedState._set
2006-04-03 20:59:07,814 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] handlerName: DistributedState methodName: _set
2006-04-03 20:59:07,814 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Handle: DistributedState._set
2006-04-03 20:59:07,814 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] rpc call return value: null
2006-04-03 20:59:07,814 DEBUG [org.jgroups.protocols.UNICAST] [vaish:3047 (additional data: 17 bytes)] <-- ACK(RAJESHNARKHED:3139 (additional data: 16 bytes): #40)
2006-04-03 20:59:07,814 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #8
2006-04-03 20:59:07,814 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 7]
2006-04-03 20:59:07,814 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=vaish:3047 (additional data: 17 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=8, range=null], STABLE=[STABILITY]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [-1 : 0 (6)], vaish:3047 (additional data: 17 bytes): [-1 : 0 (6)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:07,814 DEBUG [org.jgroups.protocols.UDP] received (mcast) 359 bytes from /10.21.0.195:3048 (size=359 bytes)
2006-04-03 20:59:07,814 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: vaish:3047 (additional data: 17 bytes) (3 headers), size = 0 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=8, range=null], STABLE=[STABILITY]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [-1 : 0 (6)], vaish:3047 (additional data: 17 bytes): [-1 : 0 (6)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:07,814 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1144078090876, rsp_expected=false], dest_mbrs=[vaish:3047 (additional data: 17 bytes)]
2006-04-03 20:59:07,814 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jboss.ha.framework.server.HAPartitionImpl) with request 1144078090876
2006-04-03 20:59:07,814 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Partition RAJESH received msg
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] pre methodName: DistributedState._set
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] handlerName: DistributedState methodName: _set
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Handle: DistributedState._set
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] rpc call return value: null
2006-04-03 20:59:07,830 DEBUG [org.jgroups.blocks.RequestCorrelator] header is [Header: name=MessageDispatcher, type=REQ, id=1144078090923, rsp_expected=false], dest_mbrs=[vaish:3047 (additional data: 17 bytes)]
2006-04-03 20:59:07,830 DEBUG [org.jgroups.blocks.RequestCorrelator] calling (org.jboss.ha.framework.server.HAPartitionImpl) with request 1144078090923
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Partition RAJESH received msg
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] pre methodName: DistributedState._remove
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] handlerName: DistributedState methodName: _remove
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] Handle: DistributedState._remove
2006-04-03 20:59:07,830 TRACE [org.jboss.ha.framework.interfaces.HAPartition.RAJESH] rpc call return value: null
2006-04-03 20:59:07,830 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <vaish:3047 (additional data: 17 bytes)#8>
2006-04-03 20:59:07,830 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vaish:3047 (additional data: 17 bytes) is: received_msgs: [8 - 8 (size=1, missing=0)], delivered_msgs: [0 - 7]
2006-04-03 20:59:07,830 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=1793
2006-04-03 20:59:07,830 DEBUG [org.jgroups.protocols.pbcast.STABLE] stability vector is [RAJESHNARKHED:3139 (additional data: 16 bytes)#0, vaish:3047 (additional data: 17 bytes)#0]
2006-04-03 20:59:07,830 DEBUG [org.jgroups.protocols.pbcast.STABLE] cancelling stability task (running=false)
2006-04-03 20:59:07,830 DEBUG [org.jgroups.protocols.pbcast.NAKACK] received digest [RAJESHNARKHED:3139 (additional data: 16 bytes): [-1 : 0 (6)], vaish:3047 (additional data: 17 bytes): [-1 : 0 (6)]]
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 8 (8)]] (num_gossip_runs=3, max_gossip_runs=3)
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #9
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 8]
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=vaish:3047 (additional data: 17 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=9, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 8 (8)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.UDP] received (mcast) 359 bytes from /10.21.0.195:3048 (size=359 bytes)
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: vaish:3047 (additional data: 17 bytes) (3 headers), size = 0 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=9, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 8 (8)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <vaish:3047 (additional data: 17 bytes)#9>
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vaish:3047 (additional data: 17 bytes) is: received_msgs: [9 - 9 (size=1, missing=0)], delivered_msgs: [0 - 8]
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=1817
2006-04-03 20:59:23,095 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#14 (14), vaish:3047 (additional data: 17 bytes)#8 (8) from vaish:3047 (additional data: 17 bytes)
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 9 (9)]] (num_gossip_runs=2, max_gossip_runs=3)
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #10
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 9]
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=vaish:3047 (additional data: 17 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=10, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 9 (9)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.UDP] received (mcast) 359 bytes from /10.21.0.195:3048 (size=359 bytes)
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: vaish:3047 (additional data: 17 bytes) (3 headers), size = 0 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=10, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 9 (9)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <vaish:3047 (additional data: 17 bytes)#10>
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vaish:3047 (additional data: 17 bytes) is: received_msgs: [10 - 10 (size=1, missing=0)], delivered_msgs: [0 - 9]
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=1841
2006-04-03 20:59:52,296 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#14 (14), vaish:3047 (additional data: 17 bytes)#9 (9) from vaish:3047 (additional data: 17 bytes)
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.STABLE] mcasting digest [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 10 (10)]] (num_gossip_runs=1, max_gossip_runs=3)
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.STABLE] stable task terminating (num_gossip_runs=0, max_gossip_runs=3)
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sending msg #11
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.NAKACK] sent_msgs: [0 - 10]
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.UDP] sending message to 228.8.8.32:45566 (src=vaish:3047 (additional data: 17 bytes)), headers are {NAKACK=[NAKACK: MSG, seqno=11, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 10 (10)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.UDP] received (mcast) 359 bytes from /10.21.0.195:3048 (size=359 bytes)
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.UDP] message is [dst: 228.8.8.32:45566, src: vaish:3047 (additional data: 17 bytes) (3 headers), size = 0 bytes], headers are {NAKACK=[NAKACK: MSG, seqno=11, range=null], STABLE=[STABLE_GOSSIP]: digest is [RAJESHNARKHED:3139 (additional data: 16 bytes): [5 : 14 (14)], vaish:3047 (additional data: 17 bytes): [0 : 10 (10)]], UDP=[UDP:group_addr=RAJESH]}
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.NAKACK] [vaish:3047 (additional data: 17 bytes)] received <vaish:3047 (additional data: 17 bytes)#11>
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.NAKACK] receiver window for vaish:3047 (additional data: 17 bytes) is: received_msgs: [11 - 11 (size=1, missing=0)], delivered_msgs: [0 - 10]
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.STABLE] received message of 24 bytes, total bytes received=1865
2006-04-03 20:59:53,484 DEBUG [org.jgroups.protocols.pbcast.STABLE] received digest RAJESHNARKHED:3139 (additional data: 16 bytes)#14 (14), vaish:3047 (additional data: 17 bytes)#10 (10) from vaish:3047 (additional data: 17 bytes)
=================================
I did try to understand whats happening in the above log, but failed to conclude anything from it. One more observation is that the two machines takes a long time for cluster node discovery...approx. 5 mins after both have started.
Here's is my cluster-service.xml:
=================================
<?xml version="1.0" encoding="UTF-8"?>
<server>
<classpath codebase="lib" archives="jbossha.jar"/>
<mbean code="org.jboss.ha.framework.server.ClusterPartition"
name="jboss:service=RAJESH">
<!-- Name of the partition being built -->
<attribute name="PartitionName">RAJESH</attribute>
<!-- Determine if deadlock detection is enabled -->
<attribute name="DeadlockDetection">False</attribute>
<!-- The JGroups protocol configuration -->
<attribute name="PartitionConfig">
<Config>
<UDP mcast_addr="228.8.8.32" mcast_port="45566" ip_ttl="16" ip_mcast="true"
mcast_send_buf_size="1000000" mcast_recv_buf_size="5000000"
ucast_send_buf_size="1000000" ucast_recv_buf_size="5000000"
loopback="false"/>
<PING timeout="2000" num_initial_members="3"
up_thread="true" down_thread="true"/>
<MERGE2 min_interval="20000" max_interval="40000"/>
<FD_SOCK down_thread="false"/>
<VERIFY_SUSPECT timeout="3000" num_msgs="3" up_thread="true" down_thread="true"/>
<pbcast.NAKACK gc_lag="1"
retransmit_timeout="300,600,1200,2400,4800"
max_xmit_size="60000"
up_thread="true"
down_thread="true"/>
<UNICAST timeout="300,600,1200,2400"
window_size="500"
min_threshold="10"
down_thread="true"/>
<pbcast.STABLE desired_avg_gossip="10000"
digest_timeout="0"
up_thread="true"
down_thread="true"
max_bytes="250000"
stability_delay="1000"/>
<pbcast.GMS join_timeout="5000"
join_retry_timeout="2000"
shun="true"
print_local_addr="true"/>
<FC max_credits="1000000" down_thread="false" min_threshold="0.10"/>
<FRAG frag_size="60000" down_thread="true" up_thread="true"/>
<pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
</Config>
</attribute>
</mbean>
<mbean code="org.jboss.ha.hasessionstate.server.HASessionStateService"
name="jboss:service=HASessionState">
<depends>jboss:service=RAJESH</depends>
<attribute name="PartitionName">RAJESH</attribute>
<attribute name="JndiName">/HASessionState/Default</attribute>
<attribute name="BeanCleaningDelay">0</attribute>
</mbean>
<mbean code="org.jboss.ha.jndi.HANamingService"
name="jboss:service=HAJNDI">
<depends>jboss:service=RAJESH</depends>
<attribute name="PartitionName">RAJESH</attribute>
<attribute name="BindAddress">${jboss.bind.address}</attribute>
<attribute name="RmiPort">0</attribute>
<attribute name="Port">1100</attribute>
<attribute name="Backlog">50</attribute>
<attribute name="AutoDiscoveryAddress">230.0.0.4</attribute>
<attribute name="AutoDiscoveryGroup">1102</attribute>
</mbean>
<mbean code="org.jboss.invocation.jrmp.server.JRMPInvokerHA"
name="jboss:service=invoker,type=jrmpha">
<attribute name="ServerAddress">${jboss.bind.address}</attribute>
</mbean>
<mbean code="org.jboss.cache.invalidation.bridges.JGCacheInvalidationBridge"
name="jboss.cache:service=InvalidationBridge,type=JavaGroups">
<depends>jboss:service=RAJESH</depends>
<depends>jboss.cache:service=InvalidationManager</depends>
<attribute name="InvalidationManager">jboss.cache:service=InvalidationManager</attribute>
<attribute name="PartitionName">RAJESH</attribute>
<attribute name="BridgeName">DefaultJGBridge</attribute>
</mbean>
</server>=================================
Please advise!
Thanks in advance,
Rajesh Narkhed