JGroups Protocol Stack Configuration & Shunning Behavior

[jiwils]

Currently, the JBoss Messaging control channel sets FD.shun=true and GMS.shun=false.

Given that JBoss Messaging doesn't use the AUTO_RECONNECT configuration for a channel due to concerns about state, should we be shunning nodes or not given that a shunned node will never attempt to rejoin the cluster?

It seems to me that we would want shunning enabled to kick out nodes that are behaving badly for whatever reason, but is our concern for the network split case reason enough to avoid this altogether?

[jay.howell]

Yeah this seems like it could be an issue. Since we don't sync state upon reconnect automatically and do it manually, a customer has run into this issue.

They are dynamically changing their datasource(change and redeploy). When they do that it causes the jgroups channel to redeploy, which does away with the state. But since the post office is not restarting, the state is null and they get a null pointer exception. I'm going to enter a bug about this, which will probably be fixed with a depends in a strategic location, so that the post office redeploys with the rest of the application.

But it looks like that when a member gets shunned, they can't rejoin the cluster without a reboot. The problem with this is that the other channels behaves differently and doesn't have to be rebooted(JBCache, Default Partition, etc). This means that after a shun, all of the other channels can rejoin except for the JBM channels. Everything else will work except for JBM. It isn't clear at that point that the server has to be restarted. If we continue, we should add messages that redisply in a loop, "This JBM member has been shunned, a reboot is REQUIRED of this server".

Thoughts? Jay:)

[belaban]

I was under the impression that JBM actually *does* rejoin the channel, but instead of letting JGroups do this automatically, they do it manually (by listening for a channelShunned() callback).

[jmesnil]

Which version of JBoss Messaging are you talking about?
In JBM 1.4.0 SP3 CP02, both FD.shun and pbcast.GMS.shun are set to true.

Am I missing something?

[jiwils]

I'm talking about the configurations in EAP (Jay can help with the version information). If they are both true in the project, then this is probably just a typo of some sort. Thanks for clarifying that.

[jiwils]

I was under the impression that JBM actually *does* rejoin the channel, but instead of letting JGroups do this automatically, they do it manually (by listening for a channelShunned() callback).

Looking at org.jboss.messaging.core.impl.postoffice.GroupMember (where MembershipListener is implemented), I don't see anything using a ChannelListener (where channelShunned is specified), but I'm not sure that's a definitive answer regarding this.

[jay.howell]

The only two places I see it are in /jms/src/main/org/jboss/jms/serverless/GroupConnection.java
Which just logs

public void channelShunned() {
log.debug("channelShunned()");
}

and
/jms/src/main/org/jboss/jms/client/p2p/P2PConnectionDelegate.java
public void channelShunned()
{
if (this.exceptionListener != null)
{
this.exceptionListener.onException(new JMSException("We were shunned."));
}
}

which I don't think are used in the clustering of the server anyway. Maybe Tim can confirm this.

But I don't find a listener that implements channelShunned in the jbm server.

Jay:)