Listening to membership changes in AS7

[suman.ganta]

For cluster communication using JBoss API, On JBoss 6 and earlier, I'm looking up JNDI for org.jboss.ha.framework.interfaces.HAPartition object and use it to listen to / post cluster events. On JBoss 6 and earlier, HAPartition was part of jbossha.jar. However, on 7.0.x and 7.1, I don't find corresponding module. I'm guessing this mechanism is completely replaced in AS7. Anyone know how to change my lookup accordingly?

Here is how I'm looking up the partition:

public HAPartition getPartition() throws Exception {
    InitialContext context = new InitialContext(); 
    mPartition = (HAPartition) context.lookup("/HAPartition/" + "myCluster");
    return mPartition;
} 

It seems org.jboss.as.clustering.service.ServiceProviderRegistryService is the way to go. But I'm not sure how exactly to use it. Basically I want to listen to membership changes and invoke methods on cluster.

I'm trying it on the latest CR release of 7.1

[pferraro]

That depends - for which cluster do you want to listen to membership changes?  There are a number of ways you could do this.  I'll enumerate a few examples:

1. Using Infinispan to detect membership changes in cluster used for web session replication:

{code}@ManagedBean
@org.infinispan.notifications.Listener
public class MyBean {
   @Resource(lookup="java:jboss/infinispan/container/web")
   private org.infinispan.manager.EmbeddedCacheManager manager;
   @PostConstruct
   public void init() {
     manager.addListener(this);
   }
   @PreDestroy
   public void destroy() {
     manager.removeListener(this);
   }
   @org.infinispan.notifications.cachemanager.annotations.ViewChanged
   public void viewChanged(org.infinispan.notifications.cachemanager.event.ViewChangedEvent event) {
     // Insert your logic here
   }
}{code}

2. Create a custom subsystem that either creates a new, or injects an existing, Service<org.jboss.as.clustering.api.GroupMembershipListener & org.jboss.as.clustering.api.GroupRpcDispatcher>.  This object returned by this service is a slimmed down, slightly tidier version of HAPartition.

[suman.ganta]

Thanks Paul for that Info. It seems like you would suggest option 1 as the replacement. When we register a listener to org.infinispan.manager.EmbeddedCacheManager, does the listener receive all changes to the cache? I just need to be notified when there is change in cluster membership. Is there a way to filter the notifications that I'm not interested in? I'm getting source for Infinispan while I'm typing this.

[pferraro]

Your listener method will only be invoked for the events in which it indicates its interest via annotations.  So, in the above example, the object will only receive cluster membership change events via the viewChanged(...) method.

[suman.ganta]

Thanks again Paul. I did not find a way to invoke methods on cluster using this cache manager. Do you suggest option 2 as the only alternative to make method calls on cluster? If I can achieve the same using infinispan, that would be nice.

[pferraro]

I can think of a number ways to go about doing this:

1. You can obtain a reference to Infinispan's rpc manager via Cache.getAdvancedCache().getRpcManager()

http://docs.jboss.org/infinispan/5.1/apidocs/org/infinispan/remoting/rpc/RpcManager.html

To use this, you'll need to implement your rpcs as a ReplicableCommand.

This is documented here:

https://docs.jboss.org/author/display/ISPN/Writing+custom+commands+in+Infinispan

2. Create a JGroups muxing rpc dispatcher and use it with Infinispan's JGroups channel:

e.g.

{code}Transport transport = manager.getTransport();
Channel channel = ((JGroupsTransport) transport).getChannel();
org.jgroups.blocks.RpcDispatcher dispatcher = new org.jgroups.blocks.mux.MuxRpcDispatcher(uniqueId, channel, null, null, this);
dispatcher.start();{code}

[suman.ganta]

Hi Paul, I tried out your suggestions, I'm not able to lookup embedded cache manager. I tried the following:

1. Tried to lookup "java:jboss/infinispan/container/cluster", "java:jboss/infinispan/cluster" (cluster was the default cache container for infinisan subsystem in my env). - did not help.

2. Added start="EAGER" attribute to the replicated-cache element, and I noticed that the channel is started. However the same lookups from #1 did not work.

3. Added jndi-name="java:jboss/infinispan/container/cluster" and then jndi-name="java:/jboss/infinispan/container/cluster", and tried to lookup. did not work either.

I'm looking up with the below code -

InitialContext context = new InitialContext();

mCacheManager = (EmbeddedCacheManager)context.lookup("java:/jboss/infinispan/container/cluster");

I'm not using managed bean and injection (as you suggested above) as I've a separate thread (during my web-app startup) that initializes my class that does the lookup

I saw your webinar on AS7 clustering and you said, the jndi will be auto bound for each cache container. Do I need to do anything else for it to happen?

[pferraro]

I've changed the jndi binding behavior since 7.1.0.CR1 - and since the webinar.

1. The default jndi name for a cache container is: "java:jboss/infinispan/container/<container-name>"

However, the cache container is not started unless something demands it to start.  The @Resource(lookup) annotation will inherently demand the referenced cache container to start.  However, if you're doing a plain jndi lookup, you need to force the cache container to start eagerly, so that the jndi binding exists before attempting to look it up.

e.g.

<cache-container name="cluster" start="EAGER">

  <!-- ... -->

</cache-container>

2. To use start="EAGER" on a cache container, you'll need to use the lastest code in master (the 7.1.0.Final release is due any day now), since this feature was added after 7.1.0.CR1.  See:

https://issues.jboss.org/browse/AS7-3386

3. Specifying a custom jndi-name doesn't affect the necessity of needing to start the cache container eagerly.

BTW - also new since 7.1.0.CR1, caches are also bound to jndi - the same rules regarding eager starting apply.  The default jndi name for a cache is:

"java:jboss/infinispan/cache/<container-name>/<cache-name>

[suman.ganta]

Thanks Paul. That explains why I couldn't lookup. I'll wait for 7.1.0Final to come out.

[crazycradd]

I used to use the HAPartition to work out how many nodes were in the cluster so I could build up a ha jindi url for my client apps

So I have used the notes above to now return a EmbeddedCacheManager from CPClusterHelper.getHASessionState().but the object does not seem to contain any addresses even though I have added the eager value to the start attribute

CPClusterHelper.getHASessionState().getCurrentHAPartition().getClusterNodes().length

<cache-container name="cluster" aliases="ha-partition" default-cache="default" start="EAGER">

I currently have 2 nodes running in standalone mode

[pferraro]

In AS7, jgroups clusters start only when demanded by an application.  I'm guessing that you aren't actually starting the "cluster" cache container, but instead starting the "web" cache container (via a <distributable/> web-app), the "ejb" cache container via a @Clustered @Stateful EJB, or "hibernate" cache container via the JPA 2nd-level cache.

Let's assume you're using clustered web sessions.  To retrieve the list of address in your cluster:

{code}@ManagedBean
@Listener
public class MyBean {
  @Resource(lookup = "java:jboss/infinispan/container/web")
  private EmbeddedCacheManager container;
  private volatile List<Address> membership;
  @PostConstruct
  public void init() {
    this.membership = this.container.getMembers();
    this.container.addListener(this);
  }
  @ViewChanged
  public void viewChanged(ViewChangedEvent event) {
    this.membership = event.getCacheManager().getMembers();
  }
}{code}

To access the Infinispan API from your application, you'll need to import the module.  To do this, add the following to your META-INF/MANIFEST.MF:

{code}Dependencies: org.infinispan{code}

[hwangarias]

Hello guys,

I have discovered this post and having a look at the code of jboss SingletonService (based on msc services) and I have several doubts:

• Could be easier, or even more "standard" build a SingletonService implementation based on the code posted by paul (based on listening to ViewChanges on a infinispan cache instead on low level msc services)?
• Even "cacheManager.getCoordinator" could be used for this aim

Thx and greetings

Juan Arias

[pferraro]

I don't quite understand what you mean by easier, or "more standard".  Neither JGroups nor Infinispan is part of the public API in Wildfly or AS7.  SingletonService uses a pluggable election policy to choose the singleton master - which is not necessarily the coordinator.

[hwangarias]

Hey Paul,

first of all thank you for your answer

this is the misunderstanding... I thought of Infinispan as the standard way to do clustering in JBoss, and if you don't need a policy to select the master, maybe the coordinator can be reused to do something similar

currently besides this I'm trying to implement rpc commands in my cluster using the methods you described... I'll post my results here

thank you for your info!

Juan Arias

[pferraro]

Infinispan is not part of AS7's public API, as such should not be considered a "standard" by any means.

Wildfly 8 will introduce a public API module to perform clustering operations like you've described.