1 2 Previous Next 19 Replies Latest reply on Aug 21, 2013 11:15 AM by Paul Ferraro

    Listening to membership changes in AS7

    suman.ganta Newbie

      For cluster communication using JBoss API, On JBoss 6 and earlier, I'm looking up JNDI for org.jboss.ha.framework.interfaces.HAPartition object and use it to listen to / post cluster events. On JBoss 6 and earlier, HAPartition was part of jbossha.jar. However, on 7.0.x and 7.1, I don't find corresponding module. I'm guessing this mechanism is completely replaced in AS7. Anyone know how to change my lookup accordingly?

      Here is how I'm looking up the partition:

      public HAPartition getPartition() throws Exception {

          InitialContext context = new InitialContext();

          mPartition = (HAPartition) context.lookup("/HAPartition/" + "myCluster");

          return mPartition;

      }

      It seems org.jboss.as.clustering.service.ServiceProviderRegistryService is the way to go. But I'm not sure how exactly to use it. Basically I want to listen to membership changes and invoke methods on cluster.

      I'm trying it on the latest CR release of 7.1

        • 1. Re: Listening to membership changes in AS7
          Paul Ferraro Master

          That depends - for which cluster do you want to listen to membership changes?  There are a number of ways you could do this.  I'll enumerate a few examples:

           

          1. Using Infinispan to detect membership changes in cluster used for web session replication:

           

          {code}@ManagedBean

          @org.infinispan.notifications.Listener

          public class MyBean {

             @Resource(lookup="java:jboss/infinispan/container/web")

             private org.infinispan.manager.EmbeddedCacheManager manager;

           

             @PostConstruct

             public void init() {

               manager.addListener(this);

             }

           

             @PreDestroy

             public void destroy() {

               manager.removeListener(this);

             }

           

             @org.infinispan.notifications.cachemanager.annotations.ViewChanged

             public void viewChanged(org.infinispan.notifications.cachemanager.event.ViewChangedEvent event) {

               // Insert your logic here

             }

          }{code}

           

          2. Create a custom subsystem that either creates a new, or injects an existing, Service<org.jboss.as.clustering.api.GroupMembershipListener & org.jboss.as.clustering.api.GroupRpcDispatcher>.  This object returned by this service is a slimmed down, slightly tidier version of HAPartition.

          • 2. Re: Listening to membership changes in AS7
            suman.ganta Newbie

            Thanks Paul for that Info. It seems like you would suggest option 1 as the replacement. When we register a listener to org.infinispan.manager.EmbeddedCacheManager, does the listener receive all changes to the cache? I just need to be notified when there is change in cluster membership. Is there a way to filter the notifications that I'm not interested in? I'm getting source for Infinispan while I'm typing this.

            • 3. Re: Listening to membership changes in AS7
              Paul Ferraro Master

              Your listener method will only be invoked for the events in which it indicates its interest via annotations.  So, in the above example, the object will only receive cluster membership change events via the viewChanged(...) method.

              • 4. Re: Listening to membership changes in AS7
                suman.ganta Newbie

                Thanks again Paul. I did not find a way to invoke methods on cluster using this cache manager. Do you suggest option 2 as the only alternative to make method calls on cluster? If I can achieve the same using infinispan, that would be nice.

                • 5. Re: Listening to membership changes in AS7
                  Paul Ferraro Master

                  I can think of a number ways to go about doing this:

                   

                  1. You can obtain a reference to Infinispan's rpc manager via Cache.getAdvancedCache().getRpcManager()

                  http://docs.jboss.org/infinispan/5.1/apidocs/org/infinispan/remoting/rpc/RpcManager.html

                  To use this, you'll need to implement your rpcs as a ReplicableCommand.

                  This is documented here:

                  https://docs.jboss.org/author/display/ISPN/Writing+custom+commands+in+Infinispan

                   

                  2. Create a JGroups muxing rpc dispatcher and use it with Infinispan's JGroups channel:

                  e.g.

                  {code}Transport transport = manager.getTransport();

                  Channel channel = ((JGroupsTransport) transport).getChannel();

                  org.jgroups.blocks.RpcDispatcher dispatcher = new org.jgroups.blocks.mux.MuxRpcDispatcher(uniqueId, channel, null, null, this);

                  dispatcher.start();{code}

                  • 6. Re: Listening to membership changes in AS7
                    suman.ganta Newbie

                    Hi Paul, I tried out your suggestions, I'm not able to lookup embedded cache manager. I tried the following:

                     

                    1. Tried to lookup "java:jboss/infinispan/container/cluster", "java:jboss/infinispan/cluster" (cluster was the default cache container for infinisan subsystem in my env). - did not help.

                    2. Added start="EAGER" attribute to the replicated-cache element, and I noticed that the channel is started. However the same lookups from #1 did not work.

                    3. Added jndi-name="java:jboss/infinispan/container/cluster" and then jndi-name="java:/jboss/infinispan/container/cluster", and tried to lookup. did not work either.

                     

                    I'm looking up with the below code -

                    InitialContext context = new InitialContext();

                    mCacheManager = (EmbeddedCacheManager)context.lookup("java:/jboss/infinispan/container/cluster");

                     

                    I'm not using managed bean and injection (as you suggested above) as I've a separate thread (during my web-app startup) that initializes my class that does the lookup

                     

                    I saw your webinar on AS7 clustering and you said, the jndi will be auto bound for each cache container. Do I need to do anything else for it to happen?

                    • 7. Re: Listening to membership changes in AS7
                      Paul Ferraro Master

                      I've changed the jndi binding behavior since 7.1.0.CR1 - and since the webinar.

                       

                      1. The default jndi name for a cache container is: "java:jboss/infinispan/container/<container-name>"

                      However, the cache container is not started unless something demands it to start.  The @Resource(lookup) annotation will inherently demand the referenced cache container to start.  However, if you're doing a plain jndi lookup, you need to force the cache container to start eagerly, so that the jndi binding exists before attempting to look it up.

                      e.g.

                      <cache-container name="cluster" start="EAGER">

                        <!-- ... -->

                      </cache-container>

                       

                      2. To use start="EAGER" on a cache container, you'll need to use the lastest code in master (the 7.1.0.Final release is due any day now), since this feature was added after 7.1.0.CR1.  See:

                      https://issues.jboss.org/browse/AS7-3386

                       

                      3. Specifying a custom jndi-name doesn't affect the necessity of needing to start the cache container eagerly.

                       

                      BTW - also new since 7.1.0.CR1, caches are also bound to jndi - the same rules regarding eager starting apply.  The default jndi name for a cache is:

                      "java:jboss/infinispan/cache/<container-name>/<cache-name>

                      • 8. Re: Listening to membership changes in AS7
                        suman.ganta Newbie

                        Thanks Paul. That explains why I couldn't lookup. I'll wait for 7.1.0Final to come out.

                        • 9. Re: Listening to membership changes in AS7
                          peter craddock Novice

                          I used to use the HAPartition to work out how many nodes were in the cluster so I could build up a ha jindi url for my client apps

                           

                          So I have used the notes above to now return a EmbeddedCacheManager from CPClusterHelper.getHASessionState().but the object does not seem to contain any addresses even though I have added the eager value to the start attribute

                           

                          CPClusterHelper.getHASessionState().getCurrentHAPartition().getClusterNodes().length

                           

                          <cache-container name="cluster" aliases="ha-partition" default-cache="default" start="EAGER">

                           

                          I currently have 2 nodes running in standalone mode

                          • 10. Re: Listening to membership changes in AS7
                            Paul Ferraro Master

                            In AS7, jgroups clusters start only when demanded by an application.  I'm guessing that you aren't actually starting the "cluster" cache container, but instead starting the "web" cache container (via a <distributable/> web-app), the "ejb" cache container via a @Clustered @Stateful EJB, or "hibernate" cache container via the JPA 2nd-level cache.

                             

                            Let's assume you're using clustered web sessions.  To retrieve the list of address in your cluster:

                             

                            {code}@ManagedBean

                            @Listener

                            public class MyBean {

                              @Resource(lookup = "java:jboss/infinispan/container/web")

                              private EmbeddedCacheManager container;

                              private volatile List<Address> membership;

                             

                              @PostConstruct

                              public void init() {

                                this.membership = this.container.getMembers();

                                this.container.addListener(this);

                              }

                             

                              @ViewChanged

                              public void viewChanged(ViewChangedEvent event) {

                                this.membership = event.getCacheManager().getMembers();

                              }

                            }{code}

                             

                            To access the Infinispan API from your application, you'll need to import the module.  To do this, add the following to your META-INF/MANIFEST.MF:

                             

                            {code}Dependencies: org.infinispan{code}

                            • 11. Re: Listening to membership changes in AS7
                              Hwang ar Newbie

                              Hello guys,

                              I have discovered this post and having a look at the code of jboss SingletonService (based on msc services) and I have several doubts:

                              • Could be easier, or even more "standard" build a SingletonService implementation based on the code posted by paul (based on listening to ViewChanges on a infinispan cache instead on low level msc services)?
                              • Even "cacheManager.getCoordinator" could be used for this aim

                               

                              Thx and greetings

                              Juan Arias

                              • 12. Re: Listening to membership changes in AS7
                                Paul Ferraro Master

                                I don't quite understand what you mean by easier, or "more standard".  Neither JGroups nor Infinispan is part of the public API in Wildfly or AS7.  SingletonService uses a pluggable election policy to choose the singleton master - which is not necessarily the coordinator.

                                • 13. Re: Listening to membership changes in AS7
                                  Hwang ar Newbie

                                  Hey Paul,

                                  first of all thank you for your answer

                                   

                                  this is the misunderstanding... I thought of Infinispan as the standard way to do clustering in JBoss, and if you don't need a policy to select the master, maybe the coordinator can be reused to do something similar

                                   

                                  currently besides this I'm trying to implement rpc commands in my cluster using the methods you described... I'll post my results here

                                   

                                  thank you for your info!

                                  Juan Arias

                                  • 14. Re: Listening to membership changes in AS7
                                    Paul Ferraro Master

                                    Infinispan is not part of AS7's public API, as such should not be considered a "standard" by any means.

                                    Wildfly 8 will introduce a public API module to perform clustering operations like you've described.

                                    1 2 Previous Next