8 Replies Latest reply on Jan 20, 2012 12:05 AM by sudheesh p

    How to get cluster-wide statisctics in Infinispan

    sudheesh p Newbie

      Hi,

       

       

      I am using Infinispan_5.0_Final and JDK1.6. I have written a standalone client for a distributed cache configuration to capture the cache

      statistics using JMX.

      I have enabled Cache level JMX configurations. After inserting an object into the cache and reading it, i could get the cache Hits counter as 1.

       

      My code snippet is as follows:

       

                                    MBeanServerConnection mBeanServer = ManagementFactory

                                                        .getPlatformMBeanServer();

       

                                    String jmxQuery = "org.infinispan:type=Cache,name="+"\""+"InfinispanTest(dist_sync)"+"\""+

                                                                              ",manager="+"\""+"DefaultCacheManager"+"\""+

                                                            ",component=Statistics";

                                    Set<ObjectInstance> queryResults;

                                    try {

                                              queryResults = mBeanServer.queryMBeans( new ObjectName(jmxQuery), null);

       

                                              long cacheHits = 0;

                                              for (ObjectInstance objectInstance : queryResults) {

                                                        ObjectName objectName = objectInstance.getObjectName();

                                                         cacheHits = (Long) mBeanServer.getAttribute(objectName,  "Hits");

                                    }catch(Exception e){

       

                                    }

                }

       

       

      My question is how to obtain the cluster wide cache statistics when multiple cahce instances are running.

      Could you pls provide me with some example of code or configuration.

       

      Regards

      Sudheesh

        • 1. Re: How to get cluster-wide statisctics in Infinispan
          Galder Zamarreño Master

          You'd need to loop through all MBeanServers that form the backend cluster.

          • 2. Re: How to get cluster-wide statisctics in Infinispan
            sudheesh p Newbie

            Hi Galder,

             

            Thanks for your reply.

             

            In our test scenario,  the nodes are within the local loop back i.e running on 127.0.0.1[7800], 127.0.0.1[7801].

            The clients to both the nodes are doing 2 put and 1 get operations each into the cache.

             

            This means in our scenario there is only one MBeanServer.

            When i execute the first client, the "hits" count is 1. However when i execute the second client, I expect the "hits" count to be 2. But i am getting the hits as 1 only.

             

            We have a StatisticsAgent class which implements Callable<Long> ,Externalizer<Long>. The call() override method will look up the mbeanserver for the the "hits" attribute and return its value.

             

            At the client, will using DefaultExecutorService.invokeAll(Collections <StatisticsAgent>) get us an aggregated value of the "hits" attribute?

            • 3. Re: How to get cluster-wide statisctics in Infinispan
              Galder Zamarreño Master

              Right, so if you have a single MBeanServer and you have two nodes running, you have to compile the stats from the mbeans belonging to the two caches. Information on the JMX naming conventions can be found in https://docs.jboss.org/author/display/ISPN/Management+Tooling#ManagementTooling-Infinispan4.2orlater

              • 4. Re: How to get cluster-wide statisctics in Infinispan
                sudheesh p Newbie

                I decided to the use DefaultExecutorService  to post the task of calculating the statistics on each of the Infinispan nodes.

                 

                My cache client is using the following snippet:

                 

                DefaultExecutorService service =  new DefaultExecutorService(cache);

                StatisticsAgent agent = new StatisticsAgent(cache.getName(), cache.getConfiguration().getCacheMode().toString());

                 

                 

                And the StatisticsAgent class is like :

                public class StatisticsAgent  implements Callable<Long> ,Externalizer<Long>{

                 

                @Override

                          public Long call() throws Exception {

                long totalHits =0;

                MBeanServerConnection mBeanServer = ManagementFactory

                                    .getPlatformMBeanServer();

                 

                 

                                    String jmxQuery = "org.infinispan:type=Cache,name="+"\""+cacheName+"("+cacheMode+")"+"\""+

                                                          ",manager="+"\""+"DefaultCacheManager"+"\""+

                                        ",component=Statistics";

                 

                                    Set<ObjectInstance> queryResults;

                                      try {

                                              queryResults = mBeanServer.queryMBeans(new ObjectName(jmxQuery), null);

                 

                 

                                              for (ObjectInstance objectInstance : queryResults) {

                                                        ObjectName objectName = objectInstance.getObjectName();

                   long cacheHits = (Long) mBeanServer.getAttribute(objectName, "Hits");

                   long removeHits =(Long) mBeanServer.getAttribute(objectName, "RemoveHits");

                   long totalHits = cacheHits+removeHits;

                                              }

                 

                 

                                    } catch (Exception e) {

                                              throw new RuntimeException(e.getMessage(), e);

                                    }

                 

                 

                 

                                    return totalHits;

                          }

                 

                 

                 

                }

                 

                 

                 

                 

                }

                • 5. Re: How to get cluster-wide statisctics in Infinispan
                  Galder Zamarreño Master

                  @Sudheesh, good stuff. I'm wondering what the cacheHits and removeHits addition is trying to represent exactly. Those two only cover get() and remove() calls.

                  • 6. Re: How to get cluster-wide statisctics in Infinispan
                    sudheesh p Newbie

                    Galder,

                     

                              Are only cache get() operations considered for CacheHits count? Pls clarify. We added cacheHits & removeHits to reperesent overall cache operations. Is this correct?

                     

                              Another question related to computation of  statistics is while using replicated cache configuration and executing the above with 2 clients, i get the following error in the client invoking the DefaultExecutorService:

                     

                              java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Invalid response {}

                                at org.infinispan.distexec.DefaultExecutorService$DistributedRunnableFuture.retrieveResult(DefaultExecutorService.java:582)

                                at org.infinispan.distexec.DefaultExecutorService$DistributedRunnableFuture.get(DefaultExecutorService.java:529)

                                at com.hp.ispan.jmx.test.MyTest.main(MyTest.java:53)

                              Caused by: java.lang.IllegalStateException: Invalid response {}

                                ... 3 more

                     

                     

                     

                     

                    Can we use DefaultExecutorService or  DistributedRunnableFuture in a replicated cache scenario? Pls confirm

                     

                    Is there any other work around to obtain statistics for replicated cache.

                    • 7. Re: How to get cluster-wide statisctics in Infinispan
                      Galder Zamarreño Master

                      Yup, cache hits only count for get operations that returned something other than null, see https://github.com/galderz/infinispan/blob/master/core/src/main/java/org/infinispan/interceptors/CacheMgmtInterceptor.java for more info. You might also want to take into account the misses and the store calls. Depends on your use case...

                       

                      That exception is missing where the ''Invalid response {}" is coming from, so can you please attach the entire stacktrace?

                       

                      I don't see why it should not work for replicated caches. If it's not working, it could be a bug.

                      • 8. Re: How to get cluster-wide statisctics in Infinispan
                        sudheesh p Newbie

                        Galder,

                         

                         

                        I have configured the cache for replication mode.

                         

                        <clustering mode="replication"> <stateRetrieval timeout="20000" fetchInMemoryState="true"

                                                      alwaysProvideInMemoryState="true" /> <sync replTimeout="30000" /> </clustering>

                         

                         

                         

                         

                        I have 2 clients accessing the same cache.

                         

                         

                        1. The first client (MyTestAgain.java) has 2 put and 1 get operation

                        2. The second client (MyTest.java) has 2 put and 2 get operation and also the call to the statisticsAgent implementation.(as mentioned previsouly)

                            

                         

                        First Client is :

                         

                        public class MyTestAgain {

                         

                         

                                  public static void main(String[] args) {

                         

                                                      EmbeddedCacheManager manager;

                                                      try {

                                                                manager = new DefaultCacheManager("InfinispanCfgForNode1.xml");

                                                                Cache cache = manager.getCache("InfinispanClientTest");

                         

                                                                cache.put("LC", "Cat");

                                                                cache.put("PB", "Pat");

                                                                cache.get("PB");

                         

                         

                                                      } catch (IOException e) {

                         

                                                                e.printStackTrace();

                                                      }

                         

                                  }

                         

                        }

                         

                         

                        Second Client is :

                         

                        public class MyTest {

                         

                         

                                  public static void main(String[] args) {

                                            // TODO Auto-generated method stub

                                            try {

                                                      EmbeddedCacheManager manager = new DefaultCacheManager("InfinispanCfgForNode1.xml");

                                                      Cache cache = manager.getCache("InfinispanClientTest");

                         

                                                      cache.put("V", "Vidya");

                                                      cache.put("P", "Sudheesh");

                                                      cache.get("P");

                                                      cache.get("V");

                         

                         

                                                      DefaultExecutorService service =  new DefaultExecutorService(cache);

                                                      StatisticsAgent agent = new StatisticsAgent(cache.getName(), cache.getConfiguration().getCacheModeString());

                         

                         

                                                      try {

                         

                                                                          long cacheHits =0;

                                                                          List<Future<Long>> futures = service.submitEverywhere(agent);

                         

                                                                          for(Future<Long> future : futures){

                                                                                    System.out.println("*************"+future);

                                                                                    System.out.println("future.get = " + future.get());

                                                                              cacheHits += (Long)future.get();

                                                                              System.out.println("Total Hits is  = " + cacheHits);

                                                                          }

                         

                         

                                                      } catch (InterruptedException e) {

                                                                // TODO Auto-generated catch block

                                                                e.printStackTrace();

                                                      } catch (ExecutionException e) {

                                                                // TODO Auto-generated catch block

                                                                e.printStackTrace();

                                                      }

                         

                         

                                                      service.shutdown();

                         

                                            } catch (IOException e) {

                                                      // TODO Auto-generated catch block

                                                      e.printStackTrace();

                                            }

                                  }

                         

                         

                        }

                         

                        I get the exception while doing  at line future.get()

                         

                         

                        This error is occuring when i run the second client while the first one is still running. This error will not occur when the second client is run as stand alone