1 2 Previous Next 16 Replies Latest reply on Sep 8, 2009 3:42 AM by Marcin Matuszak

    How to set up a two-node cluster

    Steve Lihn Newbie

      Hi, I just downloaded infinispan and ran the demo. It looks greate.

      I am very interested in infinispan in a cluster environment, but I can't find any document how to set it up as a cluster (e.g. two node). Can anyone provide a very basic instruction to demo this ? Or it is just as simple as bringing them up and they will talk to each other and form a cluster? (Using TCP or UDP?)

      We have Hibernate running with Ehcache in Tomcat. I am also familar with jboss jgroup. If you can point me to something along these lines. That would be great...

      Thanks,
      Steve

        • 1. Re: How to set up a two-node cluster
          Manik Surtani Master

          This doc is still work in progress, but do have a look. Feel free to add to the doc, help make it more user-friendly, etc.

          http://www.jboss.org/community/wiki/SettingupanInfinispanCluster

          • 2. Re: How to set up a two-node cluster
            Weitat Wee Newbie

             

            "stevehtl" wrote:
            Hi, I just downloaded infinispan and ran the demo. It looks greate.

            I am very interested in infinispan in a cluster environment, but I can't find any document how to set it up as a cluster (e.g. two node). Can anyone provide a very basic instruction to demo this ? Or it is just as simple as bringing them up and they will talk to each other and form a cluster? (Using TCP or UDP?)

            We have Hibernate running with Ehcache in Tomcat. I am also familar with jboss jgroup. If you can point me to something along these lines. That would be great...

            Thanks,
            Steve


            Here's a quick sample code for reference:
            String cacheName="myCache";
             // Create cachemanagers
             CacheManager cm1 = createManager();
             CacheManager cm2 = createManager();
             cm1.defineCache(cacheName, getDefaultClusteredConfig(CacheMode.REPL_ASYNC));
             cm2.defineCache(cacheName, getDefaultClusteredConfig(CacheMode.REPL_ASYNC));
            
             // Get the caches
             Cache<String, String> cache1 = cm1.getCache(cacheName);
             Cache<String, String> cache2 = cm2.getCache(cacheName);
            
             // Put and access cache
             cache1.put("hello", "world");
             System.out.println(cache2.get("hello"));
            


            One mistake I made earlier was that I was putting stuff into cache1 before I call cm2.getCache(cacheName). As a result, the value which I put into cache1 was not replicated over to cache2. So watch out for that! :)

            Infinispan looks great so far :)

            • 3. Re: How to set up a two-node cluster
              Chandra Sreeraman Newbie

               

              "weitat" wrote:
              "stevehtl" wrote:
              Hi, I just downloaded infinispan and ran the demo. It looks greate.

              I am very interested in infinispan in a cluster environment, but I can't find any document how to set it up as a cluster (e.g. two node). Can anyone provide a very basic instruction to demo this ? Or it is just as simple as bringing them up and they will talk to each other and form a cluster? (Using TCP or UDP?)

              We have Hibernate running with Ehcache in Tomcat. I am also familar with jboss jgroup. If you can point me to something along these lines. That would be great...

              Thanks,
              Steve


              Here's a quick sample code for reference:
              String cacheName="myCache";
               // Create cachemanagers
               CacheManager cm1 = createManager();
               CacheManager cm2 = createManager();
               cm1.defineCache(cacheName, getDefaultClusteredConfig(CacheMode.REPL_ASYNC));
               cm2.defineCache(cacheName, getDefaultClusteredConfig(CacheMode.REPL_ASYNC));
              
               // Get the caches
               Cache<String, String> cache1 = cm1.getCache(cacheName);
               Cache<String, String> cache2 = cm2.getCache(cacheName);
              
               // Put and access cache
               cache1.put("hello", "world");
               System.out.println(cache2.get("hello"));
              


              One mistake I made earlier was that I was putting stuff into cache1 before I call cm2.getCache(cacheName). As a result, the value which I put into cache1 was not replicated over to cache2. So watch out for that! :)

              Infinispan looks great so far :)


              Is there a way for Late joiners to see the data from an existing cache?

              One of the requirements I have is to pre-load a cache and have a bunch of short running programs access from the cache (from a different JVM). Would that be possible?

              Thanks

              • 4. Re: How to set up a two-node cluster
                Chandra Sreeraman Newbie

                ..to clarify further - JBossCache has the buddy-replication system (and in the GUI you can see this), where a late joiner can see data from their buddies. I am womdering if there is a similar mechanism at play here

                • 5. Re: How to set up a two-node cluster
                  Manik Surtani Master

                  chsreer - yes, if you are using REPLICATION as your cache mode, make sure you enable state retrieval.

                   <clustering>
                   <stateRetrieval timeout="20000" fetchInMemoryState="true"/>
                   </clustering>
                  


                  • 6. Re: How to set up a two-node cluster
                    Chandra Sreeraman Newbie

                     

                    "manik.surtani@jboss.com" wrote:
                    chsreer - yes, if you are using REPLICATION as your cache mode, make sure you enable state retrieval.

                     <clustering>
                     <stateRetrieval timeout="20000" fetchInMemoryState="true"/>
                     </clustering>
                    


                    Thank You, Manik.

                    • 7. Re: How to set up a two-node cluster
                      Chandra Sreeraman Newbie

                       

                      "manik.surtani@jboss.com" wrote:
                      chsreer - yes, if you are using REPLICATION as your cache mode, make sure you enable state retrieval.

                       <clustering>
                       <stateRetrieval timeout="20000" fetchInMemoryState="true"/>
                       </clustering>
                      


                      Manik

                      I did as you mentioned and when I invoke the cache from a different program I am getting an error.

                      Here is the code to get the cache instance

                       if (cache == null) {
                      
                       try {
                       cache = cacheManager.getCache();
                       } catch (Exception e) {
                       e.printStackTrace();
                       }
                       } else if (cache.getStatus() != ComponentStatus.RUNNING) {
                       cache.start();
                       }
                      


                      ...and here is a snippet of the exception

                      org.infinispan.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.start() throws org.infinispan.statetransfer.StateTransferException on object
                       at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:137)
                       at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:851)
                       at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:694)
                       at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:584)
                       at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:142)
                       at org.infinispan.CacheDelegate.start(CacheDelegate.java:245)
                       at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:342)
                       at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:306)
                       at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:284)
                       at com.miax.itx.core.impl.ITXConfiguration.startCache(ITXConfiguration.java:212)
                       at com.miax.itx.cache.ITXDefaultCacheManagerTest.setUp(ITXDefaultCacheManagerTest.java:29)
                      



                      Can you please let me know what I am doing wrong?

                      Thanks in advance

                      Chandra Sreeraman

                      • 8. Re: How to set up a two-node cluster
                        Chandra Sreeraman Newbie

                        ..and here is the rest of the exception

                        Caused by: java.lang.reflect.InvocationTargetException
                         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                         at java.lang.reflect.Method.invoke(Method.java:597)
                         at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:134)
                         ... 30 more
                        Caused by: org.infinispan.statetransfer.StateTransferException: Unable to fetch state on startup
                         at org.infinispan.remoting.rpc.RpcManagerImpl.retrieveState(RpcManagerImpl.java:165)
                         at org.infinispan.statetransfer.StateTransferManagerImpl.start(StateTransferManagerImpl.java:121)
                         ... 35 more
                        


                        • 9. Re: How to set up a two-node cluster
                          Manik Surtani Master

                          For whatever reason it cannot fetch state on startup. There must be more in that stack trace? Have you tried enabling trace level logging?

                          • 10. Re: How to set up a two-node cluster
                            Rafael Ribeiro Novice

                            As Steve I downloaded Infinispan and tried to run the guiDemo on multiple machines on the same network but using default configuration and even a modified one (I tried using a sample I had from jgroups), the guiDemo was unable to detect other nodes. Cluster view always shown only a single node.
                            Is there any change I need to do in order for a node to detect the other one? Is there any traces I can enable to figure out why nodes are not joining the same cluster view?
                            Using tcpview on Windows I could even see a quick connection between the nodes, but connection was closed almost instantly.

                            regards,

                            • 11. Re: How to set up a two-node cluster
                              Manik Surtani Master

                              Hi - did you use the startup script to launch the GUI demo? If not, what jvm params did you pass in? Also do you have IP multicast enabled and working on your env?

                              • 12. Re: How to set up a two-node cluster
                                Rafael Ribeiro Novice

                                Yes I used the startup script and also tried the workaround I mentioned of editing the default configuration (inside the jar, I know I should have a copy of it somewhere instead of editing it directly).
                                About IP multicasting in my environment I need to check.
                                May I reply with another question?
                                Suppose I have a JGroups really silly test I did here and it worked. Wasn't Infinispan supposed to work with the same setup?
                                I also noticed a funny thing over here after I changed log to debug:

                                2009-08-05 13:25:16,355 DEBUG [UDP] (pool-1-thread-1) sockets will use interface
                                 127.0.0.1
                                2009-08-05 13:25:16,371 DEBUG [UDP] (pool-1-thread-1) socket information:
                                , mcast_addr=228.10.10.10:45588, bind_addr=/127.0.0.1, ttl=2
                                sock: bound to 127.0.0.1:4840, receive buffer size=20000000, send buffer size=64
                                0000
                                

                                Isnt it weird that it is listenning on loopback interface?

                                • 13. Re: How to set up a two-node cluster
                                  Manik Surtani Master

                                  Yes, the GUI demo was designed to be run on 1 machine. So, 2 or 3 instances of it on 1 machine, all listening on loopback. You will notice the -Dbind.address=127.0.0.1 VM param in the startup script.

                                  To test that multicast is working, you should download the latest (2.8.0.Beta2) version of JGroups and try the Draw program:

                                  http://jgroups.org/tutorial/html/ch01.html#d0e124

                                  • 14. Re: How to set up a two-node cluster
                                    Rafael Ribeiro Novice

                                    Hi Manik!

                                    Removing the -Dbind.address did the trick!
                                    The infinispan demo is working!

                                    tks a lot!

                                    1 2 Previous Next