1 2 Previous Next 25 Replies Latest reply on Sep 6, 2012 9:50 AM by ksermas Go to original post
      • 15. Re: How to cluster two Content Engine server to share CPU load
        ksermas

        Yes that is correct, we should see ""Number of cluster members: 2" in both logfiles but we are not.

         

        Below is the actual output of the process:

         

        jboss    21147     1 31 Jun06 ?        2-18:23:05 /usr/lib/jvm/java/bin/java -Xms2048m -Xmx2048m -XX:MaxPermSize=512m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.messaging.ServerPeerID=1 -Dcom.sun.jndi.ldap.connect.pool.maxsize=20 -Dcom.sun.jndi.ldap.connect.pool.prefsize=10 -Dcom.sun.jndi.ldap.connect.pool.timeout=300000 -Djava.endorsed.dirs=/home/jboss/jboss-as/lib/endorsed -classpath /home/jboss/jboss-as/bin/run.jar:/usr/lib/jvm/java/lib/tools.jar org.jboss.Main -c ContentEngine -g ContentEngine -b 0.0.0.0

         

        When I look at the stdout.log file, I see that there are different clusters being initiated but none of them are working:

         

        18:09:58,728 INFO  [JMXConnectorServerService] JMX Connector server: service:jmx:rmi://vrceprodlnx01.chec.l
        ocal/jndi/rmi://vrceprodlnx01.chec.local:1090/jmxconnector
        18:09:58,973 INFO  [MailService] Mail Service bound to java:/Mail
        18:09:59,786 INFO  [SnmpAgentService] SNMP agent going active
        18:10:02,655 INFO  [ContentEngine] Initializing partition ContentEngine
        18:10:03,129 INFO  [STDOUT]
        ---------------------------------------------------------
        GMS: address is xx.xx.177.62:32886 (cluster=ContentEngine)
        ---------------------------------------------------------
        18:10:03,308 INFO  [PlatformMBeanServerRegistration] JBossCache MBeans were successfully registered to the
        platform mbean server.
        18:10:03,362 INFO  [STDOUT]
        ---------------------------------------------------------
        GMS: address is xx.xx.177.62:32886 (cluster=ContentEngine-HAPartitionCache)
        ---------------------------------------------------------
        18:10:05,189 INFO  [ContentEngine] Number of cluster members: 1
        18:10:05,190 INFO  [ContentEngine] Other members: 0
        18:10:05,370 INFO  [RPCManagerImpl] Received new cluster view: [xx.xx.177.62:32886|0] [xx.xx.177.62:32886
        ]
        18:10:05,372 INFO  [RPCManagerImpl] Cache local address is xx.xx.177.62:32886
        18:10:05,387 INFO  [RPCManagerImpl] state was retrieved successfully (in 2.02 seconds)
        18:10:05,407 INFO  [ComponentRegistry] JBoss Cache version: JBossCache 'Cascabel' 3.1.0.GA
        18:10:05,407 INFO  [ContentEngine] Fetching serviceState (will wait for 30000 milliseconds):
        18:10:05,408 INFO  [ContentEngine] State could not be retrieved (we are the first member in group)
        18:10:05,498 INFO  [HANamingService] Started HAJNDI bootstrap; jnpPort=1100, backlog=50, bindAddress=/0.0.0     <-(jnp Port is set to 1100 here, shoudn't it be 1099?)
        .0
        18:10:05,510 INFO  [DetachedHANamingService$AutomaticDiscovery] Listening on /0.0.0.0:1102, group=230.0.0.4
        , HA-JNDI address=xx.xx.177.62:1100

         

        ---------------------------------------------------------
        GMS: address is xx.xx.172.62:32886 (cluster=MessagingPostOffice-CTRL)
        ---------------------------------------------------------
        18:10:14,802 INFO  [GroupMember] org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipList
        ener@2b5fab got new view [xx.xx.177.62:32886|0] [xx.xx.177.62:32886], old view is null
        18:10:14,802 INFO  [GroupMember] I am (xx.xx.177.62:32886)
        18:10:14,802 INFO  [GroupMember] New Members : 1 ([xx.xx.177.62:32886])
        18:10:14,803 INFO  [GroupMember] All Members : 1 ([xx.xx.177.62:32886])
        18:10:14,833 INFO  [STDOUT]

         

        Then there is this error that repeats:

         

        18:10:43,520 WARN  [NAKACK] xx.xx.177.62:32886] discarded message from non-member xx.xx.177.63:51724, my view is [xx.xx.177.62:32886|0] [xx.xx.17
        7.62:32886]
        18:10:43,527 INFO  [STDOUT] 91953 [OOB-8,xx.xx.177.62:32886] WARN org.jgroups.protocols.pbcast.NAKACK  - xx.xx.177.62:32886] discarded message from
        non-member xx.xx.177.63:51724, my view is [xx.xx.177.62:32886|0] [xx.xx.177.62:32886]
        18:21:56,638 INFO  [STDOUT] 765071 [Incoming-5,xx.xx.177.62:32886] ERROR org.jgroups.protocols.pbcast.NAKACK  - range is null
        18:21:56,679 INFO  [ContentEngine] New cluster view for partition ContentEngine (id: 1, delta: 1) : [xx.xx.177.62:1099, xx.xx.177.63:1099]
        18:21:56,680 INFO  [STDOUT] 765112 [Incoming-5,xx.xx.177.62:32886] INFO org.jboss.ha.framework.interfaces.HAPartition.lifecycle.ContentEngine  - New
        cluster view for partition ContentEngine (id: 1, delta: 1) : [xx.xx.177.62:1099, xx.xx.177.63:1099]

         

        I hope these messages can show you why it's not clustering.

         

        Thanks again!!

        • 16. Re: How to cluster two Content Engine server to share CPU load
          wdfink

          How do you start the other member? And how the log looks like here?

           

          One reason might be a firewall or misconfigured network in between that did not allow multicast.

          You might check this with this JGroups test

          • 17. Re: How to cluster two Content Engine server to share CPU load
            ksermas

            Yes both servers are started the same way and both logs look the same.  I'm not able to run that cluster test because the example on that page is

            with a PC directory structure,  it doesn't show what the equivalent directory is in Linux.

            • 18. Re: How to cluster two Content Engine server to share CPU load
              wdfink

              For the test you need only to locate the jars in the JBoss directory and start the java command:

              java -cp lib\concurrent.jar;server\default\lib\jgroups.jar;server\default\lib\commons-logging.jar org.jgroups.demos.ViewDemo -props c:\test.xml

               

              do this on each box where your JBoss instances are.

               

              If you start both instances with the '-g ContentEngine' parameter they should build a cluster.

              Another test might to copy both instances to the same server and start it to check whether a cluster is build correct here.

              But remember you have to use port-binding manager or different interface binding (for two instances you can use -b 127.0.0.1 and -b x.x.177.62 (or63 just the real ip address) .

              • 19. Re: How to cluster two Content Engine server to share CPU load
                ksermas

                I can't run that test because I'm on a Linux box.  I have tried to run what may be the equivelant in Linux but I get the error:

                 

                $ java -jar jboss-5.1.0.GA/client/concurrent.jar;jboss-5.1.0.GA/server/ContentEngine/lib/jgroups.jar;/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar org.jgroups.demos.ViewDemo -props /tmp/test.xml
                Failed to load Main-Class manifest attribute from jboss-5.1.0.GA/client/concurrent.jar

                Version:      2.6.10.GA
                CVS:          $Id: Version.java,v 1.59.2.18 2009/04/28 14:25:33 vlada Exp $

                Failed to load Main-Class manifest attribute from
                /home/jboss/jboss-5.1.0.GA/client/commons-logging.jar

                 

                 

                 

                • 20. Re: How to cluster two Content Engine server to share CPU load
                  wdfink

                  Belive me I run that on different OS and different JBoss versions

                  But you use the wrong java command

                   

                  I (and the wiki)   => java -cp ....

                  You                       => java -jar ...

                   

                  That is the reason for the failure.

                  So check again

                  • 21. Re: How to cluster two Content Engine server to share CPU load
                    ksermas

                    I ran the command from my jboss home directory '/home/jboss'

                    Here's the error I get when I run the command with 'java -cp'

                     

                    [ ~]$ java -cp jboss-5.1.0.GA/client/concurrent.jar;jboss-5.1.0.GA/server/ContentEngine/lib/jgroups.jar;/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar org.jgroups.demos.ViewDemo -props /tmp/test.xml
                    Usage: gij [OPTION] ... CLASS [ARGS] ...
                              to invoke CLASS.main, or
                           gij -jar [OPTION] ... JARFILE [ARGS] ...
                              to execute a jar file
                    Try `gij --help' for more information.

                    Version:      2.6.10.GA
                    CVS:          $Id: Version.java,v 1.59.2.18 2009/04/28 14:25:33 vlada Exp $

                    Failed to load Main-Class manifest attribute from
                    /home/jboss/jboss-5.1.0.GA/client/commons-logging.jar

                    • 22. Re: How to cluster two Content Engine server to share CPU load
                      wdfink

                      As you use a unix/linux box the separator of the -cp parameters is ':' instead of ';' (for Windows).

                      Also ensure that you have the correct java  (test with java -version)

                      • 23. Re: How to cluster two Content Engine server to share CPU load
                        ksermas

                        Yes changing them to ':' did allow it to execute but then I got this error:

                         

                        [jboss@vrtestlnx01 Desktop]$ java -cp jboss-5.1.0.GA/client/concurrent.jar:jboss-5.1.0.GA/server/ContentEngine/lib/jgroups.jar:/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar org.jgroups.demos.ViewDemo -props /tmp/test.xml
                        Exception in thread "main" java.lang.NoClassDefFoundError: org.jgroups.demos.ViewDemo
                           at gnu.java.lang.MainThread.run(libgcj.so.7rh)
                        Caused by: java.lang.ClassNotFoundException: org.jgroups.demos.ViewDemo not found in gnu.gcj.runtime.SystemClassLoader{urls=[file:/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar], parent=gnu.gcj.runtime.ExtensionClassLoader{urls=[], parent=null}}
                           at java.net.URLClassLoader.findClass(libgcj.so.7rh)
                           at java.lang.ClassLoader.loadClass(libgcj.so.7rh)
                           at java.lang.ClassLoader.loadClass(libgcj.so.7rh)
                           at gnu.java.lang.MainThread.run(libgcj.so.7rh)

                        • 24. Re: How to cluster two Content Engine server to share CPU load
                          wdfink

                          Please do your homework and check that the jar files exists and you have the correct java command.

                          I suppose there is something wrong with your jar's or the classpath.

                          • 25. Re: How to cluster two Content Engine server to share CPU load
                            ksermas

                            After all that I found out that multicasting indeed was NOT enabled on our network   I'm sure once it's enabled this will work.

                             

                            Thanks very much Wolf-Dieter. 

                            1 2 Previous Next