-
15. Re: How to cluster two Content Engine server to share CPU load
ksermas Jun 15, 2012 11:01 AM (in response to wdfink)Yes that is correct, we should see ""Number of cluster members: 2" in both logfiles but we are not.
Below is the actual output of the process:
jboss 21147 1 31 Jun06 ? 2-18:23:05 /usr/lib/jvm/java/bin/java -Xms2048m -Xmx2048m -XX:MaxPermSize=512m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.messaging.ServerPeerID=1 -Dcom.sun.jndi.ldap.connect.pool.maxsize=20 -Dcom.sun.jndi.ldap.connect.pool.prefsize=10 -Dcom.sun.jndi.ldap.connect.pool.timeout=300000 -Djava.endorsed.dirs=/home/jboss/jboss-as/lib/endorsed -classpath /home/jboss/jboss-as/bin/run.jar:/usr/lib/jvm/java/lib/tools.jar org.jboss.Main -c ContentEngine -g ContentEngine -b 0.0.0.0
When I look at the stdout.log file, I see that there are different clusters being initiated but none of them are working:
18:09:58,728 INFO [JMXConnectorServerService] JMX Connector server: service:jmx:rmi://vrceprodlnx01.chec.l
ocal/jndi/rmi://vrceprodlnx01.chec.local:1090/jmxconnector
18:09:58,973 INFO [MailService] Mail Service bound to java:/Mail
18:09:59,786 INFO [SnmpAgentService] SNMP agent going active
18:10:02,655 INFO [ContentEngine] Initializing partition ContentEngine
18:10:03,129 INFO [STDOUT]
---------------------------------------------------------
GMS: address is xx.xx.177.62:32886 (cluster=ContentEngine)
---------------------------------------------------------
18:10:03,308 INFO [PlatformMBeanServerRegistration] JBossCache MBeans were successfully registered to the
platform mbean server.
18:10:03,362 INFO [STDOUT]
---------------------------------------------------------
GMS: address is xx.xx.177.62:32886 (cluster=ContentEngine-HAPartitionCache)
---------------------------------------------------------
18:10:05,189 INFO [ContentEngine] Number of cluster members: 1
18:10:05,190 INFO [ContentEngine] Other members: 0
18:10:05,370 INFO [RPCManagerImpl] Received new cluster view: [xx.xx.177.62:32886|0] [xx.xx.177.62:32886
]
18:10:05,372 INFO [RPCManagerImpl] Cache local address is xx.xx.177.62:32886
18:10:05,387 INFO [RPCManagerImpl] state was retrieved successfully (in 2.02 seconds)
18:10:05,407 INFO [ComponentRegistry] JBoss Cache version: JBossCache 'Cascabel' 3.1.0.GA
18:10:05,407 INFO [ContentEngine] Fetching serviceState (will wait for 30000 milliseconds):
18:10:05,408 INFO [ContentEngine] State could not be retrieved (we are the first member in group)
18:10:05,498 INFO [HANamingService] Started HAJNDI bootstrap; jnpPort=1100, backlog=50, bindAddress=/0.0.0 <-(jnp Port is set to 1100 here, shoudn't it be 1099?)
.0
18:10:05,510 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /0.0.0.0:1102, group=230.0.0.4
, HA-JNDI address=xx.xx.177.62:1100---------------------------------------------------------
GMS: address is xx.xx.172.62:32886 (cluster=MessagingPostOffice-CTRL)
---------------------------------------------------------
18:10:14,802 INFO [GroupMember] org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipList
ener@2b5fab got new view [xx.xx.177.62:32886|0] [xx.xx.177.62:32886], old view is null
18:10:14,802 INFO [GroupMember] I am (xx.xx.177.62:32886)
18:10:14,802 INFO [GroupMember] New Members : 1 ([xx.xx.177.62:32886])
18:10:14,803 INFO [GroupMember] All Members : 1 ([xx.xx.177.62:32886])
18:10:14,833 INFO [STDOUT]Then there is this error that repeats:
18:10:43,520 WARN [NAKACK] xx.xx.177.62:32886] discarded message from non-member xx.xx.177.63:51724, my view is [xx.xx.177.62:32886|0] [xx.xx.17
7.62:32886]
18:10:43,527 INFO [STDOUT] 91953 [OOB-8,xx.xx.177.62:32886] WARN org.jgroups.protocols.pbcast.NAKACK - xx.xx.177.62:32886] discarded message from
non-member xx.xx.177.63:51724, my view is [xx.xx.177.62:32886|0] [xx.xx.177.62:32886]
18:21:56,638 INFO [STDOUT] 765071 [Incoming-5,xx.xx.177.62:32886] ERROR org.jgroups.protocols.pbcast.NAKACK - range is null
18:21:56,679 INFO [ContentEngine] New cluster view for partition ContentEngine (id: 1, delta: 1) : [xx.xx.177.62:1099, xx.xx.177.63:1099]
18:21:56,680 INFO [STDOUT] 765112 [Incoming-5,xx.xx.177.62:32886] INFO org.jboss.ha.framework.interfaces.HAPartition.lifecycle.ContentEngine - New
cluster view for partition ContentEngine (id: 1, delta: 1) : [xx.xx.177.62:1099, xx.xx.177.63:1099]I hope these messages can show you why it's not clustering.
Thanks again!!
-
16. Re: How to cluster two Content Engine server to share CPU load
wdfink Jun 15, 2012 11:44 AM (in response to ksermas)How do you start the other member? And how the log looks like here?
One reason might be a firewall or misconfigured network in between that did not allow multicast.
You might check this with this JGroups test
-
17. Re: How to cluster two Content Engine server to share CPU load
ksermas Jun 15, 2012 12:10 PM (in response to wdfink)Yes both servers are started the same way and both logs look the same. I'm not able to run that cluster test because the example on that page is
with a PC directory structure, it doesn't show what the equivalent directory is in Linux.
-
18. Re: How to cluster two Content Engine server to share CPU load
wdfink Jun 15, 2012 3:26 PM (in response to ksermas)For the test you need only to locate the jars in the JBoss directory and start the java command:
java -cp lib\concurrent.jar;server\default\lib\jgroups.jar;server\default\lib\commons-logging.jar org.jgroups.demos.ViewDemo -props c:\test.xml
do this on each box where your JBoss instances are.
If you start both instances with the '-g ContentEngine' parameter they should build a cluster.
Another test might to copy both instances to the same server and start it to check whether a cluster is build correct here.
But remember you have to use port-binding manager or different interface binding (for two instances you can use -b 127.0.0.1 and -b x.x.177.62 (or63 just the real ip address) .
-
19. Re: How to cluster two Content Engine server to share CPU load
ksermas Jun 15, 2012 5:21 PM (in response to wdfink)I can't run that test because I'm on a Linux box. I have tried to run what may be the equivelant in Linux but I get the error:
$ java -jar jboss-5.1.0.GA/client/concurrent.jar;jboss-5.1.0.GA/server/ContentEngine/lib/jgroups.jar;/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar org.jgroups.demos.ViewDemo -props /tmp/test.xml
Failed to load Main-Class manifest attribute from jboss-5.1.0.GA/client/concurrent.jarVersion: 2.6.10.GA
CVS: $Id: Version.java,v 1.59.2.18 2009/04/28 14:25:33 vlada Exp $Failed to load Main-Class manifest attribute from
/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar -
20. Re: How to cluster two Content Engine server to share CPU load
wdfink Jun 16, 2012 6:05 AM (in response to ksermas)Belive me I run that on different OS and different JBoss versions
But you use the wrong java command
I (and the wiki) => java -cp ....
You => java -jar ...
That is the reason for the failure.
So check again
-
21. Re: How to cluster two Content Engine server to share CPU load
ksermas Jun 18, 2012 9:55 AM (in response to wdfink)I ran the command from my jboss home directory '/home/jboss'
Here's the error I get when I run the command with 'java -cp'
[ ~]$ java -cp jboss-5.1.0.GA/client/concurrent.jar;jboss-5.1.0.GA/server/ContentEngine/lib/jgroups.jar;/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar org.jgroups.demos.ViewDemo -props /tmp/test.xml
Usage: gij [OPTION] ... CLASS [ARGS] ...
to invoke CLASS.main, or
gij -jar [OPTION] ... JARFILE [ARGS] ...
to execute a jar file
Try `gij --help' for more information.Version: 2.6.10.GA
CVS: $Id: Version.java,v 1.59.2.18 2009/04/28 14:25:33 vlada Exp $Failed to load Main-Class manifest attribute from
/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar -
22. Re: How to cluster two Content Engine server to share CPU load
wdfink Jun 18, 2012 11:43 AM (in response to ksermas)As you use a unix/linux box the separator of the -cp parameters is ':' instead of ';' (for Windows).
Also ensure that you have the correct java (test with java -version)
-
23. Re: How to cluster two Content Engine server to share CPU load
ksermas Jun 18, 2012 4:20 PM (in response to wdfink)Yes changing them to ':' did allow it to execute but then I got this error:
[jboss@vrtestlnx01 Desktop]$ java -cp jboss-5.1.0.GA/client/concurrent.jar:jboss-5.1.0.GA/server/ContentEngine/lib/jgroups.jar:/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar org.jgroups.demos.ViewDemo -props /tmp/test.xml
Exception in thread "main" java.lang.NoClassDefFoundError: org.jgroups.demos.ViewDemo
at gnu.java.lang.MainThread.run(libgcj.so.7rh)
Caused by: java.lang.ClassNotFoundException: org.jgroups.demos.ViewDemo not found in gnu.gcj.runtime.SystemClassLoader{urls=[file:/home/jboss/jboss-5.1.0.GA/client/commons-logging.jar], parent=gnu.gcj.runtime.ExtensionClassLoader{urls=[], parent=null}}
at java.net.URLClassLoader.findClass(libgcj.so.7rh)
at java.lang.ClassLoader.loadClass(libgcj.so.7rh)
at java.lang.ClassLoader.loadClass(libgcj.so.7rh)
at gnu.java.lang.MainThread.run(libgcj.so.7rh) -
24. Re: How to cluster two Content Engine server to share CPU load
wdfink Jun 19, 2012 1:53 AM (in response to ksermas)Please do your homework and check that the jar files exists and you have the correct java command.
I suppose there is something wrong with your jar's or the classpath.
-
25. Re: How to cluster two Content Engine server to share CPU load
ksermas Sep 6, 2012 9:50 AM (in response to ksermas)After all that I found out that multicasting indeed was NOT enabled on our network I'm sure once it's enabled this will work.
Thanks very much Wolf-Dieter.