HOWTO Step by step clustering ( JBoss 4.0.1 - Linux - Oracle
schaffal Feb 28, 2005 11:29 AMHi,
Making cluster is easy with JBoss, that's right... When it does work out-of-the box.
Of course all the info needed is there, but scattered in many forums and Wiki pages. Moreover, infos are mostly for 'older' releases since 3.2.x are seem to be the most used. Unfortunately, that can lead to confusion sometimes.
Well, anyway, here's a step by step 'how-to' I now use to make my simple hello examples work. Maybe that'll be helpfull to some, maybe not.
Goal : Quick steps I followed to have a running cluster with JBoss. (Oracle DB and Linux multi-homed machines) JBoss release : 4.0.1sp1 JVM : Sun's 1.4.2_07-b05 Machine1 : Linux (SUSE Pro8 - 2.4.18-64GB-SMP) ; multi-homed (IP1 used on eth0) Machine2 : Linux (SUSE SLES9 - 2.6.5-7.108-smp) ; multi-homed (IP2 used on eth2) Note : ******************************************* * PART 1 : Make Oracle the Default database ******************************************* Prerequisite : Oracle and listener installed on Machine1 : - sid is MYSID ; - uses IP1 and default port (1521); - valid user/password is LOGIN/PASSWORD - untarred brand new JBoss delivery 4.0.1sp1 (from sourceforge) Note : all path given relative to <JBOSS_HOME> ---------------------- 1.0 in conf/run.conf : ---------------------- - set JAVA_HOME correctly to not trust env variables ( especially if you want to use a special jvm ) ---------------------------------------------- 1.1 in file server/all/conf/login-config.xml : ---------------------------------------------- From : <module-option name = "dsJndiName">java:/DefaultDS</module-option> To : <module-option name = "dsJndiName">java:/OracleDS</module-option> -------------------------------------------------- 1.2 in server/all/conf/standardjbosscmp-jdbc.xml : -------------------------------------------------- From : <defaults> <datasource>java:/DefaultDS</datasource> <!-- optional since 4.0 <datasource-mapping>Hypersonic SQL</datasource-mapping> --> To : <defaults> <datasource>java:/OracleDS</datasource> <datasource-mapping>Oracle9i</datasource-mapping> --- From : <type-mapping> <name>Oracle9i</name> ... <mapping> <java-type>java.lang.Short</java-type> <jdbc-type>NUMERIC</jdbc-type> <!-- <jdbc-type>SMALLINT</jdbc-type> this resulted in a ClassCastException --> <sql-type>NUMBER(5)</sql-type> To : <type-mapping> <name>Oracle9i</name> ... <mapping> <java-type>java.lang.Short</java-type> <jdbc-type>SMALLINT</jdbc-type> <!-- NUMERIC does not Work for us --> <!-- <jdbc-type>SMALLINT</jdbc-type> this resulted in a ClassCastException --> <sql-type>NUMBER(5)</sql-type> --------------------------------- 1.3 Change the service provider : --------------------------------- rm server/all/deploy-hasingleton/jms/hsqldb-jdbc2-service.xml cp docs/examples/jms/oracle-jdbc2-service.xml server/all/deploy-hasingleton/jms/ cp docs/examples/jca/oracle-ds.xml server/all/deploy/ mv server/all/deploy-hasingleton/jms/hsqldb-jdbc-state-service.xml server/all/deploy-hasingleton/jms/oracle-jdbc-state-service.xml ---------------------------------------- 1.4 in server/all/deploy/oracle-ds.xml : ---------------------------------------- From : <connection-url>jdbc:oracle:thin:@youroraclehost:1521:yoursid</connection-url> To : <connection-url>jdbc:oracle:thin:@IP1:1521:MYSID</connection-url> --- From : <user-name>x</user-name> <password>y</password> To : <user-name>LOGIN</user-name> <password>PASSWORD</password> ----------------------------------------------------------------------- 1.5 in server/all/deploy-hasingleton/jms/oracle-jdbc-state-service.xml ----------------------------------------------------------------------- From : <depends optional-attribute-name="ConnectionManager">jboss.jca:service=DataSourceBinding,name=DefaultDS</depends> To : <depends optional-attribute-name="ConnectionManager">jboss.jca:service=DataSourceBinding,name=OracleDS</depends> ----------------------------- 1.6 get the classes12 library ----------------------------- From the oracle server, get classes12.zip ( file is in [ORACLE_HOME]/product/[ORACLE_VERSION]/jdbc/lib/classes12.zip ) Copy file to server/all/lib/classes12.jar ( Note that if oci is desired instead of thin, it's a whole different story ) ******************************************************* * PART 2 : File Modifications and others for clustering ******************************************************* ------------------------- 2.1 Avoid IPv6 problems : ------------------------- See http://www.jboss.org/index.html?module=bb&op=viewtopic&t=49207 and http://portal.suse.de/sdb/en/2003/10/90_mozilla_ipv6.html - in file /etc/modules.conf on Machine1 (SUSE 8 Distrib) : From : alias net-pf-10 ipv6 #alias net-pf-10 off To : #alias net-pf-10 ipv6 alias net-pf-10 off - in file /etc/modprobe.conf on Machine2 (SUSE 9.1 distrib) : From : alias net-pf-10 ipv6 To : alias net-pf-10 ipv6 install ipv6 /bin/true - For SUSE 9.0 distribs (c.f. SUSE web site), in file /etc/modprobe.conf : From : alias net-pf-10 ipv6 #alias net-pf-10 off To : #alias net-pf-10 ipv6 alias net-pf-10 off Note : /etc/sysconfig/scripts/SuSEfirewall2-custom can also be modified to allow incoming multicast packets. -------------------------- 2.2 add multicast routes : -------------------------- On Machine1 : route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0 On Machine2 : route add -net 224.0.0.0 netmask 240.0.0.0 dev eth2 Note1 : do not mix-up network devices Note2 : The netmask is to allow other default multicast IPs (228.1.2.3, 230.0.0.4, 230.1.2.3 and 224.1.9.1) Note 3 : to check routes : netstat -grn On Machine1 the important line is : 224.0.0.0 0.0.0.0 240.0.0.0 U 40 0 0 eth0 ---------------------------------------- 2.3 in server/all/conf/standardjboss.xml ---------------------------------------- From : <container-configuration> <container-name>Clustered Stateless SessionBean</container-name> ... <interceptor transaction="Container">org.jboss.ejb.plugins.ServiceEndpointInterceptor</interceptor> ... <interceptor transaction="Bean">org.jboss.ejb.plugins.ServiceEndpointInterceptor</interceptor> To : <container-configuration> <container-name>Clustered Stateless SessionBean</container-name> ... <interceptor transaction="Container">org.jboss.webservice.server.ServiceEndpointInterceptor</interceptor> ... <interceptor transaction="Bean">org.jboss.webservice.server.ServiceEndpointInterceptor</interceptor> ------------------------------------------------ 2.4 in server/all/deploy/jms/hajndi-jms-ds.xml : ------------------------------------------------ From : <attribute name="Properties"> java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces java.naming.provider.url=localhost:1100 </attribute> To : <attribute name="Properties"> java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces java.naming.provider.url=<IP1>:1100,<IP2>:1100 </attribute> Note, you can also dos2unix the file since some eof are dos formated in this file. ---------------------------------------------- 2.5 in server/all/deploy/cluster-service.xml : ---------------------------------------------- From : <UDP mcast_addr="228.1.2.3" mcast_port="45566" ip_ttl="32" ip_mcast="true" mcast_send_buf_size="800000" mcast_recv_buf_size="150000" ucast_send_buf_size="800000" ucast_recv_buf_size="150000" loopback="false" /> To : <UDP mcast_addr="228.1.2.3" mcast_port="45566" ip_ttl="32" ip_mcast="true" mcast_send_buf_size="800000" mcast_recv_buf_size="150000" ucast_send_buf_size="800000" ucast_recv_buf_size="150000" loopback="false" bind_addr="<IP1 on Machine1 ; IP2 on Machine2>" /> Note : that is the ONLY file to change from one machine to the other ( use rsync to copy modifs for second install to avoid human errors.) ------------------------ 2.5 Start command used : ------------------------ Machine1 : /opt/jboss/bin/run.sh -c all --host=IP1 Machine2 : /opt/jboss/bin/run.sh -c all --host=IP2 ****************************** * PART 3 : Some Specific usage ****************************** --------------------------- 3.1 A singleton scheduler : --------------------------- Make use of the jboss:service=ScheduleManager defined in server/all/deploy/schedule-manager-service.xml Example : SchedulableMBeanExample ------------------------------------------------------------- 3.2 Access to a queue using HA-JNDI (from inside the cluster) ------------------------------------------------------------- Solution1 : ~~~~~~~~~~~ Set 'hardcoded' properties for the context : Properties p = new Properties(); p.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory"); p.put(Context.URL_PKG_PREFIXES, "jboss.naming:org.jnp.interfaces"); p.put(Context.PROVIDER_URL, "IP1:1100,IP2:1100"); // HA-JNDI port. InitialContext context = new InitialContext(p); Queue queue = (Queue) context.lookup(queueName); pros/cons : - necessity to modify code if hardcoded or to reload properties in case of a change in IP used. - information is already in a property file (hajndi-jms-ds.xml) - complex for n-nodes cluster + easy to try/test => Solution1 should be avoided when lookup is done from inside the cluster Solution2 : ~~~~~~~~~~~ Use the partition name with code : Properties p = new Properties(); p.put("jnp.partitionName", "DefaultPartition"); InitialContext context = new InitialContext(p); Queue queue = (Queue) context.lookup(queueName); Pro/cons : - may be non-applicable on some infrastructures (forbiden multicast msgs) Solution3 ~~~~~~~~~ Use only propertie files : This solution would be the preffered solution, just using : InitialContext context = new InitialContext(); Queue queue = (Queue) context.lookup(queueName); <<Solution TBD>> Pros/cons : - may be non-applicable on some infrastructures (forbiden multicast msgs) + No need to change initial code : one person can be responsible of the code, while another person is in charge of the deployment ( no access to source code or impossibility to build ! ) => preffered solution when lookup is done from inside the cluster.
Btw, if someone knows how to deal with Solution3, that'll be great.
I had this problem posted some time ago now : http://www.jboss.org/index.html?module=bb&op=viewtopic&t=60106 but I was not precise on the desired way to address the issue. Solution 1 above is often found ; Solution2 I did not found on the forum or Wiki ( I do not say it is not there, I just did not find it) ; Solution3 is the one desired : no change to the code, only use configuration files.
Good day,
Alex.