3 Replies Latest reply on Jan 2, 2012 5:10 AM by Radoslav Husar

    JBoss 6.1 cluster with TCP

    konrad biel Newbie

      I've read:

      http://community.jboss.org/thread/164797 

      http://community.jboss.org/thread/174934?wwparam=1324543958

       

      and do everythink, but my jboss instances don't see each other.

       

      The same on each server:

       

      <stack name="tcp"

              

              <config>

                  <TCP

                       singleton_name="tcp"

                       bind_port="${jboss.jgroups.tcp.tcp_port:7600}"

                       tcp_nodelay="true"

                       loopback="true"

                       recv_buf_size="20000000"

                       send_buf_size="640000"

                       discard_incompatible_packets="true"

                       max_bundle_size="64000"

                       max_bundle_timeout="30"

                       use_incoming_packet_handler="true"

                       enable_bundling="false"

                       use_send_queues="false"

                       sock_conn_timeout="300"

                       skip_suspected_members="true"

                       timer.num_threads="12"

                       enable_diagnostics="${jboss.jgroups.enable_diagnostics:true}"

                       diagnostics_addr="${jboss.jgroups.diagnostics_addr:224.0.75.75}"

                       diagnostics_port="${jboss.jgroups.diagnostics_port:7500}"

         

                         thread_pool.enabled="true"

                         thread_pool.min_threads="20"

                         thread_pool.max_threads="200"

                         thread_pool.keep_alive_time="5000"

                         thread_pool.queue_enabled="true"

                         thread_pool.queue_max_size="1000"

                         thread_pool.rejection_policy="discard"

             

                       oob_thread_pool.enabled="true"

                         oob_thread_pool.min_threads="20"

                         oob_thread_pool.max_threads="200"

                         oob_thread_pool.keep_alive_time="1000"

                         oob_thread_pool.queue_enabled="false"

                         oob_thread_pool.rejection_policy="discard"/>

                    <!-- Alternative 1: multicast-based automatic discovery. -->  

                  <!--<MPING timeout="3000"

                         num_initial_members="3"

                         mcast_addr="${jboss.jgroups.tcp.mping_mcast_addr,jboss.partition.udpGroup:230.11.11.11}"

                         mcast_port="${jboss.jgroups.tcp.mping_mcast_port:45700}"

                         ip_ttl="${jgroups.udp.ip_ttl:2}"/>           

                   Alternative 2: non multicast-based replacement for MPING. Requires a static configuration

                       of *all* possible cluster members.-->

                  <TCPPING timeout="3000"

                           initial_hosts="${jboss.jgroups.tcp.tcpping.initial_hosts:server1[7600],server2[7600]}"

                           port_range="1"

                           num_initial_members="3"/>

                  

                  <MERGE2 max_interval="100000" min_interval="20000"/>

                  <FD_SOCK start_port="${jboss.jgroups.tcp.fd_sock_port:57600}"/>

                  <FD timeout="6000" max_tries="5"/>

                  <VERIFY_SUSPECT timeout="1500"/>

                  <BARRIER/>

                  <pbcast.NAKACK use_mcast_xmit="false" gc_lag="0"

                                 retransmit_timeout="300,600,1200,2400,4800"

                                 discard_delivered_msgs="true"/>

                  <UNICAST timeout="300,600,1200,2400,3600"/>

                  <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"

                                 max_bytes="400000"/>           

                  <VIEW_SYNC avg_send_interval="10000"/>              

                  <pbcast.GMS print_local_addr="true" join_timeout="3000"

                              view_bundling="true"

                              view_ack_collection_timeout="5000"

                              resume_task_timeout="7500"/>

                  <UFC max_credits="2000000" ignore_synchronous_response="true"/>

                  <MFC max_credits="2000000" ignore_synchronous_response="true"/>

                  <FRAG2 frag_size="60000"/>

                  <pbcast.STREAMING_STATE_TRANSFER/>

                  <!--pbcast.STATE_TRANSFER/-->

                  <pbcast.FLUSH timeout="0" start_flush_timeout="10000"/>

              </config>

          </stack>

       

      and I use to run:

       

      ./run.sh -c all -b server1 -g newPart -Djboss.default.jgroups.stack=tcp  -Djboss.messaging.ServerPeerID=2 -Djboss.service.binding.set=ports-01 -Djgroups.bind_addr=server1

       

      and second:

      ./run.sh -c all -b server2 -g newPart -Djboss.default.jgroups.stack=tcp -Djboss.messaging.ServerPeerID=3 -Djboss.service.binding.set=ports-01 -Djgroups.bind_addr=server2

       

      I also have i consol log:

       

      14:27:09,044 INFO  [ClusterPartition] Initializing partition newPart

      14:27:09,248 INFO  [TransactionManagerFactory] Using a batchMode transaction manager

      14:27:10,366 INFO  [JGroupsTransport] Starting JGroups Channel

      14:27:10,422 INFO  [JChannel] JGroups version: 2.11.0.GA

      14:27:10,945 WARN  [Configurator] TCP property skip_suspected_members was deprecated and is ignored

      14:27:10,946 WARN  [Configurator] TCP property use_incoming_packet_handler was deprecated and is ignored

      14:27:10,947 WARN  [Configurator] TCP property timer.num_threads was deprecated and is ignored

       

      What I've done wrong?

        • 1. Re: JBoss 6.1 cluster with TCP
          Radoslav Husar Master

          You need to pass correct property value for: jboss.jgroups.tcp.tcpping.initial_hosts

           

          (You see, you are shifting ports, but you are discovering members at two ports 7600 both open at the same host, which is not possible).

          1 of 1 people found this helpful
          • 2. Re: JBoss 6.1 cluster with TCP
            konrad biel Newbie

            Yes, I have to change this, but I think it is not enough. Cluster don't want to start at jboss 6.0 or 6.1. Instances don't see each other. I tried at jboss 5.1 and everythink was working.

             

            Part log from start jbosss 6.0:

             

            For me its strange:

            11:30:43,998 INFO  [STDOUT] GMS: address=server1:1199, cluster=newPart-HAPartition, physical address=server1:7700

            11:30:43,999 INFO  [STDOUT] -------------------------------------------------------------------

            11:30:47,123 INFO  [JGroupsTransport] Received new cluster view: [server1:1199|0] [server1:1199]

            11:30:47,134 INFO  [JGroupsTransport] Cache local address is server1:1199, physical addresses are [server1:7700]

             

            I set 7600 and 7601...

             

            Full log:

             

            bash-3.00$ ./run.sh -c all -b server1 -g newPart -Djboss.default.jgroups.stack=tcp  -Djboss.messaging.ServerPeerID=2 -Djboss.service.binding.set=ports-01 -Djgroups.bind_addr=server1

            =========================================================================

             

              JBoss Bootstrap Environment

             

              JBOSS_HOME: /export/home/vo/jboss/jboss-6.0.0.Final

             

              JAVA: /export/home/vo/Instal/jdk1.6.0_21/bin/java

             

              JAVA_OPTS: -server -Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dprogram.name=run.sh -Djava.library.path=/export/home/vo/jboss/jboss-6.0.0.Final/bin/native/lib

             

              CLASSPATH: /export/home/vo/jboss/jboss-6.0.0.Final/bin/run.jar

             

            =========================================================================

             

            11:29:15,167 INFO  [AbstractJBossASServerBase] Server Configuration:

             

                    JBOSS_HOME URL: file:/export/home/vo/jboss/jboss-6.0.0.Final/

                    Bootstrap: $JBOSS_HOME/server/all/conf/bootstrap.xml

                    Common Base: $JBOSS_HOME/common/

                    Common Library: $JBOSS_HOME/common/lib/

                    Server Name: all

                    Server Base: $JBOSS_HOME/server/

                    Server Library: $JBOSS_HOME/server/all/lib/

                    Server Config: $JBOSS_HOME/server/all/conf/

                    Server Home: $JBOSS_HOME/server/all/

                    Server Data: $JBOSS_HOME/server/all/data/

                    Server Log: $JBOSS_HOME/server/all/log/

                    Server Temp: $JBOSS_HOME/server/all/tmp/

             

            11:29:15,183 INFO  [AbstractServer] Starting: JBossAS [6.0.0.Final "Neo"]

            11:29:23,895 INFO  [ServerInfo] Java version: 1.6.0_21,Sun Microsystems Inc.

            11:29:23,896 INFO  [ServerInfo] Java Runtime: Java(TM) SE Runtime Environment (build 1.6.0_21-b06)

            11:29:23,896 INFO  [ServerInfo] Java VM: Java HotSpot(TM) Server VM 17.0-b16,Sun Microsystems Inc.

            11:29:23,897 INFO  [ServerInfo] OS-System: SunOS 5.10,sparc

            11:29:23,899 INFO  [ServerInfo] VM arguments: -Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dprogram.name=run.sh -Djava.library.path=/export/home/vo/jboss/jboss-6.0.0.Final/bin/native/lib:/export/home/vo/Apache2_fs/lib:/export/home/vo/Apache2_fs/lib/iconv/lib:/usr/lib:/pkg/lib:/usr/sfw/lib: -Djava.endorsed.dirs=/export/home/vo/jboss/jboss-6.0.0.Final/lib/endorsed

            11:29:24,098 INFO  [JMXKernel] Legacy JMX core initialized

            11:29:50,770 INFO  [AbstractServerConfig] JBoss Web Services - Stack CXF Server 3.4.1.GA

            11:29:54,745 INFO  [JSFImplManagementDeployer] Initialized 3 JSF configurations: [Mojarra-1.2, MyFaces-2.0, Mojarra-2.0]

            11:30:10,021 WARNING [FileConfigurationParser] AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal

            11:30:23,442 INFO  [SnmpAgentService] SNMP agent going active

            11:30:26,334 WARNING [FileConfigurationParser] AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal

            11:30:27,067 WARN  [config] jacorb.home unset! Will use '.'

            11:30:27,068 WARN  [config] File ./jacorb.properties for configuration jacorb not found

            11:30:27,072 INFO  [config] configuration jacorb loaded from classpath

            11:30:27,334 INFO  [orb] Property "jacorb.hashtable_class" is set to: java.util.Hashtable

            11:30:27,413 WARN  [codeset] Warning - unknown codeset (ASCII) - defaulting to ISO-8859-1

            11:30:27,443 WARN  [config] jacorb.home unset! Will use '.'

            11:30:27,444 WARN  [config] File ./jacorb.properties for configuration jacorb not found

            11:30:27,448 INFO  [config] configuration jacorb loaded from classpath

            11:30:27,450 INFO  [singleton] created ORBSingleton

            11:30:27,841 INFO  [interceptors] InterceptorManager started with 2 Server Interceptors, 2 Client Interceptors and 4 IOR Interceptors

            11:30:28,327 INFO  [orb] ORB run

            11:30:28,954 INFO  [JMXConnector] starting JMXConnector on host server1:1190

            11:30:29,673 INFO  [MailService] Mail Service bound to java:/Mail

            11:30:29,739 INFO  [poa] oid:

            72 6F 6F 74                                                 root

            object is activated

            11:30:29,960 INFO  [CorbaNamingService] CORBA Naming Started

            11:30:32,408 INFO  [DefaultCacheContainerRegistry] Binding hibernate cache container to java:CacheManager/entity

            11:30:33,793 INFO  [UnifiedInvokerHA] Service name is jboss:service=invoker,type=unifiedha

            11:30:36,385 INFO  [HornetQServerImpl] live server is starting..

            11:30:36,693 INFO  [JournalStorageManager] Using NIO Journal

            11:30:36,789 WARNING [HornetQServerImpl] Security risk! It has been detected that the cluster admin user and password have not been changed from the installation default. Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.

            11:30:40,082 INFO  [NettyAcceptor] Started Netty Acceptor version 3.2.1.Final-r2319 server1:5545 for CORE protocol

            11:30:40,119 INFO  [NettyAcceptor] Started Netty Acceptor version 3.2.1.Final-r2319 server1:5555 for CORE protocol

            11:30:40,141 INFO  [HornetQServerImpl] HornetQ Server version 2.1.2.Final (Colmeia, 120) started

            11:30:40,424 INFO  [WebService] Using RMI server codebase: http://server1:8183/

            11:30:40,512 INFO  [ClusterPartition] Initializing partition newPart

            11:30:40,766 INFO  [TransactionManagerFactory] Using a batchMode transaction manager

            11:30:42,067 INFO  [JGroupsTransport] Starting JGroups Channel

            11:30:42,133 INFO  [JChannel] JGroups version: 2.11.0.GA

            11:30:42,767 WARN  [Configurator] TCP property skip_suspected_members was deprecated and is ignored

            11:30:42,768 WARN  [Configurator] TCP property use_incoming_packet_handler was deprecated and is ignored

            11:30:42,769 WARN  [Configurator] TCP property timer.num_threads was deprecated and is ignored

            11:30:43,996 INFO  [STDOUT]

            11:30:43,997 INFO  [STDOUT] -------------------------------------------------------------------

            11:30:43,998 INFO  [STDOUT] GMS: address=server1:1199, cluster=newPart-HAPartition, physical address=server1:7700

            11:30:43,999 INFO  [STDOUT] -------------------------------------------------------------------

            11:30:47,123 INFO  [JGroupsTransport] Received new cluster view: [server1:1199|0] [server1:1199]

            11:30:47,134 INFO  [JGroupsTransport] Cache local address is server1:1199, physical addresses are [server1:7700]

            11:30:47,136 INFO  [GlobalComponentRegistry] Infinispan version: Infinispan 'Ursus' 4.2.0.FINAL

            11:30:47,304 INFO  [ComponentsJmxRegistration] Could not register object with name: org.infinispan:type=Cache,name="distributed-state(repl_sync)",manager="ha-partition",component=Cache

            11:30:47,307 INFO  [CacheJmxRegistration] MBeans were successfully registered to the platform mbean server.

            11:30:47,308 INFO  [ComponentRegistry] Infinispan version: Infinispan 'Ursus' 4.2.0.FINAL

            11:30:47,325 INFO  [DefaultCacheContainerFactory] Started "distributed-state" cache from "ha-partition" container

            11:30:47,395 INFO  [newPart] Number of cluster members: 1

            11:30:47,398 INFO  [newPart] Fetching initial service state (will wait for 30000 milliseconds for each service):

            11:30:48,200 INFO  [HANamingService] Started HAJNDI bootstrap; jnpPort=1200, backlog=50, bindAddress=/server1

            11:30:48,246 INFO  [DetachedHANamingService$AutomaticDiscovery] Listening on /server1:1102, group=230.0.0.4, HA-JNDI address=server1:1200

            11:30:48,260 INFO  [TransactionManagerFactory] Using a batchMode transaction manager

            11:30:48,333 INFO  [ComponentsJmxRegistration] Could not register object with name: org.infinispan:type=Cache,name="distributed-tree(repl_sync)",manager="ha-partition",component=Cache

            11:30:48,336 INFO  [CacheJmxRegistration] MBeans were successfully registered to the platform mbean server.

            11:30:48,336 INFO  [ComponentRegistry] Infinispan version: Infinispan 'Ursus' 4.2.0.FINAL

            11:30:48,337 INFO  [DefaultCacheContainerFactory] Started "distributed-tree" cache from "ha-partition" container

            11:30:52,137 INFO  [jbossatx] ARJUNA-32010 JBossTS Recovery Service (tag: JBOSSTS_4_14_0_Final) - JBoss Inc.

            11:30:52,162 INFO  [arjuna] ARJUNA-12324 Start RecoveryActivators

            11:30:52,251 INFO  [arjuna] ARJUNA-12296 ExpiredEntryMonitor running at Fri, 23 Dec 2011 11:30:52

            11:30:52,569 INFO  [arjuna] ARJUNA-12310 Recovery manager listening on endpoint server1:4812

            11:30:52,570 INFO  [arjuna] ARJUNA-12344 RecoveryManagerImple is ready on port 4812

            11:30:52,575 INFO  [jbossatx] ARJUNA-32013 Starting transaction recovery manager

            11:30:52,626 INFO  [arjuna] ARJUNA-12163 Starting service com.arjuna.ats.arjuna.recovery.ActionStatusService on port 4813

            11:30:52,664 INFO  [arjuna] ARJUNA-12337 TransactionStatusManagerItem host: server1 port: 4813

            11:30:52,682 INFO  [arjuna] ARJUNA-12170 TransactionStatusManager started on port 4813 and host server1 with service com.arjuna.ats.arjuna.recovery.ActionStatusService

            11:30:52,832 INFO  [jbossatx] ARJUNA-32017 JBossTS Transaction Service (JTA version - tag: JBOSSTS_4_14_0_Final) - JBoss Inc.

            11:30:53,013 INFO  [arjuna] ARJUNA-12202 registering bean jboss.jta:type=ObjectStore.

            11:30:53,942 INFO  [AprLifecycleListener] The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /export/home/vo/jboss/jboss-6.0.0.Final/bin/native/lib:/export/home/vo/Apache2_fs/lib:/export/home/vo/Apache2_fs/lib/iconv/lib:/usr/lib:/pkg/lib:/usr/sfw/lib:

            11:30:54,632 INFO  [HAModClusterService$ClusteredModClusterService] Initializing mod_cluster 1.1.0.Final

            11:30:54,696 INFO  [TomcatDeployment] deploy, ctxPath=/invoker

            11:30:56,272 INFO  [RARDeployment] Required license terms exist, view vfs:/export/home/vo/jboss/jboss-6.0.0.Final/server/all/deploy/jboss-local-jdbc.rar/META-INF/ra.xml

            11:30:56,337 INFO  [RARDeployment] Required license terms exist, view vfs:/export/home/vo/jboss/jboss-6.0.0.Final/server/all/deploy/jboss-xa-jdbc.rar/META-INF/ra.xml

            11:30:56,389 INFO  [RARDeployment] Required license terms exist, view vfs:/export/home/vo/jboss/jboss-6.0.0.Final/server/all/deploy/jms-ra.rar/META-INF/ra.xml

            11:30:56,463 INFO  [HornetQResourceAdapter] HornetQ resource adaptor started

            11:30:56,504 INFO  [RARDeployment] Required license terms exist, view vfs:/export/home/vo/jboss/jboss-6.0.0.Final/server/all/deploy/mail-ra.rar/META-INF/ra.xml

            11:30:56,568 INFO  [RARDeployment] Required license terms exist, view vfs:/export/home/vo/jboss/jboss-6.0.0.Final/server/all/deploy/quartz-ra.rar/META-INF/ra.xml

            11:30:56,878 INFO  [SimpleThreadPool] Job execution threads will use class loader of thread: Thread-2

            11:30:57,076 INFO  [SchedulerSignalerImpl] Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl

            11:30:57,077 INFO  [QuartzScheduler] Quartz Scheduler v.1.8.3 created.

            11:30:57,089 INFO  [RAMJobStore] RAMJobStore initialized.

            11:30:57,098 INFO  [QuartzScheduler] Scheduler meta-data: Quartz Scheduler (v1.8.3) 'JBossQuartzScheduler' with instanceId 'NON_CLUSTERED'

              Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.

              NOT STARTED.

              Currently in standby mode.

              Number of jobs executed: 0

              Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.

              Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.

             

            11:30:57,099 INFO  [StdSchedulerFactory] Quartz scheduler 'JBossQuartzScheduler' initialized from an externally opened InputStream.

            11:30:57,100 INFO  [StdSchedulerFactory] Quartz scheduler version: 1.8.3

            11:30:57,101 INFO  [QuartzScheduler] Scheduler JBossQuartzScheduler_$_NON_CLUSTERED started.

            11:30:58,847 INFO  [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=DefaultDS' to JNDI name 'java:DefaultDS'

            11:30:59,692 INFO  [TomcatDeployment] deploy, ctxPath=/juddi

            11:30:59,784 INFO  [RegistryServlet] Loading jUDDI configuration.

            11:30:59,811 INFO  [RegistryServlet] Resources loaded from: /WEB-INF/juddi.properties

            11:30:59,812 INFO  [RegistryServlet] Initializing jUDDI components.

            11:31:00,742 INFO  [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:service=ConnectionFactoryBinding,name=JmsXA' to JNDI name 'java:JmsXA'

            11:31:01,391 INFO  [xnio] XNIO Version 2.1.0.CR2

            11:31:01,438 INFO  [nio] XNIO NIO Implementation Version 2.1.0.CR2

            11:31:02,287 INFO  [remoting] JBoss Remoting version 3.1.0.Beta2

            11:31:02,683 INFO  [TomcatDeployment] deploy, ctxPath=/

            11:31:02,854 INFO  [service] Removing bootstrap log handlers

            11:31:03,534 INFO  [org.apache.coyote.http11.Http11Protocol] Starting Coyote HTTP/1.1 on http-server1-8180

            11:31:03,548 INFO  [org.apache.coyote.ajp.AjpProtocol] Starting Coyote AJP/1.3 on ajp-server1-8109

            11:31:03,562 INFO  [org.jboss.bootstrap.impl.base.server.AbstractServer] JBossAS [6.0.0.Final "Neo"] Started in 1m:48s:356ms

            • 3. Re: JBoss 6.1 cluster with TCP
              Radoslav Husar Master

              Hm, if it works in AS5 and not in AS6 it might as well be a regression or just a change in how stuff is configured.

               

              Nevertheless, I dont like the concept of port shifting (-Djboss.service.binding.set=ports-01) are you still using it? If yes can you try to bind to different hosts (ie. localhost and your ip adress on the network for instance)?

               

              Ideally you could switch to AS7 ;-)