3 Replies Latest reply on Dec 13, 2006 4:04 PM by Brian Stansberry

    Buddy Replication Issues

    Vishal K Newbie

      Hi Friends,
      I am using jboss 4.0.5GA
      I have the following properties se for my node to enable buddy replication.

      <attribute name="BuddyReplicationConfig">
       <config>
       <buddyReplicationEnabled>true</buddyReplicationEnabled>
       <buddyLocatorClass>org.jboss.cache.buddyreplication.NextMemberBuddyLocator</buddyLocatorClass>
       <buddyLocatorProperties>
       numBuddies = 1
       ignoreColocatedBuddies = true
       </buddyLocatorProperties>
      
       <buddyPoolName>default</buddyPoolName>
       <buddyCommunicationTimeout>2000</buddyCommunicationTimeout>
      
       <autoDataGravitation>false</autoDataGravitation>
       <dataGravitationRemoveOnFind>true</dataGravitationRemoveOnFind>
       <dataGravitationSearchBackupTrees>true</dataGravitationSearchBackupTrees>
       </config>
      </attribute>


      But when I start two nodes and shutdown one node, the application crashes. Failover does not work.I am not able to understand why.

      If I disable buddyclustering and let the session replicate across all nodes
      then application clustering works fine for failover.

      This is my jboss console log:-

      -------------------------------------------------------
      15:48:41,914 INFO [TreeCache] TreeCache local address is www:7811
      15:48:41,914 INFO [BuddyManager] New buddy group: BuddyGroup: (dataOwner: www:7811, groupName: www_7811, buddies: [])
      15:48:41,914 INFO [TreeCache] parseConfig(): PojoCacheConfig is empty
      15:48:41,914 INFO [TreeCache] viewAccepted(): [www:7810|5] [www:7810, www:7811]
      15:48:41,960 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying wi
      colocated buddies as well.
      15:48:41,960 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying ag
      n, ignoring buddy pool hints.
      15:48:41,960 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying wi
      colocated buddies as well.
      15:48:41,960 INFO [BuddyManager] Assigning new buddies to buddy group [www_7811]. New buddies are [www:7810]
      15:48:41,960 INFO [TreeCache] locking the subtree at / to transfer state
      15:48:41,992 INFO [StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)
      15:48:42,164 INFO [BuddyManager] Assigning self to buddy group BuddyGroup: (dataOwner: www:7810, groupName: www_7810, buddies:
      ww:7811])
      15:48:42,164 INFO [TreeCache] received the state (size=4096 bytes)
      15:48:42,179 INFO [BuddyManager] New buddy group: BuddyGroup: (dataOwner: www:7811, groupName: www_7811, buddies: [www:7810])
      15:48:42,804 INFO [DefaultPartition] Initializing
      15:48:42,882 INFO [STDOUT]

      Need help
      Thanks
      Vishal

        • 1. Re: Buddy Replication Issues
          Brian Stansberry Master

          I'm not sure how your log posting relates to what you described. You describe a shutdown problem with one of two caches, but your log shows a node joining the group, not shutting down. And it doesn't show any apparent problem.

          Please better describe what is happening.

          • 2. Re: Buddy Replication Issues
            Vishal K Newbie

            Thanks a lot for your response. I have three instances off jboss running on the same machine

            For my Node 1 ;; -

            <buddyReplicationEnabled>true</buddyReplicationEnabled>
            =====================================================================
            -------------------------------------------------------
            GMS: address is www:7810
            -------------------------------------------------------
            12:12:13,937 INFO [TreeCache] TreeCache local address is www:7810
            12:12:13,937 INFO [BuddyManager] New buddy group: BuddyGroup: (dataOwner: www:7810, groupName: www_7810, buddies: [])
            12:12:13,952 INFO [TreeCache] parseConfig(): PojoCacheConfig is empty
            12:12:13,952 INFO [TreeCache] viewAccepted(): [www:7811|4] [www:7811, www:7812, www:7810]
            12:12:13,983 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying wi
            colocated buddies as well.
            12:12:13,983 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying ag
            n, ignoring buddy pool hints.
            12:12:13,983 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying wi
            colocated buddies as well.
            12:12:13,983 INFO [BuddyManager] Assigning new buddies to buddy group [www_7810]. New buddies are [www:7811]
            12:12:13,983 INFO [TreeCache] locking the subtree at / to transfer state
            12:12:13,983 INFO [StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)
            12:12:14,155 INFO [BuddyManager] New buddy group: BuddyGroup: (dataOwner: www:7810, groupName: www_7810, buddies: [www:7811])
            12:12:15,062 INFO [DefaultPartition] Initializing
            12:12:15,515 INFO [STDOUT]
            -------------------------------------------------------
            GMS: address is www:7800 (additional data: 16 bytes)
            -------------------------------------------------------
            12:12:15,577 INFO [DefaultPartition] Number of cluster members: 3
            12:12:15,577 INFO [DefaultPartition] Other members: 2
            12:12:15,577 INFO [DefaultPartition] Fetching state (will wait for 30000 milliseconds):
            12:12:15,671 INFO [DefaultPartition] state was retrieved successfully (in 94 milliseconds)
            12:12:15,812 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1100, backlog=50, bindAddress=/0.0.0.0
            12:12:15,812 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /0.0.0.0:1102, group=230.0.0.4, HA-JNDI address=16.
            0.8.76:1100
            12:12:17,030 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436
            E746578744578743A312E3000000000000200000000000000E8000102000000000C31362E3135302E382E3736000DC80000000000114A426F73732F4E616D696
            72F726F6F74000000000000050000000000000008000000004A414300000000010000001C0000000000010001000000010501000100010109000000010501000
            000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C31362E3135302E382E3736000DC90040000000000
            000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000
            0010000002000000000000000020000002000000004000000000000001F0000000400000003]
            12:12:17,155 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F70
            5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000E8000102000000000C31362E3135302E382E3736000DC800000000
            144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100
            0001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C31362E3135302E38
            3736000DC90040000000000000000000000010040100080606678102010101000000000000000000000000000000000000000000000020000000040000000000
            001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
            12:12:18,796 INFO [Embedded] Catalina naming disabled
            12:12:18,921 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
            12:12:18,921 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
            12:12:19,671 INFO [Http11BaseProtocol] Initializing Coyote HTTP/1.1 on http-0.0.0.0-8080
            12:12:19,671 INFO [Catalina] Initialization processed in 750 ms



            -------------------------------------------------------------------------------------------------------------------------------------
            -------------------------------------------------------------------------------------------------------------------------------------

            Now Node 2::-
            <buddyReplicationEnabled>false</buddyReplicationEnabled>
            =====================================================================

            class org.jboss.cache.interceptors.CallInterceptor
            class org.jboss.cache.interceptors.PessimisticLockInterceptor
            class org.jboss.cache.interceptors.DataGravitatorInterceptor
            class org.jboss.cache.interceptors.UnlockInterceptor
            class org.jboss.cache.interceptors.ReplicationInterceptor
            class org.jboss.cache.interceptors.TxInterceptor
            class org.jboss.cache.interceptors.CacheMgmtInterceptor
            11:52:21,203 INFO [STDOUT]
            -------------------------------------------------------
            GMS: address is www:7811
            -------------------------------------------------------
            11:52:26,796 INFO [TreeCache] TreeCache local address is www:7811
            11:52:26,796 INFO [BuddyManager] New buddy group: BuddyGroup: (dataOwner: www:7811, groupName: www_7811, buddies: [])
            11:52:26,796 INFO [TreeCache] viewAccepted(): [www:7810|1] [www:7810, www:7811]
            11:52:26,859 INFO [TreeCache] parseConfig(): PojoCacheConfig is empty
            11:52:26,874 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying wi
            colocated buddies as well.
            11:52:26,874 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying ag
            n, ignoring buddy pool hints.
            11:52:26,874 INFO [NextMemberBuddyLocator] Expected to look for 1 buddies but could only find 0 suitable candidates - trying wi
            colocated buddies as well.
            11:52:26,874 INFO [BuddyManager] Assigning new buddies to buddy group [www_7811]. New buddies are [www:7810]
            11:52:26,874 INFO [TreeCache] locking the subtree at / to transfer state
            11:52:26,906 INFO [StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)
            11:52:27,343 INFO [BuddyManager] Assigning self to buddy group BuddyGroup: (dataOwner: www:7810, groupName: www_7810, buddies:
            ww:7811])
            11:52:27,359 INFO [TreeCache] received the state (size=1024 bytes)
            11:52:27,390 INFO [BuddyManager] New buddy group: BuddyGroup: (dataOwner: www:7811, groupName: www_7811, buddies: [www:7810])
            11:52:28,093 INFO [DefaultPartition] Initializing
            11:52:28,156 INFO [STDOUT]
            -------------------------------------------------------
            GMS: address is www:7801 (additional data: 16 bytes)
            -------------------------------------------------------
            11:52:33,671 INFO [DefaultPartition] Number of cluster members: 2
            11:52:33,671 INFO [DefaultPartition] Other members: 1
            11:52:33,671 INFO [DefaultPartition] Fetching state (will wait for 30000 milliseconds):
            11:52:33,968 INFO [DefaultPartition] state was retrieved successfully (in 297 milliseconds)
            11:52:34,109 INFO [HANamingService] Started ha-jndi bootstrap jnpPort=1300, backlog=50, bindAddress=/0.0.0.0
            11:52:34,125 INFO [DetachedHANamingService$AutomaticDiscovery] Listening on /0.0.0.0:1102, group=230.0.0.4, HA-JNDI address=16.
            0.8.76:1300
            11:52:35,500 INFO [CorbaNamingService] Naming: [IOR:000000000000002B49444C3A6F6D672E6F72672F436F734E616D696E672F4E616D696E67436
            E746578744578743A312E3000000000000200000000000000E8000102000000000C31362E3135302E382E3736000E900000000000114A426F73732F4E616D696
            72F726F6F74000000000000050000000000000008000000004A414300000000010000001C0000000000010001000000010501000100010109000000010501000
            000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C31362E3135302E382E3736000DC90040000000000
            000000000001004010008060667810201010100000000000000000000000000000000000000000000002000000004000000000000001F0000000400000003000
            0010000002000000000000000020000002000000004000000000000001F0000000400000003]
            11:52:35,625 INFO [CorbaTransactionService] TransactionFactory: [IOR:000000000000003049444C3A6F72672F6A626F73732F746D2F69696F70
            5472616E73616374696F6E466163746F72794578743A312E30000000000200000000000000E8000102000000000C31362E3135302E382E3736000E9000000000
            144A426F73732F5472616E73616374696F6E732F46000000050000000000000008000000004A414300000000010000001C000000000001000100000001050100
            0001010900000001050100010000002100000060000000000000000100000000000000240000001E0000007E00000000000000010000000C31362E3135302E38
            3736000DC90040000000000000000000000010040100080606678102010101000000000000000000000000000000000000000000000020000000040000000000
            001F0000000400000003000000010000002000000000000000020000002000000004000000000000001F0000000400000003]
            11:52:37,015 INFO [Embedded] Catalina naming disabled
            11:52:37,078 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
            11:52:37,078 INFO [ClusterRuleSetFactory] Unable to find a cluster rule set in the classpath. Will load the default rule set.
            11:52:37,890 INFO [Http11BaseProtocol] Initializing Coyote HTTP/1.1 on http-0.0.0.0-8280
            11:52:37,906 INFO [Catalina] Initialization processed in 828 ms
            11:52:37,906 INFO [StandardService] Starting service jboss.web
            11:52:37,906 INFO [StandardEngine] Starting Servlet Engine: Apache Tomcat/5.5.20
            11:52:38,109 INFO [StandardHost] XML validation disabled
            11:52:38,140 INFO [Catalina] Server startup in 234 ms
            11:52:39,078 INFO [TomcatDeployer] deploy, ctxPath=/invoker, warUrl=.../deploy/httpha-invoker.sar/invoker.war/
            11:52:39,515 INFO [WebappLoader] Dual registration of jndi stream handler: factory already defined
            11:52:42,015 INFO [TomcatDeployer] deploy, ctxPath=/, warUrl=.../deploy/jbossweb-tomcat55.sar/ROOT.war/
            11:52:42,406 INFO [TomcatDeployer] deploy, ctxPath=/jbossws, warUrl=.../tmp/deploy/tmp22398jbossws-context-exp.war/
            11:52:42,843 INFO [TomcatDeployer] deploy, ctxPath=/juddi, warUrl=.../deploy/juddi-service.sar/juddiws.war/
            11:52:43,218 INFO [RegistryServlet] Loading jUDDI configuration.
            11:52:43,234 INFO [RegistryServlet] Resources loaded from: /WEB-INF/juddi.properties
            11:52:43,234 INFO [RegistryServlet] Initializing jUDDI components.
            11:52:44,578 INFO [TomcatDeployer] deploy, ctxPath=/web-console, warUrl=.../deploy/management/console-mgr.sar/web-console.war/
            11:52:45,781 INFO [MailService] Mail Service bound to java:/Mail
            11:52:46,656 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-local-jdbc.rar
            11:52:46,765 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-ha-xa-jdbc.rar
            11:52:46,812 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-local-jdbc.rar
            11:52:46,875 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jboss-xa-jdbc.rar
            11:52:46,953 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/jms/jms-ra.rar
            11:52:47,047 INFO [RARDeployment] Required license terms exist, view META-INF/ra.xml in .../deploy/mail-ra.rar
            11:52:48,422 INFO [WrapperDataSourceService] Bound ConnectionManager 'jboss.jca:name=DefaultDS,service=DataSourceBinding' to JN
            name 'java:DefaultDS'
            11:52:49,547 INFO [ConnectionFactoryBindingService] Bound ConnectionManager 'jboss.jca:name=JmsXA,service=ConnectionFactoryBind
            g' to JNDI name 'java:JmsXA'
            11:52:50,562 INFO [TomcatDeployer] deploy, ctxPath=/OnlineShopping, warUrl=.../tmp/deploy/tmp22434OnlineShopping-exp.war/
            11:52:51,219 INFO [JBossCacheManager] init(): replicationGranularity_ is 0 and invaldateSessionPolicy is 2
            11:52:51,265 INFO [JBossCacheManager] Starting JBossManager
            11:52:51,281 INFO [JBossCacheManager] We are using mod_jk(2) for load-balancing. Will add JvmRouteValve.
            11:52:51,594 INFO [TomcatDeployer] deploy, ctxPath=/jmx-console, warUrl=.../deploy/jmx-console.war/
            11:52:52,219 INFO [FarmMemberService] **** pullNewDeployments ****
            11:52:52,219 INFO [ClusterFileTransfer] Start pull of file cluster-examples-service.xml from cluster.
            11:52:52,265 INFO [ClusterFileTransfer] Finished cluster pull of file cluster-examples-service.xml to cluster-examples-service.
            l
            11:52:52,656 INFO [Http11BaseProtocol] Starting Coyote HTTP/1.1 on http-0.0.0.0-8280
            11:52:53,234 INFO [ChannelSocket] JK: ajp13 listening on /0.0.0.0:8209
            11:52:53,375 INFO [JkMain] Jk running ID=0 time=0/235 config=null
            11:52:53,531 INFO [Server] JBoss (MX MicroKernel) [4.0.5.GA (build: CVSTag=Branch_4_0 date=200610162339)] Started in 53s:844ms



            I have my war file in deploy folder of each node.

            I start accessing the application Everything works fine. Some Data I have in session.
            I also have Set in the jboss-service.xml DATA_REPLICATIOn to SESSIOn level in my war file

            Now I stop node 1. I loose all session data.

            Repeating I have

            true in node 1
            false in node 2

            Am I doing right?


            I want to learn JBOSS ON too - Please suggest.

            Thanks,
            Vishal















            • 3. Re: Buddy Replication Issues
              Brian Stansberry Master

              Sorry, this got buried in the e-mail pile.

              You need to have buddyReplicationEnabled set to true on all nodes.