3 Replies Latest reply on Jul 21, 2005 10:31 AM by jameselliot

    My two nodes won't stop arguing.

    jameselliot

      I now have two servers running TreeCache as a hibernate 2nd level cache. The final aim is to have a synchronous replicated cache, each server has its own database and I hope to be able to write all data to both databases using TreeCache.

      Currently my two cache setups refuse to talk to each other:

      First cache to start says:

      2005-07-20 16:00:15,068 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.10:36755] discarded message from non-member 10.0.10.50:33375
      2005-07-20 16:00:32,550 ERROR [org.jgroups.protocols.pbcast.Digest] index 2 out of bounds, please create new Digest if you want more members !
      2005-07-20 16:00:32,551 ERROR [org.jgroups.protocols.pbcast.Digest] index 2 out of bounds, please create new Digest if you want more members !
      2005-07-20 16:00:32,564 ERROR [org.jgroups.protocols.pbcast.Digest] index 2 out of bounds, please create new Digest if you want more members !
      2005-07-20 16:00:32,565 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.10:36755] discarded message from non-member 10.0.10.50:33376
      2005-07-20 16:00:52,168 ERROR [org.jgroups.protocols.pbcast.Digest] index 2 out of bounds, please create new Digest if you want more members !
      2005-07-20 16:00:52,169 ERROR [org.jgroups.protocols.pbcast.Digest] index 2 out of bounds, please create new Digest if you want more members !
      2005-07-20 16:00:52,169 ERROR [org.jgroups.protocols.pbcast.Digest] index 2 out of bounds, please create new Digest if you want more members !
      2005-07-20 16:00:52,170 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.10:36755] discarded message from non-member 10.0.10.50:33377
      


      Second cache to start says:

      2005-07-20 16:04:01,084 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.50:33379] discarded message from non-member 10.0.10.10:36755
      2005-07-20 16:04:01,084 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.50:33379] discarded message from non-member 10.0.10.10:36755
      2005-07-20 16:04:01,084 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.50:33379] discarded message from non-member 10.0.10.10:36755
      2005-07-20 16:04:01,084 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.50:33379] discarded message from non-member 10.0.10.10:36755
      2005-07-20 16:04:01,084 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.50:33379] discarded message from non-member 10.0.10.10:36755
      2005-07-20 16:04:01,084 WARN [org.jgroups.protocols.pbcast.NAKACK] 10.0.10.50:33379] discarded message from non-member 10.0.10.10:36755
      2005-07-20 16:04:01,600 INFO [STDOUT]
      -------------------------------------------------------
      GMS: address is 10.0.10.50:33380
      -------------------------------------------------------
      2005-07-20 16:04:03,604 INFO [org.jboss.cache.TreeCache] viewAccepted(): new members: [10.0.10.50:33380]
      2005-07-20 16:04:03,604 INFO [org.jboss.cache.TreeCache] new cache is null (maybe first member in cluster)
      



      Both caches are configured the same with:

       <server>
       <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>
      
       <mbean code="org.jboss.invocation.jrmp.server.JRMPProxyFactory" name="localhost:service=proxyFactory,type=jrmp,target=factory">
       <attribute name="InvokerName">jboss:service=invoker,type=jrmp</attribute>
       <attribute name="TargetName">jboss.cache:service=TreeCache</attribute>
       <attribute name="JndiName">VicopTestCache</attribute>
       <attribute name="InvokeTargetMethod">true</attribute>
       <attribute name="ExportedInterface">org.jboss.cache.TreeCacheMBean</attribute>
       <attribute name="ClientInterceptors">
       <iterceptors>
       <interceptor>org.jboss.proxy.ClientMethodInterceptor</interceptor>
       <interceptor>org.jboss.proxy.SecurityInterceptor</interceptor>
       <interceptor>org.jboss.invocation.InvokerInterceptor</interceptor>
       </iterceptors>
       </attribute>
       <depends>jboss:service=invoker,type=jrmp</depends>
       <depends>jboss.cache:service=TreeCache</depends>
       <attribute name="IsolationLevel">REPEATABLE_READ</attribute>
       <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>
       <attribute name="ClusterName">VicopTestCluster</attribute>
       <attribute name="CacheMode">REPL_SYNC</attribute>
       <attribute name="ClusterConfig">
       <config>
       <!-- UDP: if you have a multihomed machine,
       set the bind_addr attribute to the appropriate NIC IP address -->
       <!-- UDP: On Windows machines, because of the media sense feature
       being broken with multicast (even after disabling media sense)
       set the loopback attribute to true -->
       <UDP mcast_addr="225.10.10.20" mcast_port="45566"
       ip_ttl="64" ip_mcast="true"
       mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
       ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
       loopback="false"/>
       <PING timeout="2000" num_initial_members="3"
       up_thread="false" down_thread="false"/>
       <MERGE2 min_interval="10000" max_interval="20000"/>
       <FD shun="true" up_thread="true" down_thread="true"/>
       <VERIFY_SUSPECT timeout="1500"
       up_thread="false" down_thread="false"/>
       <pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800"
       max_xmit_size="8192" up_thread="false" down_thread="false"/>
       <UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10"
       down_thread="false"/>
       <pbcast.STABLE desired_avg_gossip="20000"
       up_thread="false" down_thread="false"/>
       <FRAG frag_size="8192"
       down_thread="false" up_thread="false"/>
       <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
       shun="true" print_local_addr="true"/>
       <pbcast.STATE_TRANSFER up_thread="false" down_thread="false"/>
       </config>
       </attribute>
       </mbean>
      </server>
      


      I have had a good hunt around for configuring a Digest but have found nothing, a few posts have mentioned the 'discarded message from non-member' error but do not appear to have solved the problem.

      Any suggestions would be most welcome.

      Thank you, --James.


        • 1. Re: My two nodes won't stop arguing.
          belaban

          The members don't find each other. I suggest download JGroups and follow the instructions in INSTALL.html for troubleshooting.
          Use ViewDemo or Draw to see whether they find each other and form a group. Only after this is successful would I suggest to try JBossCache

          • 2. Re: My two nodes won't stop arguing.
            jameselliot

            I had to open up a lot of other ports so the two servers could find each other.

            Now one server seems to have added its self to the group lots of times:

            2005-07-21 12:15:45,647 INFO [org.jboss.cache.TreeCache] viewAccepted(): new members: [10.0.10.50:32779, 10.0.10.10:36760, 10.0.10.10:36761, 10.0.10.10:36762, 10.0.10.10:36763, 10.0.10.10:36764, 10.0.10.10:36765, 10.0.10.10:36766, 10.0.10.10:36767, 10.0.10.10:36768, 10.0.10.10:36769, 10.0.10.10:36770, 10.0.10.10:36771, 10.0.10.10:36772, 10.0.10.10:36773, 10.0.10.10:36774, 10.0.10.10:36775, 10.0.10.10:36776, 10.0.10.10:36777, 10.0.10.10:36778, 10.0.10.10:36779, 10.0.10.10:36780, 10.0.10.10:36781, 10.0.10.10:36782, 10.0.10.10:36783, 10.0.10.10:36784, 10.0.10.10:36785, 10.0.10.10:36786, 10.0.10.10:36787, 10.0.10.10:36788, 10.0.10.10:36789, 10.0.10.10:36790, 10.0.10.10:36791, 10.0.10.10:36792, 10.0.10.10:36793, 10.0.10.10:36796, 10.0.10.10:36799, 10.0.10.10:36802, 10.0.10.10:36806]


            Can this be right?

            Has anyone done a worked example of jboss-cache with hibernate? Considering these are two core parts of Jboss they seem very badly documented. Can you recommend a book, or an article somewhere?

            Still seem to be a long way from a replicated cache, replicating state to two databases.

            -James.

            • 3. Re: My two nodes won't stop arguing.
              jameselliot

              Tracking it down a bit further with a couple more alterations. It now appears as if they sort of hook up, but not quite.

              The first server to start says:

              2005-07-21 15:27:34,415 INFO [org.jboss.cache.TreeCache] locking the tree to obtain transient state
              2005-07-21 15:27:34,422 INFO [org.jboss.cache.TreeCache] returning the transient state (24213 bytes)
              


              While the second server to start says:

              -------------------------------------------------------
              GMS: address is 10.0.10.10:32818
              -------------------------------------------------------
              2005-07-21 15:25:42,824 INFO [org.jboss.cache.TreeCache] viewAccepted(): new members: [10.0.10.50:32807, 10.0.10.10:32768, 10.0.10.10:32771, 10.0.10.10:32774, 10.0.10.10:32777, 10.0.10.10:32780, 10.0.10.10:32781, 10.0.10.10:32782, 10.0.10.10:32783, 10.0.10.10:32784, 10.0.10.10:32785, 10.0.10.10:32786, 10.0.10.10:32787, 10.0.10.10:32788, 10.0.10.10:32789, 10.0.10.10:32790, 10.0.10.10:32791, 10.0.10.10:32792, 10.0.10.10:32793, 10.0.10.10:32794, 10.0.10.10:32795, 10.0.10.10:32796, 10.0.10.10:32797, 10.0.10.10:32798, 10.0.10.10:32799, 10.0.10.10:32800, 10.0.10.10:32801, 10.0.10.10:32802, 10.0.10.10:32803, 10.0.10.10:32804, 10.0.10.10:32805, 10.0.10.10:32806, 10.0.10.10:32807, 10.0.10.10:32808, 10.0.10.10:32811, 10.0.10.10:32814, 10.0.10.10:32817, 10.0.10.10:32818]
              2005-07-21 15:25:47,836 INFO [org.jboss.cache.TreeCache] received the state (size=30 bytes)
              


              Are these now correctly hooked up?

              How do I configure jboss-cache and hibernate to replicate state to two seperate databases on the two seperate servers?

              I recall reading that you should not use CacheLoaders with hibernate, however the documentation of Jboss-cache seems to imply that you get replication into databases using CacheLoaders?

              Thanks,

              James.