5 Replies Latest reply on Jun 8, 2007 6:40 AM by manik

    JbossCache replication and ip multicast

    mckeno
    mckeno May 22, 2007 12:03 AM

      Hi all,

      As I understand, JbossCache accomplished the clustering and replication by using JGroups. JGroups in the other hand uses ip multicasting to multicast a message to all the nodes that register to a multicast address.
      The default configure for ClusterConfig uses UDP, in the JGroups documentation it says:

      TCP is a replacement of UDP as bottom layer in cases where IP Multicast based on UDP is not desired. This may
      be the case when operating over a WAN, where routers will discard IP MCAST. As a rule of thumb UDP is used as
      transport for LANs, whereas TCP is used for WANs.

      So if I want to have my cache replicated over different server via internet (not just in LAN) I would have configure the JbossCache to use TCP protocol right? I tried to do that, but it doesn't seem to work.... could anybody help?

      Before: 
      <attribute name="ClusterConfig">
<config>
<UDP mcast_addr="228.1.2.5" mcast_port="45577"
 ip_ttl="64" ip_mcast="true"
 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
 ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
 loopback="false"/>-->
<PING timeout="2000" num_initial_members="3"
 up_thread="false" down_thread="false"/>-->
...
...

      Now: 
      change to

<attribute name="ClusterConfig">
<config>
<TCP start_port="7800"/>
<PING timeout="3000" num_initial_members="3" />
...
...
or

<attribute name="ClusterConfig">
<config>
<TCP start_port="7800"/>
<TCPPING initial_hosts="localhost[7800]"
 port_range="5" timeout="3000" num_initial_members="3"
 up_thread="true" down_thread="true"/>
...
...

but does not work ...


      BTW, how does this multicast works, do I need a host or server to bind to or something?

      Thanks in advance


      • 4010 Views
      • Tags:
        • 1. Re: JbossCache replication and ip multicast
          manik
          manik May 22, 2007 4:16 AM (in response to mckeno)

          You should set your initial_hosts (in TCPPING) to the _other_ hosts in your cluster, not to point to yourself.



          BTW, how does this multicast works, do I need a host or server to bind to or something?


          Multicasting is a networking concept which allows for delivery to multiple addresses at the same time. See Multicast and IP_Multicast for more info. It does not require a server, just that all members listen on a MCAST address.



            • Actions
          • 2. Re: JbossCache replication and ip multicast
            mckeno
            mckeno May 22, 2007 4:40 AM (in response to mckeno)

            Thank you very much for your information.

            You should set your initial_hosts (in TCPPING) to the _other_ hosts in your cluster, not to point to yourself.

            So there should always be a host which joins the cluster at first? (we have to make sure that this host always joins the cluster at the first time?)

            Anyway, why do we need this so called initial_host? what does JbossCache as well as JGroups need it for? Wasn't it suppose that every host is "equal" in a replication group (or cluster) ?

            For multicasting, i see that it uses the class D address range which is from 224.0.0.0 to 239.255.255.255. But as I am not so familiar with the network configuration and concept, I am wondering how could a host outside of the LAN "sees" the multicast address of a certain group inside a LAN.
            In the JGroups document it saids :

            As a rule of thumb UDP is used as transport for LANs, whereas TCP is used for WANs.

            So that is why I want to change my initial UDP config to TCP config (am I on the right track, anyway?)
            Again, thanks for your reply

              • Actions
            • 3. Re: JbossCache replication and ip multicast
              manik
              manik May 22, 2007 8:37 AM (in response to mckeno)

               



              So there should always be a host which joins the cluster at first? (we have to make sure that this host always joins the cluster at the first time?)

              Anyway, why do we need this so called initial_host? what does JbossCache as well as JGroups need it for? Wasn't it suppose that every host is "equal" in a replication group (or cluster) ?




              SInce you don't use UDP/MCAST, there is no way for the peers to discover each other. So JGroups is forced to use TCPPING for discovery, which needs to know where to look, so to speak.


              I am wondering how could a host outside of the LAN "sees" the multicast address of a certain group inside a LAN.


              A host outside the LAN will not see a cluster within a LAN if you're using UDP/MCAST. If you want to be able to do this, use TCP - which you are doing. If you need to tunnel through firewalls, read up on the JGroups GossipRouter.



                • Actions
              • 4. Re: JbossCache replication and ip multicast
                mckeno
                mckeno May 22, 2007 11:46 PM (in response to mckeno)

                Yup, I did try to use the gossip router yesterday and it does work.

                Sorry for didn't look at it clearly in the JGroups document (initially I was dealing with JbossCache :) ).
                To sum up (to confirm my crumbling knowledge, hope you don't mind :) ):
                For WAN there is a chance that some router will discard the IP Mcast message, So in order to multicast to different nodes in WAN JGroups solved it by sending multiple unicast messages.

                For UDP/MCAST it uses IP multicast (by default) to find initial members (PING protocol). But for TCP this cannot be used, so again JGroups provide us with 3 alternatives:
                - PING a so called GossipRouter
                - TCPPING a pre-known group member or host
                - TCPGOSSIP, same as PING a GossipRouter mechanism but we can have more than 1 GossipRouter.

                For all 3 solutions, the GossipRouter or pre-known host has alway to be start up at first(before any other member begin to join the group)

                Personally I prefer the GossipRouter solution (I am thinking of a way to have GossipRouter to start along with the Jboss AS, don't know if that is possible or not)

                I would like to have one more question, about the TCP protocol.
                As it says in JGroups document the use of send multiple unicast messages instead of one multicast message is not that efficient and use more bandwidth and there is a protocol in JGroups which is called TCP_NIO.
                This TCP_NIO protocol is said to be more scalable for large clusters than TCP, so can I use this protocol in Jboss ClusterConfig?

                  • Actions
                • 5. Re: JbossCache replication and ip multicast
                  manik
                  manik Jun 8, 2007 6:40 AM (in response to mckeno)

                   


                  This TCP_NIO protocol is said to be more scalable for large clusters than TCP, so can I use this protocol in Jboss ClusterConfig?

                  Yes you can.

                    • Actions