3 Replies Latest reply on Jan 25, 2005 3:06 AM by Norsay Norsay

    Protecting cluster from rogue nodes

    Don Smith Newbie

      In the advanced JBoss training, we were told that using JGroups encryption would protect the cluster from rogue nodes joining the cluster. I can't start the cluster with encryption enabled, nor do I see how it would provide the desired protection, due to the fact that there is no unique key among the cluster that another node would not have access to.

      I have followed these steps to enable encryption:

      1. Make sure that JDK1.4 is installed, ENCRYPT will not work with any other version under JDK1.4.
      2. After installation JDK1.4, set $JAVA_HOME to the installation root.
      3. cd $JAVA_HOME/jre/lib/security. Open java.security file.
      4. Search for security.provider.n (where n is a digit starting from 1) in the opened file.
      5. At the end of the list add Provider.{n+1}=org.bouncycastle.jce.provider.BouncyCastleProvider.
      6. Download the BouncyCastle jar from http://www.bouncycastle.org/latest_releases.html (bcprov-jdk14-125.jar), and copy to the $JAVA_HOME/jre/lib/ext directory
      7. Add the following line to the JGroups configuration section of the jboss4.0.0/server/all/deploy/cluster-service.xml file, just below the FRAG entry, above the pbcast.GMS entry, to produce the following:

      <!-- The JGroups protocol configuration -->
       <attribute name="PartitionConfig">
       <!-- UDP: if you have a multihomed machine,
       set the bind_addr attribute to the appropriate NIC IP address -->
       <!-- UDP: On Windows machines, because of the media sense feature
       being broken with multicast (even after disabling media sense)
       set the loopback attribute to true -->
       <UDP mcast_addr="" mcast_port="45566"
       ip_ttl="32" ip_mcast="true"
       mcast_send_buf_size="800000" mcast_recv_buf_size="150000"
       ucast_send_buf_size="800000" ucast_recv_buf_size="150000"
       loopback="false" />
       <PING timeout="2000" num_initial_members="3"
       up_thread="true" down_thread="true" />
       <MERGE2 min_interval="10000" max_interval="20000" />
       <FD shun="true" up_thread="true" down_thread="true"
       timeout="2500" max_tries="5" />
       <VERIFY_SUSPECT timeout="3000" num_msgs="3"
       up_thread="true" down_thread="true" />
       <pbcast.NAKACK gc_lag="50" retransmit_timeout="300,600,1200,2400,4800"
       up_thread="true" down_thread="true" />
       <UNICAST timeout="300,600,1200,2400,4800" window_size="100" min_threshold="10"
       down_thread="true" />
       <pbcast.STABLE desired_avg_gossip="20000"
       up_thread="true" down_thread="true" />
       <FRAG frag_size="8192"
       down_thread="true" up_thread="true" />
       <ENCRYPT1_4 symInit="56" asymInit="512" symAlgorithm="DES/ECB/PKCS5Padding" asymAlgorithm="RSA"/>
       <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
       shun="true" print_local_addr="true" />
       <pbcast.STATE_TRANSFER up_thread="true" down_thread="true" />

      8. Restart JBoss servers

      The first node of the cluster starts successfully, but the second produces a stack trace like below when the cluster partition initializes:

      java.security.spec.InvalidKeySpecException: Inappropriate key specification
      at com.sun.crypto.provider.DESKeyFactory.engineGenerateSecret(DashoA6275)
      at javax.crypto.SecretKeyFactory.generateSecret(DashoA6275)
      at org.jgroups.protocols.ENCRYPT1_4.decodedKey(ENCRYPT1_4.java:533)
      at org.jgroups.protocols.ENCRYPT1_4.up(ENCRYPT1_4.java:339)
      at org.jgroups.stack.UpHandler.run(Protocol.java:59)

      The only method I can see for protecting the cluster (other than using firewalls) is to statically define the cluster nodes, like the following:

      <!-- The JGroups protocol configuration -->
       <attribute name="PartitionConfig">
       <TCP start_port="7900" bind_addr="host1"/>
       <TCPPING initial_hosts="host1[7900],host2[7900]" port_range="1" timeout="3000"
       num_initial_members="1" up_thread="true" down_thread="true"/>
       <FD shun="true" up_thread="true" down_thread="true" timeout="2500" max_tries="5" />
       <VERIFY_SUSPECT timeout="1500" up_thread="false" down_thread="false" />
       <pbcast.NAKACK gc_lag="100" retransmit_timeout="3000" up_thread="true" down_thread="true" />
       <pbcast.STABLE desired_avg_gossip="20000" up_thread="false" down_thread="false" />
       <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="false" print_local_addr="false" down_thread="true" up_thread="true" />
       <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>

      Has anyone successfully enabled encryption for their cluster? Are there other methods for protecting a cluser?

        • 1. Re: Protecting cluster from rogue nodes
          Ulf Schroeter Novice

          The best cluster protection is to put all cluster servers into a separate network segment, so you can restrict UDP broadcasts to that segment.

          Using encryption or just different broadcast IP's might do the job from a technical point of view, but isn't 'waterproof' for a production environment.


          • 2. Re: Protecting cluster from rogue nodes
            appzworm Newbie

            <<The best cluster protection is to put all cluster servers into a separate network segment, so you can restrict UDP broadcasts to that segment.>>

            Hi Scrouf!

            Could you please provide an example of restricting broadcasts to a network segment. Thanks in advance

            • 3. Re: Protecting cluster from rogue nodes
              Norsay Norsay Newbie


              I can explain how we did the setup of our cluster. Perhaps that will help you.

              We have 2 Cluster Nodes. Each of them has 2 NICs. 1 NIC of each node has a valid IP for the subnet where they are attached. The 2nd NIC of each node is used as an interconnecten between the 2 nodes. So they both have a "fake" network address. Moreover the nodes are connected directly (it's like you connect them with a cross cable directly fron NIC to NIC). To avoid the MC broad cast, I enabled MC features only for the 2nd NIC on each node... That's it.

               | |
               | eth0: | eth0:
               + +
               Node 1 +-------------------------------------+ Node 2
               eth1*: eth1*:
              *) connected with a crosscable from NIC-to-NIC

              Hope that helps ;-)