Protecting cluster from rogue nodes

[proudchief]

In the advanced JBoss training, we were told that using JGroups encryption would protect the cluster from rogue nodes joining the cluster. I can't start the cluster with encryption enabled, nor do I see how it would provide the desired protection, due to the fact that there is no unique key among the cluster that another node would not have access to.

I have followed these steps to enable encryption:

1. Make sure that JDK1.4 is installed, ENCRYPT will not work with any other version under JDK1.4.
2. After installation JDK1.4, set $JAVA_HOME to the installation root.
3. cd $JAVA_HOME/jre/lib/security. Open java.security file.
4. Search for security.provider.n (where n is a digit starting from 1) in the opened file.
5. At the end of the list add Provider.{n+1}=org.bouncycastle.jce.provider.BouncyCastleProvider.
6. Download the BouncyCastle jar from http://www.bouncycastle.org/latest_releases.html (bcprov-jdk14-125.jar), and copy to the $JAVA_HOME/jre/lib/ext directory
7. Add the following line to the JGroups configuration section of the jboss4.0.0/server/all/deploy/cluster-service.xml file, just below the FRAG entry, above the pbcast.GMS entry, to produce the following:

<!-- The JGroups protocol configuration -->
 <attribute name="PartitionConfig">
 <Config>
 <!-- UDP: if you have a multihomed machine,
 set the bind_addr attribute to the appropriate NIC IP address -->
 <!-- UDP: On Windows machines, because of the media sense feature
 being broken with multicast (even after disabling media sense)
 set the loopback attribute to true -->
<!--
 <UDP mcast_addr="228.1.2.3" mcast_port="45566"
 ip_ttl="32" ip_mcast="true"
 mcast_send_buf_size="800000" mcast_recv_buf_size="150000"
 ucast_send_buf_size="800000" ucast_recv_buf_size="150000"
 loopback="false" />
 <PING timeout="2000" num_initial_members="3"
 up_thread="true" down_thread="true" />
 <MERGE2 min_interval="10000" max_interval="20000" />
 <FD shun="true" up_thread="true" down_thread="true"
 timeout="2500" max_tries="5" />
 <VERIFY_SUSPECT timeout="3000" num_msgs="3"
 up_thread="true" down_thread="true" />
 <pbcast.NAKACK gc_lag="50" retransmit_timeout="300,600,1200,2400,4800"
 max_xmit_size="8192"
 up_thread="true" down_thread="true" />
 <UNICAST timeout="300,600,1200,2400,4800" window_size="100" min_threshold="10"
 down_thread="true" />
 <pbcast.STABLE desired_avg_gossip="20000"
 up_thread="true" down_thread="true" />
 <FRAG frag_size="8192"
 down_thread="true" up_thread="true" />
 <ENCRYPT1_4 symInit="56" asymInit="512" symAlgorithm="DES/ECB/PKCS5Padding" asymAlgorithm="RSA"/>
 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
 shun="true" print_local_addr="true" />
 <pbcast.STATE_TRANSFER up_thread="true" down_thread="true" />
 </Config>
 </attribute>

8. Restart JBoss servers

The first node of the cluster starts successfully, but the second produces a stack trace like below when the cluster partition initializes:

java.security.spec.InvalidKeySpecException: Inappropriate key specification
at com.sun.crypto.provider.DESKeyFactory.engineGenerateSecret(DashoA6275)
at javax.crypto.SecretKeyFactory.generateSecret(DashoA6275)
at org.jgroups.protocols.ENCRYPT1_4.decodedKey(ENCRYPT1_4.java:533)
at org.jgroups.protocols.ENCRYPT1_4.up(ENCRYPT1_4.java:339)
at org.jgroups.stack.UpHandler.run(Protocol.java:59)

The only method I can see for protecting the cluster (other than using firewalls) is to statically define the cluster nodes, like the following:

<!-- The JGroups protocol configuration -->
 <attribute name="PartitionConfig">
 <Config>
 <TCP start_port="7900" bind_addr="host1"/>
 <TCPPING initial_hosts="host1[7900],host2[7900]" port_range="1" timeout="3000"
 num_initial_members="1" up_thread="true" down_thread="true"/>
 <FD shun="true" up_thread="true" down_thread="true" timeout="2500" max_tries="5" />
 <VERIFY_SUSPECT timeout="1500" up_thread="false" down_thread="false" />
 <pbcast.NAKACK gc_lag="100" retransmit_timeout="3000" up_thread="true" down_thread="true" />
 <pbcast.STABLE desired_avg_gossip="20000" up_thread="false" down_thread="false" />
 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="false" print_local_addr="false" down_thread="true" up_thread="true" />
 <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
 </Config>
 </attribute>
 </mbean>

Has anyone successfully enabled encryption for their cluster? Are there other methods for protecting a cluser?

[schrouf]

The best cluster protection is to put all cluster servers into a separate network segment, so you can restrict UDP broadcasts to that segment.

Using encryption or just different broadcast IP's might do the job from a technical point of view, but isn't 'waterproof' for a production environment.

Regards
Ulf

[appzworm]

<<The best cluster protection is to put all cluster servers into a separate network segment, so you can restrict UDP broadcasts to that segment.>>

Hi Scrouf!

Could you please provide an example of restricting broadcasts to a network segment. Thanks in advance

[norsay]

Hi

I can explain how we did the setup of our cluster. Perhaps that will help you.

We have 2 Cluster Nodes. Each of them has 2 NICs. 1 NIC of each node has a valid IP for the subnet where they are attached. The 2nd NIC of each node is used as an interconnecten between the 2 nodes. So they both have a "fake" network address. Moreover the nodes are connected directly (it's like you connect them with a cross cable directly fron NIC to NIC). To avoid the MC broad cast, I enabled MC features only for the 2nd NIC on each node... That's it.

192.168.1.0/24 LAN
===================================================================
 | |
 | eth0: 192.168.1.1 | eth0: 192.168.1.2
 + +
 Node 1 +-------------------------------------+ Node 2
 eth1*: 1.2.3.1 eth1*: 1.2.3.2

*) connected with a crosscable from NIC-to-NIC

Hope that helps ;-)