Definition

S3_PING uses Amazon S3 to discover initial members. It's designed specifically for members running on Amazon EC2, where multicast traffic is not allowed and thus MPING will not work. Each instance uploads a small file to an S3 bucket and each instance reads the files out of this bucket to determine the other members.

There are three different ways to use S3_PING, each having its own tradeoffs between security and ease-of-use. These are described in more detail below after the Configuration sections:

• Private buckets, Amazon AWS credentials given to each instance
• Public readable and writable buckets, no credentials given to each instance
• Public readable but private writable buckets, pre-signed URLs given to each instance

Pre-signed URLs are the most secure method since writing to buckets still requires authorization and you don't have to pass Amazon AWS credentials to every instance. However, they are also the most complex to setup.

Configuration Examples

Configuration example for private buckets with credentials given to each instance:

<S3_PING location="my_bucket" access_key="access_key"
         secret_access_key="secret_access_key" timeout="2000"
         num_initial_members="3"/>

Configuration example for public buckets with no credentials:

<S3_PING location="my_bucket"
         timeout="2000" num_initial_members="3"/>

Configuration example for public readable buckets with pre-signed URLs:

<S3_PING pre_signed_put_url="http://s3.amazonaws.com/my_bucket/DemoCluster/node1?AWSAccessKeyId=access_key&Expires=1316276200&Signature=it1cUUtgCT9ZJyCJDj2xTAcRTFg%3D"
         pre_signed_delete_url="http://s3.amazonaws.com/my_bucket/DemoCluster/node1?AWSAccessKeyId=access_key&Expires=1316276200&Signature=u4IFPRq%2FL6%2FAohykIW4QrKjR23g%3D"
         timeout="2000" num_initial_members="3"/>

Configuration Parameters

See also Protocol Configuration Common Parameters.

Detailed Configuration Scenarios

Pre-Signed URLs

Using S3_PING with pre-signed URLs is the recommended approach. In this scenario, you have to pre-create a bucket that is publically readable but requires authorization for writing. Then you need to generate a pair of pre-signed URLs for each node, one for each http method PUT and DELETE. If your configuration file gets compromised, it would only allow an attacker to upload files to a single S3 bucket and key until your pre-signed URL expires instead of giving them full access to your entire Amazon AWS account.

Amazon requires you to give an expiration date for pre-signed URLs, so make sure to choose a date far into the future (many years) if these are on machines that you intend to keep running for a long time.

Generating Pre-Signed URLs

The S3_PING class in JGroups comes with a utility method to generate these pre-signed URLs.

Example:

String putUrl = S3_PING.generatePreSignedUrl("access_key", "secret_access_key", "put",
                                             "my_bucket", "DemoCluster/node1",
                                              1234567890);

String deleteUrl = S3_PING.generatePreSignedUrl("access_key", "secret_access_key", "delete",
                                                "my_bucket", "DemoCluster/node1",
                                                1234567890);

Edit and run the attached [^s3_presigned_urls.groovy.zip] if that's more convenient to you.

If you'd like to generate pre-signed URLs without using the utility provided by S3_PING, please refer to Amazon's Rest Authentication documentation. You'll need to ensure the PUT url is generated with the header 'x-amz-acl' with a value of 'public-read' and both the PUT and DELETE urls should omit 'Content-MD5' and 'Content-Type' headers. S3_PING's utility method does this automatically.

Private Buckets with AWS Credentials

Before JGroups 2.10.1, this was the only scenario support. In this scenario your buckets are private and no one can read or write their data without authorization. However, you have to put your Amazon AWS credentials in the configuration file for each node and thus have to secure the configuration files very carefully to prevent someone from capturing your credentials.

Public Buckets

It is possible to use S3_PING without passing AWS credentials or pre-signed URLs to the nodes. You need to pre-create an S3 bucket that is readable and writable by anonymous users, pass this bucket as the location parameter in your configuration, and omit the access_key, secret_access_key, pre_signed_put_url, and pre_signed_delete_url parameters. Please note that this is not recommended as anyone that figures out your bucket name can upload data to your bucket.

Back To JGroups