The information below is for perf.Test which was the default perf test up to and including JGroups 3.0. In 3.1, perf.Test was replaced with MPerf. See http://www.jgroups.org/manual-3.x/html/user-advanced.html#MPerf for details on MPerf.

Setup

The performance test measures the time it takes to send a number of messages to all members of a group. It computes messages/sec and throughput for each member in the group.

The topology is defined in a configuration file, which lists

• num_msgs: the total number of messages a sender is to send

• msg_size: the size of each message, in bytes

• num_members: the number of members in the group

• num_senders: the number of senders in that group. Note that num_senders has to be less than or equal to num_members

A member is started as follows:

java org.jgroups.tests.perf.Test -sender|-receiver -config c:\config.txt -props c:\fc-fast.xml

The -sender options makes the member a sender, the -receiver option makes it a receiver.

-config points to the configuration file that needs to be the same for all members (place it on a shared file system). Same for the -props option which points to the stack configuration to be used by all members.

I suggest use the JGroups/conf/config.txt configuration file and JGroups/conf/fc-fast.xml stack configuration as starting points, and modify them to change the configuration and/or fine-tune the stack.

Running the tests

Let's say we're running a 4 node cluster, and have 2 senders. The steps to do are the following:

• Copy the config.txt file to c:\ and modify it, e.g.:

transport=org.jgroups.tests.perf.transports.JGroupsTransport
#transport=org.jgroups.tests.perf.transports.UdpTransport
#transport=org.jgroups.tests.perf.transports.TcpTransport

# Number of messages a sender multicasts
num_msgs=20000

# Message size in bytes.
msg_size=1000


# Expected number of group members.
num_members=4

# Number of senders in the group. Min 1, max num_members.
num_senders=2

• Start a receiver on 2 boxes, looks somewhat like the output below:

java org.jgroups.tests.perf.Test -receiver -config c:\config.txt -props c:\fc-fast.xml
-------------------------------------------------------
GMS: address is 172.16.130.33:33415
-------------------------------------------------------
-- waiting for 4 members to join
-- 172.16.128.33:18532 joined
-- members: 1
-- 172.16.130.33:33415 joined
-- members: 2

• Start a sender on 2 boxes:

java org.jgroups.tests.perf.Test -sender -config c:\config.txt -props c:\fc-fast.xml
-------------------------------------------------------
GMS: address is 172.16.128.33:18535
-------------------------------------------------------
-- waiting for 4 members to join
-- 172.16.128.33:18535 joined
-- members: 1
-- 172.16.130.33:33415 joined
-- members: 2
-- 172.16.128.33:18532 joined
-- members: 3

• When the 4th member (a sender) has been started, and the group has formed, all senders will start sending messages. In the above example, there are 2 senders, each sending 20000 1K messages, so the group will receive a total of 40000 1K messages (including the senders). If the group doesn't form, the senders will never start.

• The output on the last sender looks like this:

-- fetching results (from 4 members)
-- received 20000 messages from 172.16.132.33:51062
-- received all responses

-- results:

-- results from 172.16.132.33:51062:
sender: 172.16.132.33:51062: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6305ms, msgs/sec=3172.09, throughput=3.17MB/se
c
sender: 172.16.128.33:18535: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6192ms, msgs/sec=3229.97, throughput=3.23MB/se
c

combined: num_msgs_expected=40000, num_msgs_received=40000 (loss rate=0%), recei
ved=40MB, time=6374ms, msgs/sec=6275.49, throughput=6.28MB/sec

-- results from 172.16.128.33:18532:
sender: 172.16.132.33:51062: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6401ms, msgs/sec=3124.51, throughput=3.12MB/se
c
sender: 172.16.128.33:18535: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6330ms, msgs/sec=3159.56, throughput=3.16MB/se
c

combined: num_msgs_expected=40000, num_msgs_received=40000 (loss rate=0%), recei
ved=40MB, time=6425ms, msgs/sec=6225.68, throughput=6.23MB/sec

-- results from 172.16.128.33:18535:
sender: 172.16.132.33:51062: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6304ms, msgs/sec=3172.59, throughput=3.17MB/se
c
sender: 172.16.128.33:18535: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6263ms, msgs/sec=3193.36, throughput=3.19MB/se
c

combined: num_msgs_expected=40000, num_msgs_received=40000 (loss rate=0%), recei
ved=40MB, time=6372ms, msgs/sec=6277.46, throughput=6.28MB/sec

-- results from 172.16.130.33:33415:
sender: 172.16.132.33:51062: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6375ms, msgs/sec=3137.25, throughput=3.14MB/se
c
sender: 172.16.128.33:18535: num_msgs_expected=20000, num_msgs_received=20000 (l
oss rate=0%), received=20MB, time=6213ms, msgs/sec=3219.06, throughput=3.22MB/se
c

combined: num_msgs_expected=40000, num_msgs_received=40000 (loss rate=0%), recei
ved=40MB, time=6398ms, msgs/sec=6251.95, throughput=6.25MB/sec

• Each sender contacts all receivers and asks them for their stats, so in the above output, the last sender gets 4 responses (also from itself). Then it aggregates the results and prints the aggregated stats (look for the last line, starting with "combined"). In the example above, we received 40000 1K messages in 6.3 seconds, which is a message rate of roughly 6000 1K messages/sec.

• Note that the test driver is JGroups independent. The test can be used on

  • JGroups

  • UDP (using IP multicast packets)

  • TCP and

  • JMS

Misc

Note that the tests will run much faster if logging is turned off. So make sure you don't use log4j.properties or log4j.xml. Alternatively, simply comment all appenders in log4j.properties.