JMS distributed performance testing framework complete
I have completed and committed the JMS distributed performance testing framework.
The framework captures a variety of performance benchmarks against a particular JMS provider, using multiple jms clients on potentially multiple machines.
It also charts results providing output in html format with jpegs for the charts.
Charts show the historical development of a benchmark over time, as well as allow you to graphically compare the results for various different JMS providers, so far I have only run it against JBossMessaging and JBossMQ, but ActiveMQ is the next on my list.
I shall shortly be adding a wiki page to describe how to use the framework.
The following benchmarks are currently being captured: (More benchmarks can be easily added if required). In my experience we are taking a much broader range of jms performance metrics than any other jms performance benchmark suite i have seen.
If anyone can think of any other useful measurements we can take I shall take note.
Message size
============
MessageSizeThroughput - Throughput is measure as a functional of message size. This gives us a picture of how various providers cope with messages of different sizes.
Scalability measurements
========================
Various scalability measurements are taken:
QueueScale1 - We see how the JMS provider scales with number of queues by measuring overall throughput as we vary the number of queues being concurrently sent to/received from.
QueueScale2 - We measure how the jms provider scales with number of sending connections.
QueueScale3 - We measure how the jms provider scales with number of sending sessions (all sessions sharing same connection)
TopicScale1 - We see how the JMS provider scales with number of topics by measuring overall throughput as we vary the number of topics being concurrently sent to/received from.
TopicScale2 - We measure how the jms provider scales with number of receiving connections. One subscriber per connection.
TopicScale3 - We measure how the jms provider scales with number of subscribers sharing same connection.
Simple Queue benchmarks
=======================
Queue1 - Simple non transactional send of non-persistent messages to single queue with no consumers. Number of msgs/sec is captured.
Queue2 - Simple non transactional send of persistent messages to single queue with no consumers. Number of msgs/sec is captured.
Queue3 - Simple transactional send of non-persistent messages to single queue with no consumers. Number of msgs/sec is captured.
Queue4 - Simple transactional send of persistent messages to single queue with no consumers. Number of msgs/sec is captured.
Queue5 - Concurrent send and receive of non-persistent messages to queue. Non transactional (both sender and receiver). Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Queue6 - Concurrent send and receive of persistent messages to queue. Non transactional (both sender and receiver). Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Queue7 - Concurrent send and receive of non-persistent messages to queue. Transactional (both sender and receiver). Overall throughput in msgs/sec is captured.
Queue8 - Concurrent send and receive of persistent messages to queue. Transactional (both sender and receiver). Overall throughput in msgs/sec is captured.
Queue9 - Concurrent send and receive of non-persistent messages to queue. Non transactional (both sender and receiver). Overall throughput in msgs/sec is captured. Ack mode of DUPS_OK_ACKNOWLEDGE.
Queue10 - Concurrent send and receive of persistent messages to queue. Non transactional (both sender and receiver). Overall throughput in msgs/sec is captured. Ack mode of DUPS_OK_ACKNOWLEDGE.
Queue11 - Receive only of persistent messages from queue (Messages are already existent on queue). Non transactional. Ack mode of AUTO_ACKNOWLEDGE. Message receive rate in msgs/sec is captured.
Queue12 - Receive only of non-persistent messages from queue (Messages are already existent on queue). Non transactional. Ack mode of AUTO_ACKNOWLEDGE. Message receive rate in msgs/sec is captured.
Queue13 - Receive only of persistent messages from queue (Messages are already existent on queue). Transactional. Message receive rate in msgs/sec is captured.
Queue14 - Receive only of non-persistent messages from queue (Messages are already existent on queue). Transactional. Message receive rate in msgs/sec is captured.
Queue15 - Receive only of persistent messages from queue with selector (Messages are already existent on queue). Non transactional. Ack mode of AUTO_ACKNOWLEDGE. Message receive rate in msgs/sec is captured.
Queue16 - Receive only of non-persistent messages from queue with selector (Messages are already existent on queue). Non transactional. Ack mode of AUTO_ACKNOWLEDGE. Message receive rate in msgs/sec is captured.
Queue17 - Browse persistent messages on queue. Browse rate in msgs/sec is captured.
Queue18 - Browse non-persistent messages on queue. Browse rate in msgs/sec is captured.
Queue19 - Browse non-persistent messages on queue with selector. Browse rate in msgs/sec is captured.
Simple Topic benchmarks
=======================
Topic1 - Concurrent send and receive of non-persistent messages to topic with single non-durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Topic2 - Concurrent send and receive of persistent messages to topic with single non-durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Topic3 - Concurrent send and receive of non-persistent messages to topic with single non-durable subscriber. Transactional. Overall throughput in msgs/sec is captured.
Topic4 - Concurrent send and receive of persistent messages to topic with single non-durable subscriber. Transactional. Overall throughput in msgs/sec is captured.
Topic5 - Concurrent send and receive of non-persistent messages to topic with single non-durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of DUPS_OK_ACKNOWLEDGE.
Topic6 - Concurrent send and receive of persistent messages to topic with single non-durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of DUPS_OK_ACKNOWLEDGE.
Topic7 - Concurrent send and receive with selector of non-persistent messages to topic with single non-durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Topic8 - Concurrent send and receive with selector of persistent messages to topic with single non-durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Topic9 - Concurrent send and receive of persistent messages to topic with single durable subscriber. Non transactional. Overall throughput in msgs/sec is captured. Ack mode of AUTO_ACKNOWLEDGE.
Topic10 - Concurrent send and receive of persistent messages to topic with single durable subscriber. Transactional. Overall throughput in msgs/sec is captured.
