JMS automated perf. testing framework
I've been working on a JMS performance testing framework for the last few days - it's about 75% complete.
I had a look at what was available on the market:
IBM provide a simple tool, but it's quite limited in functionality and has no ability for co-ordinating multiple clients.
Also, Sonic provide one, but again limited functionality.
JMeter also, but again limited functionality and no co-ordination of multiple clients.
The one I've knocked together works something like the following:
A simple server runs at all the machines that you want to use as JMS clients. This server uses JBoss remoting to listen for requests.
Each server can run multiple jobs.
A job is one of:
Sender
------
A sender send messages to a destination.
For a sender the following params are configurable:
Number of jms connections to use
Number of jms sessions to use (connections are allocated to sessions on a round robin basis)
Ramp delay (delay between starting threads when there is more than one thread)
Run length - how long to run the sender for
Destination - the jms destination
Transacted - whether the sending session is transacted
Transaction size - if transacted then how many sends per tx
Anon - Use an anonymous producer?
Message Size- message size
Message type - any of the JBoss message types, plus a foreign message
Delivery mode - persistent or non persistent
Receiver
--------
A receiver receives messages from a destination
For a receiver the following params are configurable:
Number of jms connections to use
Number of jms sessions to use (connections are allocated to sessions on a round robin basis)
Ramp delay (delay between starting threads when there is more than one thread)
Run length - how long to run the sender for
Destination - the jms destination
Transacted - whether the sending session is transacted
Transaction size - if transacted then how many acks per tx
Acknowledgement Mode
Durable subscription name
Message Selector
No Local?
Use message listener rather than synch. receive?
Browser
-------
A browser job repeatedly browses messages in a destination (TODO)
A server can run any number of jobs at any one time
The server measures the throughput in msgs/s for each job at predetermined intervals.
Another application, the controller is responsible for sending requests to the servers.
Requests are as follows:
Start job on server
Stop job on server
Get current data for job on server (only throughput currently supported).
Throttle a send job.
Depending on the particular test to be done, the controller communicates remotely with the servers telling them to start or stop jobs.
E.g. the controller could tell servers A, B and C to start send jobs to queue Q, and tell server D to start a receive job on queue Q.
The controller would then intermittently communicate with the servers to get current throughput (msgs sent/received) for each server.
If we want to measure max. throughput of the server for a particular configuration of senders and receivers, then we really need to reach a steady state, i.e. messages aren't being sent faster then they are being received causing them to build up on the server.
Although this isn't currently implemented, this could be achieved by the server noticing that the msgs received per sec from the receivers is lower then the msgs sent per sec from the senders and telling the senders to slow down (throttle them). (and vice versa).
In such a way we should home in on the steady state.
Another important test IMHO is to test peak send rate. This is the max. number of sends the system can cope with when the sender is suddenly unthrottled from a steady state. This tests the systems ability to deal with spikes.
(What other metrics are we interested in?)
We need to think about what tests and configurations we're really interested in.
Also, currently data is just persisted to a CSV file.
It would be nice to store it in a database somewhere so a reporting tool can generate graphs and reports from it.
-Tim
