QA Metrics for JEMS Matrix

[wrzep]

Hello,


After discussing measuring productization, I'd like to focus on QA metrics.


From the beginning of JMM, statistics from CruiseControl seemed as the most valuable.

Test coverage should be obtainable from cruisecontrol once project has implemented it. This will basically involve following links from a build report page like:
http://cruisecontrol.jboss.com/cc/buildresults/jms-head-testsuite

Using Kosmos we can get the latest build stats from CruiseControl like number of tests and number of errors. These could be useful metrics, but to make them more meaningful I'd consider measuring number of errors (or tests) per number of code lines (or, perhaps, thousands of code lines). These metrics are already implemented :)


The point is, this affects only the latest build. Overall statistics are also interesting: http://cruisecontrol.jboss.com/cc/buildresults/jbosside?tab=metrics.

We can obtain from there:
- Number of Build Attempts,
- Number of Broken Builds and
- Number of Successful Builds.

This is what I'd like to include in JMM as well.

All in all, we've got the latest build stats and total stats. Having one more category of metrics would be cool.

What do you think?

Cheers

[dsicore]

Sounds good, but our JBoss Labs build will have the worst statistics ever. ;)