JEMS Matrix in Development Dashboard.

[dsicore]

In the Development Dashboard we need a new portlet which is a "matrix" representation of a given project's current status.

The first version of this matrix should consist of at least the following rows:

1. Development Metrics such as: downloads, number of lines of code, commits per day/week, number of forum posts, etc.

2. Current QA status: Test Code Coverage, Flag indicating QA actually exists

3. JEMS Product Management Status: Whatever flags that Shaun Connolly says should be here... we need to coordinate with him.

So... to lay this out:

Add a portlet to the Development Dashboard....

That portlet has a list of projects.... with their current "score"

Choose a project by clicking on one on the list...

This will give you their "JEMS Matrix" which is a list of scores for each row.

We need to make the matrix configurable by row and column... so a "Development Row" should be configurable... as should any other "Row"...

Let's start the discussion. :)

Please see:

http://jira.jboss.com/jira/browse/JBLAB-415

[wrzep]

Well, the point is what should be on the "main" page of this portlet, and what should be shown on the page specific to each project.
So.. we will have a matrix like http://labs.jboss.com/portal/index.html?ctrl:id=window.default.JBossJiraMonitoring&ctrl:type=render with all these downloads, posts and so on and after selecting project we display it's JEMS Project Matrix...
But I've no idea what should be included in JEMS Project Matrix. What flags?

Pawel

[adamw]

Hello,
On the main page, I think we could put project's "ranking", calculated basing on some formula (for ex. sqrt(number of lines of code) + (number of downloads)^3 / commiters ^ 2 :) ). I think it would be good to divide this into plugins, and then each plugin could generate a set of rows - plus, projects could configure behaviour of these plugins basing on some file in the CMS (per project file).
To do some parts we could use existing things:
- kosmos does svn repo traversing/ gatheres jira stats. As it has a "server" and a "portlet", I guess we could try reading data from the server
- number of downloads - Rysiek is just finishing something similar - monitoring downloads of specified paths in a project. As far as I know he stores this data in forge cache, so it should be easily readable too.

Also such things as searching for a project with most code lines etc would be fun ;).

--
Cheers,
Adam

[szimano]

Labs Metrics Generation 2:

- show projects by number of code lines
- by number of comments
- by number of use @Inject annotations
- by number of the f... word as as variable
- by number of ... etc. :D

Seriously i like the ranking idea...

my way of formula is:
(sqrt(number of lines of code) + (number of downloads)^3 / commiters ^ 2)^(number of "Labs Team is great" occurrence in source code)

Cheers,
Tomek

[ivelin.ivanov]

The dashboard will be a very valuable tool for monitoring a project's maturity. Let's start with the list of metrics Damon points out and gradually expand.

Ideally there would be a way to assign weights to each data point and a function that produces a result between 0..100. The weights should be configurable per project.

Only a certain group of users should be allowed to edit these weights.

Ivelin

[shaun.connolly]

SourceForge has a way to calculate a project's activity percentile which is a blend of mail/forum activity, downloads, website hits, etc. This is one type of calculation worth creating....it can apply to new projects and older projects.

We can also have another calculation around quality (or something like that) which factors in productization types of metrics like outstanding bugs (with weighting based on criticality), quality and breadth of docs, demo apps/tutorial, etc. Basically all the stuff that enables people to have a quality experience with the technology.

Another factor that requires more brainstoring is impact on overall success of JEMS. Ex. the project works its way into a range of critical JEMS products or something like that...not sure how to do this actually, but fogured I;d toss out there to see if anybody has ideas.

[ryan.campbell]

Test coverage should be obtainable from cruisecontrol once project has implemented it. This will basically involve following links from a build report page like:

http://cruisecontrol.jboss.com/cc/buildresults/jms-head-testsuite

This will have the build timestamp, which we can then use to get the build artifacts like this one:

http://cruisecontrol.jboss.com/cc/artifacts/jms-head-testsuite/20051111090938/results/index.html

[ryan.campbell]

Also, each metric should have a timestamp, with policies for removing them from the dashboard after a certain period.

So if a project build is failing for a long period, the test coverage metric should eventually become invalid.

[aron.gombas]

Test coverage should be obtainable from cruisecontrol once project has implemented it.

Well, Kosmos has a dedicated CruiseControl portlet that support listing testcases per result (success/failure/error) and displaying this info in a piechart, too.
It might be worth to take a look at that.

[adamw]

"aron.gombas" wrote:

Well, Kosmos has a dedicated CruiseControl portlet that support listing testcases per result (success/failure/error) and displaying this info in a piechart, too.
It might be worth to take a look at that.

Does it already work remotely, that is, when the kosmos-server is on a different machine then cruise control (and thus, doesn't have access to the log files)?

[aron.gombas]

"adamw" wrote:
Does it already work remotely, that is, when the kosmos-server is on a different machine then cruise control (and thus, doesn't have access to the log files)?

No, it doesn't, as I still don't know what could be the right solution to detect if a new build was done. If the service has access to the logdir, then it can watch whether a new logfile appeared there. If the result is visible only online, then it seems harder...
Any good idea? Are the XMLs accessible in your pages, so should I grab the HTML and extract the XML for that? (A good idea here would give a lot of value to the applicability of that portlet.)

[ryan.campbell]

If you grab the "latest" link, you should be able to get the XML from that (2nd option on menu at top)

http://cruisecontrol.jboss.com/cc/buildresults/jms-head-testsuite

Its not raw XML, but you should be able to get what you need.

[ryan.campbell]

Ah, but cruisecontrol is down for maintenance this weekend, so you will have to wait.

[wrzep]

"aron.gombas" wrote:
Any good idea? Are the XMLs accessible in your pages, so should I grab the HTML and extract the XML for that? (A good idea here would give a lot of value to the applicability of that portlet.)

If you grab the "latest" link, you should be able to get the XML from that (2nd option on menu at top)

http://cruisecontrol.jboss.com/cc/buildresults/jms-head-testsuite

Its not raw XML, but you should be able to get what you need.

It would be really helpful to obtain test results from kosmos-server.
So, Aron, could you make Kosmos CruiseControl service work remotely?

Pawel

[aron.gombas]

"wrzep" wrote:
So, Aron, could you make Kosmos CruiseControl service work remotely?

Yep, I'll do this and let you know when it's ready.