Productization metrics for JEMS Matrix

[wrzep]

Hello,


In the JEMS Maturity Matrix we would like to have measures describing software productization. I'm not entirely sure what should be listed here, but I've got some ideas and would appreciate your comments.

I like the following, short article about productization:
Effective productization in open source projects

Productization means turning the software into a "whole product" -- providing all the pieces that are necessary for mainstream users to get value from the software. This includes documentation, support and training. It also involves some type of branding (how people identify the product, like logos and company image).

Let's say we are measuring single project, mainly from user's point of view. We can consider:

- available documentation (manuals, wiki, FAQ, javadoc),
- professional support,
- training,
- if it is packaged for download,
- easy installation (web installer or installation manual),
- JEMS integration,
- user forums (?, as a kind of support),
- project version.

These are mainly booleans, but considered together they can give you some sense of how mature project is.

Metrics listed above could take necessary data from the project.xml files. Some info is already present (like JEMS integration), rest would be easy to include (like links to docs) and could be useful in the other contexts as well.

Also, project branding was mentioned above. It seems hard to measure how people recognize or appreciate the product. Idea that came to my mind was kind of google metric (for instance, how many web pages link to the product page), but it might be difficult to do it fairly.

Project popularity could be also measured with number of website visits. Or, maybe, we should introduce metrics based on polls?

I wonder what's your opinion.


Cheers,

[unibrew]

Hello

Another variable in this productization statistics could be how active the project is. It could be based for example on the amount of jira tasks resolved for a month. Or maybe on that how often new versions/upgrades/or whatever are released.
Moreover, activity on user forum and downloads per month might be good statistics.

Maybe some predefined polls as you mentioned. With questions like: "Do you consider this project a good product?"

This is mainly documentation but product should have trailblazers, tutorials and other helpful materials for people who want to start using project fast, without reading tons of manuals.

Cheers
------------------------
Ryszard Kozmik
JBoss Labs Team

[wrzep]

Thank you, Rysiek.

"unibrew" wrote:

Another variable in this productization statistics could be how active the project is. It could be based for example on the amount of jira tasks resolved for a month. Or maybe on that how often new versions/upgrades/or whatever are released.
Moreover, activity on user forum and downloads per month might be good statistics.

Yeah, stats you mentioned are really valuable and we are considering them now. These are actually community activity metrics (compare JMM mockup proposed by Damon).


Perfectly, we would like to have 3 groups of productization metrics. Let's say:

- how easy is to start using product (Please consider name contest as open ;-) )
This would include "packed for download" metric and measuring available project documentation.

- support
Here I mean professional support services, training and (optionally) activity on user's forums.

- project popularity / customer satisfaction


The point is, how to measure it.

"unibrew" wrote:

Maybe some predefined polls as you mentioned. With questions like: "Do you consider this project a good product?"

This is mainly documentation but product should have trailblazers, tutorials and other helpful materials for people who want to start using project fast, without reading tons of manuals.

Yes. It came to my mind that polls could be useful not only measuring how product meets users' requirements (customer satisfaction), but these are also a good place to ask them how documentation is helpful (the first group of metrics).

What do you think?
How about google metric? Seems tempting to measure how popular project is using google search.

Cheers,

[dsicore]

I think adding those booleans to the project.xml is a good place to start, and each of the booleans mentioned seem reasonable. Let's start with those.

Ideally, we'd have a portlet that product managers can use to set these values and maybe track productization for each project. Specifically, product managers are associated with each project--or rather, those projects that can be productized. We need to make it easy for them to add productization data. They won't be keen on using xml to do this. This is something we'll have to build an interface for. And, like the contributor's agreement portlet, we'll have special product manager roles which enable the portlet.

[unibrew]

Ok. So, Pawel could you make a list of new things which you need in project.xml ? Some time ago I sent you all a specification of new project.xml. Please check it and list what do you want to add.

Cheers
------------------------
Ryszard Kozmik
JBoss Labs Team

[wrzep]

"unibrew" wrote:
Ok. So, Pawel could you make a list of new things which you need in project.xml ? Some time ago I sent you all a specification of new project.xml. Please check it and list what do you want to add.

OK, I'd like to include in project.xml:

- support and forums links:

<project>
 ...
 <support-services type="support">http://</support-services>
 <support-services type="training">http://</support-services>
 <forum type="user">http://...</forum>
 <forum type="dev">http://...</forum>
 ...</project>

To make those links visible inside project menu, one would put the following in the menu tag:

<menu>
 ...
 <support-services />
 <forums />
</menu>

Links to wiki, repo and JIRA work now in this way (Rysiek, plase update your project.xml schema here).


- documentation info:

<project>
 ...
 <documentation type="FAQ">http://...</documentation>
 <documentation type="manual">http://...</documentation>
 ...
 </project>

(types could be: manual, wiki, FAQ, javadoc, quickStart, installation)

Project class will have getDocumentation() method then.

Displaying it in the same way as above can be useful, even though it might be sometimes not enough, for example when project has it's own, separate documentation page (side note: AFAIK, wiki link can be generated automatically).


- additional download info

It would be cool to know if downloads defined in project.xml include binaries or sources ready to install. The simplest solution is to add an optional type attribute:

<files>
 ...
 <file>
 <id>source</id>
 <name>Source Beta Version</name>
 <description>source distribution</description>
 <type>source</type>
 </file>
</files>

That's only a proposal, of course :)

Cheers,