Code Coverage Tools

[kennardconsulting]

Hi guys,

Just curious: what is the consensus on code coverage tools within JBoss?

I used to use Ant and Emma, and it gave me great, combined unit test and integration test coverage reports (http://metawidget.org/doc/coverage/index.html). But since moving to Maven/Jenkins a couple years ago I haven't found an equivalent? The closest I have found is Sonar and Jacoco but these don't give a combined report? There has been a bug report open against this for some time: http://jira.codehaus.org/browse/SONAR-2804

Are there other options?

Regards,

Richard.

[jpapouse]

Hi Richard.

Well, I am trying to set code coverage for RichFaces project consisting of a lot of modules. The way I've chosen is

1. execute tests with code coverage in each module using jacoco-maven-plugin -> this generates jacoco.exec file for each module
2. combine jacoco.exec files to one HTML report  using org.jacoco.ant.jar - this part is tricky, because you need to have access to
   • jacoco.exec
   • *.java source files
   • *.class compiled files

Regards,

Jan

[jpapouse]

Here is the project providing combination of jacoco.exec files from RichFaces modules into one HTML report - https://github.com/papousek/richfaces-code-coverage

[dan.j.allen]

I would think that the lack of a combined report is perhaps something that could be solved by an Arquillian extension (if not in the long run, then certainly as a short term fix). That extension would tie into the report generation part of the lifecycle.

[dan.j.allen]

Both Max and Jason agreed that jacoco is the way to go right now. The challenge, of course, is the lack of merged reports. I'm glad to see that Jan has proposed something to start playing with today.

[mickael_istria]

Jacoco in indeed the way to go: it provides coverage report without needing instrumentation and then without adding steps to build process. It is just about setting a -Djava.agent system property. No effect is visible on performance.

About reports, there are 3 ways to get some, AFAIK:

* Using the Ant task to generate HTML reports

* Using EclEmma Eclipse plugins to see reports in your IDE and editor

* Using Sonar, which supports jacoco.exec files to integrate reports in its quality report dashboard. http://www.sonarsource.org/

IMHO the best way to go for JBoss is to set up a Sonar instance (Sonar is a quality analysis and reporting tools, well integrated with Jacoco, Jenkins and Maven - able to report coverage, checkstyle, findbugs...). It is a really useful tools for developers and really helps in improving code and fighting technical debt.

[silenius]

And since version 0.5.6 jacoco-maven-plugin can be used together with maven-site-plugin.

http://www.eclemma.org/jacoco/trunk/doc/changes.html

It will be available in the next version of Arquillian Jacoco Extension:

https://issues.jboss.org/browse/ARQ-738

[mickael_istria]

A very nice step ahead for usage of Jacoco would be the Jenkins-Jacoco plugin to see Jacoco reports in Jenkins:

https://issues.jenkins-ci.org/browse/JENKINS-10835 -> You can vote for this issue, hoping it will motivate Jenkins guys.

[dan.j.allen]

Perhaps having even more weight than voting on the issue, there is an open GSoC proposal in the JBoss Community that involves fixing this plugin.

"Add a solution for creating series of snapshots of Java coverage data over time, so the developer can track the code coverage trends and  have more information about the project. Currently the Jenkins plugin is not ready. I would like to make some improvements to bring the software to a more stable version and to finish the switching from the Emma plugin."

I've put in my vote for the project to move forward, but we'll find out if it makes the cut in a week or so.

[mickael_istria]

Good news!

The Google Summer of Code about Jacoco reports in Jenkins was accepted: http://www.google-melange.com/gsoc/project/google/gsoc2012/ognjen/25001

It will be done by Ognjen Bubalo and is mentored by Jonathan Fuerth.

A dedicated mailing-list will appear soon, I'll post the link.

[dan.j.allen]

Very good news indeed! Congratulations Ognjen!

[ognjenb]

Thank you!

So as Mickael said, we created a mailing list.

The URL: https://groups.google.com/forum/?fromgroups#!forum/jenkins-jacoco-plugin-mailing-list

The Mailing list: [e-mail]

Please join us!