Code Coverage in JBoss AS Test Suite

[ryan.campbell]

We have an urgent need for code coverage analysis in JBoss 4.2.0 specifically and the other branches (4.0 & trunk) in general.

I'd like to start a discussion on what coverage reporting tools we should use, and subsequently, the design of implementing code coverage in the JBoss testsuite.

The most popular code coverage tool as far as I can see is clover: http://www.cenqua.com/clover/. We should compare this with open source alternatives such as emma: http://emma.sourceforge.net/

The basic HTML reports look roughly approximate. I've asked Pavel to take a deeper look and come up with a recommendation. If anyone has any alternatives they've used in the past and would like to recommend, please comment.

[shelly.mcgowan]

The lastest version of EMMA - 2.1.5320 - introduces a new command line tool called ctl which allows coverage data to be captured remotely and prior to JVM exit. This works well in J2EE environments and is intended for such. Instrumentation still needs to be done prior to running tests (referred to as off-line mode) using EMMAs tool - instr. See:
http://emma.sourceforge.net/faq.html#q.runtime.appservers
which documents the few easy steps. I recently used EMMA to capture coverage data in J2EE environment but found it to be more useful when obtaining coverage metrics against a standalone component (.jar) to enhance the coverage as I developed tests.

[pavel.tsekov]

As I understand it the best way to capture coverage data in J2EE environments is by using offline instrumentation - custom class loaders won't work very well and as such runtime patching of the bytecode is not useful.

I want to start testing EMMA and Clover on some real code from the JbossAS 4.2 testsuite - would you recommend any particular tests that I should test the code coverage tools with ? The testsuite is quite big but it is split in different categories. I'd like to get one test of each category and try the tools on it, but it's not easy for me to pick the right tests.

[pavel.tsekov]

After smcgowan's comment I took a look at emma 2.1.5320 - I usually prefer to go with released code that's why I looked at 2.0.5312 first. What struck me is the fact that "2.1.5320" which is tagged "emma-testing" is year and a half old. I took the time to look at the emma forums at sf.net and the project's cvs repository. It seems like the project is not actively developed - the most recent changes in the CVS repository seem to be 10 months old. Also some of the comments on the forums indicate that the project is not being actively developed. In particular:

http://sourceforge.net/forum/forum.php?thread_id=1483955&forum_id=373867

http://sourceforge.net/forum/forum.php?thread_id=1536441&forum_id=373865

Nevertheless, emma looks to be a very good tool. I hope to come with
some real test results soon.

[ryan.campbell]

If you haven't selected some tests to focus on, you could try tests-jacc-security.