TestSuite - Coverage

[maeste]

Hi all,

From a coverage point of view our test suite is quite good. The question is how much coverage is our target, but much more important which package/module should have top priority in terms of test coverage.

ant cobertura

generate the coverage report into reports/cobertura/html/index.html. Please have a look to get an idea of current status.

As you can see there we have a good coverage in our implementations, while we are not testing a lot in an abstract way (using interfaces from javax) instead of stress the real implementation. It's a choice of course: it makes test more readable and much more linked to real implementation. Of course a more abstract tests would verify that we are providing correctly what the specs require. Of course the TCK is doing that for us, so we can leave tests stressing our own implementation safely.

Anyway in both cases, IMHO we should fine tune cobertura report to exclude from the reports classes and interface we are not stressing for a choice, and maybe (I'm not sure it's possible, I'm verifying) also method/classes that doesn't make sense to stress. I'm thinking about for example to pojo geteer and setter that is dropping down the finale coverage percentage wrongly.

let me know your thought and eventually the priority you would assign to not-well covered packages.

best regards

S.

[jesper.pedersen]

Well, there is a bit of documentation regarding out testing here: http://http://docs.jboss.org/ironjacamar/developerguide/en/html/testing.html - which of course could be expanded. Especially the "split" of test cases described in 5.1.x is important to keep in mind.

As to the most important parts - coverage of the deployment metadata is important from an end-user point of view, so we are sure that it works, and that we can provide examples for the users on how to configure their resource adapter / deployment.

From a container point of view - the most important part would be the connection manager (and friends) - especially the local / XA transaction based one.

I agree that there may be certain classes / methods that don't need full coverage - however, I don't think it is currently possible to exclude classes from testing once the JAR file is instrumented. There is a problem with Cobertura also in this area - that is why we have to copy the core-api and core-spi over in their "clean" version.