10 Replies Latest reply on Jul 10, 2009 11:31 AM by kapitanpetko

    SeamTest coverage (maven+cobertura)

    kapitanpetko

      Hello,


      I have a maven ear project (based the attchement at on http://jira.jboss.org/jira/browse/JBSEAM-2371).
      The structure is as follows:


        project
          project-ejb
          project-web
          project-ear
          project-common
      



      I have a bunch of SeamTest's in project-web that test Actions and EJB's in project-ejb.
      (having the tests in the web project makes it a lot easier to get the whole embedded setup
      running, see above link ) The test run OK (it was a bitch to get there...) but since the
      actions and EJBs are in the project-ejb.jar, I get no coverage.


      Does anyone have an idea how to get and coverage for the actions and EJBs (in project-ejb)
      when running the SeamTest's (in project-web)? An example would be great, but any ideas are
      welcome. BTW, simply unjarring the project.ejb.jar in project-web/target-classes does not work.

        • 1. Re: SeamTest coverage (maven+cobertura)

          Cobertura has to instrument your classes in order to capture code coverage data.  When used with Maven and executing the command:


          mvn cobertura:instrument



          a directory is created for each sub-project: 


          \target\generated-classes\cobertura



          These are the classes that must be in your classpath to generate code coverage analysis.  You could try jarring up the generated classes and using the instrumented jars when running SeamTest.  Should work, but I haven't tried it so no guarantees ;)


          Hope that helps.

          • 2. Re: SeamTest coverage (maven+cobertura)
            kapitanpetko

            Thanks. I'll try that tomorrow (or is it today already...)


            I was thinking that classes have to be instrumented 'in place',
            that is within the web project. That doesn't seem to be the case :)


            Sound like it should work, I'll just need to come up with some script (ant?) to get it automated.


            • 3. Re: SeamTest coverage (maven+cobertura)

              Yes, Ant is probably going to be your best bet for the script.  Let me know how it goes.  I'm sure the community would be quite appreciative if you wanted to contribute the script. I know I would :)

              • 4. Re: SeamTest coverage (maven+cobertura)

                You could try to redirect output of cobertura to the regular target/classes, see configuration of the cobertura-maven-plugin but then you will have danger of packaging instrumented classes to your production packaging. Even then instrumented classes will need to be in a jar, I am not sure whether cobertura can handle instrumented classes from jars...


                To be honest cobertura (at least if run via maven plugin) is not really a integration test coverage tool (but a 'simple' unit test coverage tool). I think you might have a look at clover for integration tests coverage (in case you can afford to pay or you're busy with OSS).


                There were some talks on maven mail lists about enabling integration testing but afaik the discussions didn't go far...


                For more reference check this and this out.


                I have a feeling this whole situation with maven may require writing a special seam plugin (too many issues)...

                • 5. Re: SeamTest coverage (maven+cobertura)
                  kapitanpetko

                  I managed to get it to work following Jacob's suggestion. Thanks a lot. I haven't had the time to automated (next week, hopefully...), but here are the steps I took:



                  1. execute the test and take coverage in project-ejb

                  2. jar the files in project-ejb/target/generated-classes/coberutra

                  3. overwrite the ejb jar int .m2/repository with the jar from step 2

                  4. copy project-ejb/target/cobertura/* to project-web/target

                  5. test project-web



                  mvn test
                  



                  6. generate coberura report in project-web


                  mvn -Dmaven.test.skip=true cobertura\cobertura
                  



                  Steps 5 and 6 are necessary because coberutra:cobertura runs the test twice and the second time they fail. It seems the classpath differs(?) and jboss embedded fails to start; haven't had the time to dig into it, but might be related to http://jira.codehaus.org/browse/SUREFIRE-257?


                  Thanks again.



                  • 6. Re: SeamTest coverage (maven+cobertura)
                    kapitanpetko

                    Siarhei Dudzin wrote on Apr 23, 2008 09:14 PM:


                    You could try to redirect output of cobertura to the regular target/classes, see configuration of the cobertura-maven-plugin but then you will have danger of packaging instrumented classes to your production packaging. Even then instrumented classes will need to be in a jar, I am not sure whether cobertura can handle instrumented classes from jars...


                    It seems instrumented jars work OK, so I might try that out. Should be cleaner that messing with ant. I could have a separate profile for testing and that should eliminate the danger of packaging instrumented classes in prodcution.



                    To be honest cobertura (at least if run via maven plugin) is not really a integration test coverage tool (but a 'simple' unit test coverage tool). I think you might have a look at clover for integration tests coverage (in case you can afford to pay or you're busy with OSS).



                    We have used clover before (the very first release) and it worked OK. I guess we could afford it but that could take some time. Anyways, thanks for the suggestion, I'll check it out. (sometime...)



                    I have a feeling this whole situation with maven may require writing a special seam plugin (too many issues)...


                    That would be really nice. Seam is great but testing is ..., well, not for the faint of heart :)


                    BTW, thanks for the sample project at JBSEAM-2371, it was huge help!

                    • 7. Re: SeamTest coverage (maven+cobertura)
                      kapitanpetko

                      This is quite an old thread, but since I've finally had a bit of time to look into it, I thought I'd share my findings.


                      The maven cobertura task is not flexible enough to handle multiple projects (there is no way to specify source folders, aggregate coverage, etc.; there are a few pending FR's about that), so I ended up using the ant tasks in my maven project. I defined a profile that aggregates the ejb module coverage with the SeamTests coverage (in the web module) and creates a report.


                      How to use:



                      1. Run the tests and take coverage of the ejb-project (mvn cobertura:cobertura)

                      2. Go to the web project dir and execute mvn -Pcoverage integration-test

                      3. The coverage report is output to project-web/target/site/cobertura



                      Here is the profile:


                       <profile>
                            <id>coverage</id>
                            <build>
                              <plugins>
                                <plugin>
                                  <artifactId>maven-antrun-plugin</artifactId>
                                  <dependencies>
                                    <dependency>
                                      <groupId>net.sourceforge.cobertura</groupId>
                                      <artifactId>cobertura</artifactId>
                                      <version>1.9</version>
                                    </dependency>
                                  </dependencies>
                                  <executions>
                                    <execution>
                                      <phase>process-test-classes</phase>
                                      <id>instrument-ejb-jar</id>
                                      <configuration>
                                        <tasks>
                                          <taskdef classpathref="maven.runtime.classpath"
                                            resource="tasks.properties" />
                                          <mkdir dir="${project.build.directory}/cobertura" />
                                          <copy
                                            todir="${project.build.directory}/cobertura">
                                            <fileset
                                              dir="../project-ejb/target/cobertura" />
                                          </copy>
                                          <copy
                                            file="${settings.localRepository}/org/foobar/project-ejb/${project.version}/project-ejb-${project.version}.jar"
                                            tofile="${settings.localRepository}/org/foobar/project-ejb/${project.version}/project-ejb-${project.version}.jar.org" />
                                          <cobertura-instrument>
                                            <includeClasses regex=".*" />
                                             <excludeClasses
                                              regex=".*\DontLikeInstrumentation.*" />
                                            <instrumentationClasspath>
                                              <pathelement
                                                location="${settings.localRepository}/org/foobar/project-ejb/${project.version}/project-ejb-${project.version}.jar" />
                                            </instrumentationClasspath>
                      
                                              <!-- or use this:-->
                                     <!--
                                            <fileset
                                              dir="${settings.localRepository}/org/foobar/project-ejb/${project.version}">
                                              <include
                                                name="project-ejb-${project.version}.jar" />
                                            </fileset>
                                            -->
                                          </cobertura-instrument>
                                        </tasks>
                                      </configuration>
                                      <goals>
                                        <goal>run</goal>
                                      </goals>
                                    </execution>
                                    <execution>
                                      <phase>integration-test</phase>
                                      <id>cobertura-report</id>
                                      <configuration>
                                        <tasks>
                                          <taskdef classpathref="maven.runtime.classpath"
                                            resource="tasks.properties" />
                                          <mkdir
                                            dir="${project.build.directory}/site/cobertura" />
                                          <cobertura-report format="html"
                                            datafile="${project.build.directory}/cobertura/cobertura.ser"
                                            destdir="${project.build.directory}/site/cobertura">
                                            <fileset dir="${basedir}/src/main/java">
                                              <include name="**/*.java" />
                                            </fileset>
                                            <fileset
                                              dir="../project-ejb/src/main/java">
                                              <include name="**/*.java" />
                                            </fileset>
                                          </cobertura-report>
                      
                                          <copy overwrite="true"
                                            file="${settings.localRepository}/org/foobar/project-ejb/${project.version}/project-ejb-${project.version}.jar.org"
                                            tofile="${settings.localRepository}/org/foobar/project-ejb/${project.version}/project-ejb-${project.version}.jar" />
                                        </tasks>
                                      </configuration>
                                      <goals>
                                        <goal>run</goal>
                                      </goals>
                                    </execution>
                                  </executions>
                                </plugin>
                              </plugins>
                            </build>
                          </profile>
                      

                      • 8. Re: SeamTest coverage (maven+cobertura)
                        sjmenden

                        Mind adding that to the knowledge base?

                        • 9. Re: SeamTest coverage (maven+cobertura)
                          kapitanpetko

                          Samuel Mendenhall wrote on Aug 22, 2008 05:05:


                          Mind adding that to the knowledge base?


                          Done.


                          wiki://38091

                          • 10. Re: SeamTest coverage (maven+cobertura)
                            kapitanpetko

                            I added a more robust and (almost) no Ant solution to the KB article.