10 Replies Latest reply on Sep 23, 2011 5:05 AM by Matija Mazi

    TestNG Groups with Arquillian

    Tom Barlow Newbie

      Hi,

       

      I'm trying to separate my tests so that I can run my unit tests independent to my Arquillian integration tests.  It works fine if I add <excludes> patterns to the surefire plugin, but what I was hoping is to use TestNG's groups functionality.  I had noticed that the example pom.xml in the Arquillian git repository doesn't use TestNG groups either.

       

      I've tried it, and it seems to pick up that my Arquillian test is not included, or is included.  The issue then is that it encounters issues with the Arquillian class that it has extended.

       

       

      With Arquillian test excluded via TestNG groups through maven-surefire-plugin:

       

      Running TestSuite

      19-Sep-2011 22:32:20 org.jboss.arquillian.container.impl.client.container.ContainerRegistryCreator getActivatedConfiguration

      INFO: Could not read active container configuration: null

      Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 0.545 sec <<< FAILURE!

       

       

      Results :

       

       

      Failed tests:   arquillianBeforeSuite(com.acme.corp): Could not create a new instance of class org.jboss.arquillian.test.impl.EventTestRunnerAdaptor see cause

       

      With test included, and running in container:

       

      Failed tests:   arquillianBeforeClass(com.acme.corp): Could not invoke deployment method: public static org.jboss.shrinkwrap.api.Archive com.acme.corp.createDeployment()

        arquillianAfterTest(com.acme.corp)

        arquillianAfterTest(com.acme.corp)

        arquillianAfterClass(com.acme.corp)

       

      Any hints on getting this working are appreciated.

       

      Cheers.

        • 1. Re: TestNG Groups with Arquillian
          Aslak Knutsen Master

          Do you have more of the stack traces?

           

          The last one seems to simply imply that for what ever reason it can't create the ShrinkWrap archive...

          • 2. Re: TestNG Groups with Arquillian
            Tom Barlow Newbie

            Aslak,

             

            Thanks for the reply.  I'd been battling with this today, Jose in #jbosstesting managed to help me to sort out the last one.  This was done by updating the dependencies (from the kitchensink jboss archetype), from 7.0.0.CR1 to 7.0.1.Final.  Also ensuring that TestNG was at 6.1.1.

             

            There was also an issue that I came across that was covered in another post here, where I had to specify Servlet 3.0 in my arquillian.xml.

             

            The issue I am still seeing is that I have one unit test under group "unit" and an arquillian test under group "integration", when I run with just group "unit".  TestNG is trying to execute the arquillianBeforeSuite() method in the Arquillian class.

             

            Maybe this is a TestNG bug?

            • 3. Re: TestNG Groups with Arquillian
              Tom Barlow Newbie

              It seems this is causing the issue:

               

              In the Arquillian base class for a test:

               

                 @BeforeSuite(alwaysRun = true)

                 public void arquillianBeforeSuite() throws Exception

               

              The other Before/After methods have got the alwaysRun parameter also.

               

              The TestNG documentation:

               

              alwaysRun - For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will be run regardless of what groups it belongs to.

              • 4. Re: TestNG Groups with Arquillian
                Aslak Knutsen Master

                yea, they were added so the methods would always be called regardless of failure, e.g. BeforeXX fail, AfterXX should still be called. But they seems to have multiple 'strange' side effects..

                • 5. Re: TestNG Groups with Arquillian
                  Tom Barlow Newbie

                  OK so I guess this isn't going to get resolved anytime soon

                   

                  I've raised a JIRA here:

                  https://issues.jboss.org/browse/ARQ-605

                   

                  I think the workaround for now is probably to separate tests into a unit-testng.xml and arquillian-testng.xml or so, then use the surefire plugin to run those perhaps?

                  • 6. Re: TestNG Groups with Arquillian
                    Aslak Knutsen Master

                    i'll see if i find a soution that fits all usecases here and keep grouping with non Arq tests in mind.

                     

                    Either spit out the Arq tests in a separate module or use surefire fiters/suite files seems to be the options.

                    • 7. Re: TestNG Groups with Arquillian
                      Aslak Knutsen Master

                      Which level are you placing the group? pr Method or Class ?

                      • 8. Re: TestNG Groups with Arquillian
                        Matija Mazi Newbie

                        Hi,

                         

                        I have the same problem: I can't use sureFire's <excludedGroups> to exclude the Arquillian tests. This seems to me a very common use-case; I imagine many people want to put their Arquillian tests in a separate group and run them much less often than simple unit tests since they are slow and may require a server to be running.

                         

                        I'm seeing the problem with group definition at class level:

                         

                        @Test(groups = "in-container")

                        public class LogoutRedirectTest extends Arquillian { ... }

                         

                        Matija

                        • 9. Re: TestNG Groups with Arquillian
                          Tom Barlow Newbie

                          Tried it at both class and method level.  The issue is that annotation with the 'alwaysRun' parameter.  Has that side-effect that there must always be a container available.

                           

                          I agree with Matija here, this is a common use-case that would be helpful if resolved soon.

                           

                          The workaround relying upon exclusion via test suite XML files is very nasty, requires a lot more configuration as test config is done at the parent level on my project.  Therefore I need to dump empty xml files in each of the modules for projects that do not have Arquillian tests.

                          • 10. Re: TestNG Groups with Arquillian
                            Matija Mazi Newbie

                            I've done it now using exclusions by name in pom.xml:

                             

                            <?xml version="1.0" encoding="UTF-8"?>

                            <project>

                                <properties>

                                    <testng.excludedTests>**/*ArqTest.java</testng.excludedTests>

                                </properties>

                                <build>

                                    <pluginManagement>

                                        <plugins>

                                            <plugin>

                                                <artifactId>maven-surefire-plugin</artifactId>

                                                <configuration>

                                                    <excludes>

                                                        <exclude>${testng.excludedTests}</exclude>

                                                    </excludes>

                                                </configuration>

                                            </plugin>

                                        </plugins>

                                    </pluginManagement>

                                </build>

                                <profiles>

                                    <profile>

                                        <id>arq.jbossas-remote-7</id>

                                        <properties>

                                            <testng.excludedTests>**/XXX</testng.excludedTests>

                                        </properties>

                                    </profile>

                                </profiles>

                            </project>

                             

                            This seems to work for me but I would prefer the solution with groups because it would be cleaner and wouldn't require the naming convention that this config relies on (that all Arquillian test classes be named *ArqTest). The amount of config one has to write is the same with this as with groups, though.