1 2 Previous Next 20 Replies Latest reply on Nov 21, 2012 4:35 PM by dan.j.allen

    Default container/group in arquillian.xml enhancements

    kpiwko

      Hi All,

       

      while implementing Lukas Fryc's new configuration pattern for better IDE support for Arquillian, I've noticed that current mapping between container and container's configuration is a pita.

       

      The pattern allows maven-resources-plugin to replace placeholders in arquillian.xml with values specified as properties in pom.xml file. The gain is clear here, you can change test configuration by simply selecting a different profile. Because IDEs generally avoid touching surefire configuration - which is imho correct this is a build test configuration, not a single "Run As JUnit" configuration - recopying all surefire properties into test execution settings does not bring you a nice user experience.

       

      We are aming to something like:

       

      Select Maven profiles.png

       

      -Parq-jbossas-remote,webdriver-android compatible from both cmd line and IDE

       

      Here are the problems with current approach:

      • If you do not set default="true" for even for a single container, nothing is loaded and takes some time until user realizes what is wrong there
      • You cannot select an active configuration easily due to boolean true/false logic

       

      I've created an extension at https://github.com/kpiwko/arquillian-extension-container-selector, which allows you to select active configuration per container using container qualifier, disabling previous behavior. I think it should be a part of default arquillian.xml possibilities. We have <defaultProtocol>, so we should have <defaultContainer> tag as well.

       

      Would you like to have such stuff in Core (https://issues.jboss.org/browse/ARQ-1096)? If so, how many release cycles we need in order for it to happen, I guess 1.0.3.Final is a no-go, right?

        • 1. Re: Default container/group in arquillian.xml enhancements
          kpiwko

          The approach with filtering arquillian.launch which contains a name of container to be activated works as well.

          • 2. Re: Default container/group in arquillian.xml enhancements
            dan.j.allen

             

            while implementing Lukas Fryc's new configuration pattern for better IDE support for Arquillian, I've noticed that current mapping between container and container's configuration is a pita.

             

             

            I totally agree. I've ranted about this problem on many nights in the bar (and elsewhere). So +1 to working on a solution to the problem and eventually integrating it into core. I'm behind you

             

            The argument I typically make in my rant is, "If I can clearly see a choice for configuration given my container adapter, why can't Arquillian figure it out?" For instance, if I have a configuration for JBoss AS and a configuration for GlassFish, when I use the GlassFish container adapter, it seems pretty silly for Arquillian to be attempting to use the JBoss AS one (same goes for the managed, embedded, remote variants).

             

            At one point I proposed making an extension point that allows the user to insert container selection logic. For instance, you could load a rules file to work out which container should be selected.

             

            However, one thing is for absolute certain. Container adapters desperately need globally unique identifiers (GUIDs)! When I pull out a DeployableContainer from the context, I poke at it from all sides unable to find some logical and reliable way to determine which adapter I have.

             

            The following IDs would make a whole lot of sense if available on the respective DeployableContainers:

             

            jbossas-remote-7

            jbossas-managed-7

            glassfish-remote-3.1

            glassfish-managed-3.1

             

            We may have to iron out how best to handle the versions (both container version and adapter version). But I'd settle for something!

            • 3. Re: Default container/group in arquillian.xml enhancements
              kpiwko

              I totally agree here. Having unique qualifier per container is something users are desperately looking for for some time.

               

              Often it happen this way: Hey, I have a container qualifer="jboss" but Arquillian is not picking that configuration. What name should I use instead? People don't get "any name" situation with are in right now.

               

              We have done some effort in Drone to have fixed qualifier per configuration. If you need more WebDrivers, we allow you to use extra CDI-like @Qualifier annotation and extra configuration in xml/system properties, determined by original qualifer + qualifier annotation name.

              As Aslak merged configuration retrieval into Arquillian Core, so having the same system for containers makes perfect sense to me.

              • 4. Re: Default container/group in arquillian.xml enhancements
                dan.j.allen

                The same proposal I suggested in the thread about extension configuration applies here as well. We should have a global unique identifier for each container adapter (call it a canonical name), and then use qualifier to differentiate between multiple configurations of that same container adapter (whether they used in tandem in the same run or for different runs).

                 

                Here's an example for configuring the JBoss AS remote adapter for dev, qa and staging.

                 

                <container name="jbossas-remote-7.1" qualifier="dev" default="true">
                    <property name="managementAddress">dev-machine</property>
                </container>
                
                <container name="jbossas-remote-7.1" qualifier="qa">
                   <property name="managementAddress">qa-machine</property>
                </container>
                
                <container name="jbossas-remote-7.1" qualifier="staging">
                   <property name="managementAddress">staging-machine</property>
                </container>
                
                <container name="glassfish-remote-3.1" qualifier="dev" default="true">
                    <property name="adminHost">dev-machine</property>
                </container>
                
                ...
                

                 

                If you are using the JBoss AS Remote 7.1 adapter, the only possible options are the configurations with the name attribute equal to jbossas-remote-7.1. No other configurations even make any sense. If no qualifier is specified, the one in that set marked default is the one selected. If a qualifier is chosen, then obviously that one is selected instead of the default one.

                 

                This way, if you are staying in the dev (default) environment, and you are changing adapters, it requires no special launch configuration! This eliminates a major pain point in Arquillian.

                 

                To get there, however, we have to add canonical names to the containers. As a stopgap measure, we can maintain an internal registry that is consulted if a container adapter does not provide an identifier. I'm feeling very good about this solution.

                • 5. Re: Default container/group in arquillian.xml enhancements
                  dan.j.allen

                  Btw, I also propose renaming arquillian.launch to one of the following alternatives:

                   

                  • arquillian.profile
                  • arquillian.qualifier
                  • arquillian.env

                   

                  Obviously, qualifier makes the most sense, but developers are most accustomed to profile.

                  • 6. Re: Default container/group in arquillian.xml enhancements
                    dan.j.allen

                    Also, the extension .launch has special meaning in Eclipse, so we are throwing it off by having that file in the project. I'm in support of eliminating that file since use of it becomes very rare if we can intelligently select containers (and we also get support for controlling this parameter in the Eclipse Arquillian plugin).

                    • 7. Re: Default container/group in arquillian.xml enhancements
                      aslak

                      Dan Allen wrote:

                       

                      We should have a global unique identifier for each container adapter (call it a canonical name), and then use qualifier to differentiate between multiple configurations of that same container adapter (whether they used in tandem in the same run or for different runs).

                       

                      It being an GUID the XML attribute should probably be called id and not name..

                      • 8. Re: Default container/group in arquillian.xml enhancements
                        dan.j.allen

                        Aslak Knutsen wrote:

                         

                        Dan Allen wrote:

                         

                        We should have a global unique identifier for each container adapter (call it a canonical name), and then use qualifier to differentiate between multiple configurations of that same container adapter (whether they used in tandem in the same run or for different runs).

                         

                        It being an GUID the XML attribute should probably be called id and not name..

                         

                        I'm fine with that. No objection either way, but I do agree id would be doing it right.

                        • 9. Re: Default container/group in arquillian.xml enhancements
                          kpiwko

                          In that case, we should consider changing <extension qualifier="foobar" />, resp. <extension qualifier="foobar-dev" into <extension id="foobar" /> , resp. <extension id="foobar" qualifier="dev" />

                           

                          Edit: renaming arquillian.launch into arquillian.qualifier collides with multinode tests, e.g.:

                           

                          <container id="jboss-XYZ" qualifier="node1" >

                          ...

                          </container>

                           

                          <container id="jboss-XYZ" qualifier="node2" >

                          ...

                          </container>

                          • 10. Re: Default container/group in arquillian.xml enhancements
                            dan.j.allen

                            Karel Piwko wrote:

                             

                            In that case, we should consider changing <extension qualifier="foobar" />, resp. <extension qualifier="foobar-dev" into <extension id="foobar" /> , resp. <extension id="foobar" qualifier="dev" />

                             

                             

                            +1

                             

                            Shall I write up an issue (as a story card) for this?

                             

                             

                            Edit: renaming arquillian.launch into arquillian.qualifier collides with multinode tests, e.g.:

                             

                            <container id="jboss-XYZ" qualifier="node1" >

                            ...

                            </container>

                             

                            <container id="jboss-XYZ" qualifier="node2" >

                            ...

                            </container>

                             

                            Ah, so we would need plural? Then it would be arquillian.qualifiers. Actually, to be nice to Java, we might instead want to use a properties file for this.

                             

                            arquillian-activation.properties

                            node1=true
                            node2=true
                            node3=false
                            

                             

                            That is infinitely clearer (and it will make Max happy). You also don't have to comment out lines.

                             

                            Alternatively, we could just use a single property (leaving room for other properties in the future)

                             

                            arquillian-activation.properties

                            active.qualifiers=node1,node2
                            

                             

                            (or just "qualifiers")

                             

                            I think I like that better.

                            • 11. Re: Default container/group in arquillian.xml enhancements
                              kpiwko

                              Shall I write up an issue (as a story card) for this?

                               

                               

                              Yes, definitely.

                               

                               

                              Alternatively, we could just use a single property (leaving room for other properties in the future)

                               

                              arquillian-activation.properties

                              active.qualifiers=node1,node2
                              

                               

                              (or just "qualifiers")

                               

                              I think I like that better.

                               

                              I like that better as well. The only thing I'm missing are the rules how to autoactivate qualifiers in case arquillian-activation.properties is missing, which would be 95% of the cases.

                               

                               

                              @RunWith(Arquillian.class)
                              public class MethodSpecificDriverTestCase {
                              
                                  @Test
                                  @InSequence(1)
                                  // here we want to use <extension id="webdriver" qualifier="methodspecific" >
                                  public void simpleWebdriverTest(@Drone @MethodSpecific FirefoxDriver webdriver) {
                                      LoginPage page = new LoginPage(webdriver);
                                      page.login("demo", "demo");
                                      page.logout();
                                  }
                              
                                  @Test
                                  @InSequence(2)
                                  // here we want to use <extension id="webdriver" >
                                  public void simpleWebdriverChromeTest(@Drone ChromeDriver webdriver) {
                                      LoginPage page = new LoginPage(webdriver);
                                      page.login("demo", "demo");
                                      page.logout();
                                  }
                              }
                              
                              

                               

                              Seems to me that qualifiers from arquillian-activation.properties does not match the story for extensions. We need to solve whether we want to prefer code over properties or the other way around.

                               

                              Also, for multinode tests, suppose the following:

                               

                               

                              @RunWith(Arquillian.class)
                              public class ClusteredCacheTestCase
                              {
                                 @Deployment(name = "dep.active-1")
                                 @TargetsContainer("container.active-1")
                                 public static WebArchive createTestDeployment()
                                 {
                                    return Deployments.createActiveClient("test-1.war");
                                 }
                              
                                 @Deployment(name = "dep.active-2")
                                 @TargetsContainer("container.active-2")
                                 public static WebArchive createTestDeployment2()
                                 {
                                    return Deployments.createActiveClient("test-2.war");
                                 }
                              
                                 @Deployment(name = "dep.active-3")
                                 @TargetsContainer("container.active-3")
                                 public static WebArchive createTestDeployment3()
                                 {
                                    return Deployments.createActiveClient("test-3.war");
                                 }
                              
                                 @Inject
                                 private Cache<String, Integer> cache;
                              
                                 @Test @InSequence(1) @OperateOnDeployment("dep.active-1")
                                 public void callActive1() throws Exception
                                 {
                                    int count = incrementCache(cache);
                                    System.out.println("Cache incremented, current count: " + count);
                                    Assert.assertEquals(1, count);
                                 }
                              
                                 @Test @InSequence(2) @OperateOnDeployment("dep.active-2")
                                 public void callActive2() throws Exception
                                 {
                                    int count = incrementCache(cache);
                                    System.out.println("Cache incremented, current count: " + count);
                                    Assert.assertEquals(2, count);
                                 }
                              
                              

                               

                              What happens if you set arquillian.qualifiers="container.active-1,container.active-2" ? Will the test break?

                               

                              IMHO, we could activate qualifiers only for non-qualified configuration used, given that qualifier matches the id.

                               

                               

                              <arquillian>
                                 <container id="jboss-remote">
                                 ...
                                 </container>
                                 <container id="jboss-remote" qualifier="node1">
                                 ...
                                 </container>
                                 <container id="jboss-remote" qualifier="node2">
                                 ...
                                 </container>
                                 <container id="jboss-remote" qualifier="jenkinsas">
                                 ...
                                 </container>
                              
                                 <extension id="webdriver">
                                 </extension>
                              
                                 <extension id="webdriver" qualifier="ie10">
                                 </extension>
                                 
                                 <extension id="webdriver" qualifier="jenkinsbrowser">
                                 </extension>
                              
                              </arquillian>
                              

                               

                              So, by default tests will be using configuration without qualifiers where qualifier are not referenced in the code. This means active.qualifiers will have no way how to override node1, node2, ie10, because these are directly used in tests.

                               

                              However, if you set active.qualifiers=jenkinsas,jenkinsbrowser, you'll get different browser for non-qualified Drones as well as different jboss-remote acting as default non-qualifier container. Obviously, given the arquillian.xml above, the only valid combinations will be:

                               

                              • active.qualifiers= (default)
                              • active.qualifiers=any single qualifier
                              • active.qualifiers=node1,jenkinsbrowser
                              • active.qualifiers=node1,ie10
                              • active.qualifiers=node2,jenkinsbrowser
                              • active.qualifiers=node2,ie10
                              • active.qualifiers=jenkinsas,ie10
                              • active.qualifiers=jenkinsas,jenkinsbrowser (that's what users want to do on jenkins - note, they would likely create jenkins qualifier and share for both webdriver and jboss-remote unless running matrix jobs)
                              • active.qualifiers=node1,node2 <= not valid, what is default jboss-romote configuration then?
                              • active.qualifiers=jenkinsbrowser,ie10 <= not valid, what is default browser configuration then?

                               

                               

                              You got the point. How does this sound to you? Is this readable enough for users?

                              • 12. Re: Default container/group in arquillian.xml enhancements
                                dan.j.allen

                                That's exactly the data I needed to validate the design. Give me a chance to work through it and I'll follow-up.

                                • 13. Re: Default container/group in arquillian.xml enhancements
                                  aslak

                                  Karel Piwko wrote:

                                   

                                  In that case, we should consider changing <extension qualifier="foobar" />, resp. <extension qualifier="foobar-dev" into <extension id="foobar" /> , resp. <extension id="foobar" qualifier="dev" />

                                   

                                  Edit: renaming arquillian.launch into arquillian.qualifier collides with multinode tests, e.g.:

                                   

                                  <container id="jboss-XYZ" qualifier="node1" >

                                  ...

                                  </container>

                                   

                                  <container id="jboss-XYZ" qualifier="node2" >

                                  ...

                                  </container>

                                   

                                  In multi node test you refer to the group element qualifier, not the container. The current rule is qualifier must be unique between group@qualifier and container@qualifier on top level.

                                   

                                  <group qualifier="X">

                                    <container id="" qualifier="Z" />

                                  <container id="" qualifier="V" />

                                  </group>

                                  <container id="" qualifier="A" />

                                   

                                  arquillian.qualifier=X launch group x

                                  arquillian.qualifier=A launch container A

                                  • 14. Re: Default container/group in arquillian.xml enhancements
                                    aslak

                                    We need to separate between launching containers for:

                                     

                                      - testing interaction between multiple containers

                                       - testing same tests on multiple different containers

                                     

                                    arquillian.qualifiers=containerConfigX,containerConfigY sound to me like "run the test suite against both containers", which is different from tests that are written to interact with more then one container in one TestClass (as the cluster test above).

                                     

                                    arquillian.qualifier=groupConfigX,groupConfigY could tho work the same, but against a cluster of GlassFish vs a clutser of JBossAS in the same suite run.

                                    1 2 Previous Next