Version 11
Created by mkouba on Mar 22, 2012 5:39 AM. Last modified by mkouba on Mar 21, 2013 3:15 AM.

    Since CDI TCK 1.1 left JBoss Test Harness and runs on Arquillian now it's highly recommended to go through Arquillian documentation first.

     

     

    1. A new test

    Every test class must extend org.jboss.cdi.tck.AbstractTest. Basically AbstractTest doesn't do a lot of things - it's a common base that enables Arquillian (extends org.jboss.arquillian.testng.Arquillian),  injects javax.enterprise.inject.spi.BeanManager,  and provides some convenient util methods.

    Don't forget to annotate the test class with @SpecVersion (with values corresponding to <specification> attributes in cdi-tck/impl/src/main/resources/tck-audit-cdi.xml) otherwise the test assertions will not be included in the generated report.

    1.1 Arquillian stuff

     

    1.1.1 Test archive

     

  • Every test class must must define at least one static deployment method (@Deployment). General convention is to name this method createTestArchive. TCK contains several multi-deployment test scenarios as well (be careful when making changes to these tests - multi-deployment Arquillian tests tend to be tricky to build and debug).

    TCK test should use org.jboss.shrinkwrap.api.spec.WebArchive whenever possible. Use org.jboss.shrinkwrap.api.spec.EnterpriseArchive only if explicitly testing EAR structure or full Java EE platform services (JMS, JAX-WS, ...). Since TCK tests are a little bit specific (porting package) use TCK builders instead of ShrinkWrap APIs (org.jboss.cdi.tck.shrinkwrap.WebArchiveBuilder and org.jboss.cdi.tck.shrinkwrap.EnterpriseArchiveBuilder; check builders javadoc for more info). These builders can save a lot of time. And of course if you need more functionality than builders provide use ShrinkWrap API to adapt final archive.
  • Tip: TCK test do have to be portable - do not rely on implementation specific features and bugs! If in doubt check related specification (CDI, Java EE, Servlet, EJB, ...).

  • 1.1.2 Test run modes

    • Arquillian defines three types of test run modes. in-container, as-client and mixed.

    • in-container - test methods executed remotely (in the container)
    • as-client - test methods executed locally
    • mixed - some test test methods executed remotely and some locally

    All modes are used accross the test base though in-container is the most frequent. Pay attention to the test mode in case you have to debug your test.

    Tip: Keep in mind that broken tests that use @ShouldThrowException (see 1.3 Testing deployment failures) are always forced to use as-client mode.

  • 1.1.3 Test case injection points

  • Remember that enrichment of the test class is sometimes a bit tricky. First TCK uses Arquillian servlet protocol (org.jboss.arquillian.protocol.servlet.runner.ServletTestRunner) which means a test class is enriched in the context of a web module (even if your app is packaged as an EAR) -> this affects CDI bean resolution. Also don't forget to set org.testng.annotations.Test#dataProvider() attribute value to org.jboss.arquillian.testng.Arquillian.ARQUILLIAN_DATA_PROVIDER when leveraging method argument injection:
    • @Test(dataProvider = ARQUILLIAN_DATA_PROVIDER)
public void testSomething(Foo foo) {
   assertNotNull(foo);
   ...     
}
  • And finally Arquillian test enrichers may contain bugs (e.g. ARQ-77 and ARQ-836).

  • 1.1.4 Resource injection

    • Always use Arquillian resource injection (@ArquillianResource) to work with request URI (up through the context path) in as-client tests (in-container does not support @ArquillianResource at the moment).

  • 1.2 Test methods

  • 1.2.1 Specifying TCK assertions

  • Every test method must have at least one corresponding assertion - see tck-audit-cdi.xml. Assertions are defined via @SpecAssertion and @SpecAssertions annotations.
  • Tip: Although it's quite common in legacy test base it's not a good idea to write "Übertestmethod" that verifies dozens of assertions.

  • 1.2.2 Test groups

  • TCK defines several test groups. See constants defined in org.jboss.cdi.tck.TestGroups. The most important groups (used by TCK runner) are INTEGRATION and JAVAEE_FULL. INTEGRATION is used to exclude integration tests when running standalone container (e.g. Weld EE 1.1 - Embedded). JAVAEE_FULL delimits a subset of integration tests that require full Java EE platform - e.g. EAR packaging, JAX-WS, EJB timers, ... (tests that belong to this group are also considered to be integration tests).
  • Tip: Unless some Weld EE Embedded container issues are fixed (ARQ-753, ARQ-818, ARQ-819), all tests working with javax.enterprise.inject.spi.Extension must be marked as integration tests.
  • This was fixed in 1.0.0.CR4.

    1.2.4 Test assertions

  • Althoug a lot of legacy tests use Java assert keyword, do not use it anymore. Use testng Assert class static methods instead. One of the benefits is more clearer error info in the stacktrace.

  • 1.3 Testing deployment failures

    In TCK there is a lot scenarios testing deployment failures. Such tests are usually in a special subpackage named broken ("broken" designation has a special meaning here - such tests are not badly written or incorrect, hopefully :-). Most of them test deployment failures and "broken" configuration, in other words problems detected automatically by the container (spec chapter 2.8).

  • 1.3.1 Detecting deployment failure

    A deployment method must define @ShouldThrowException annotation with DeploymentException or DefinitionException as a value. This will cause a test failure in case of no deployment error occures. Furthermore tests class must define one test method of any name and content (will never be called). The method must define corresponding assertion and possibly testing group.
  • Tip: Note that Arquillian requires a special implementation of DeploymentExceptionTransformer to work correctly with managed and remote containers adapters (see also org.jboss.weld.tck.as7.JBossAS7DeploymentExceptionTransformer workaround).

    • 2. Test exclusion

    Occasionally TCK and implementation issues emerge. We use TestNG XML configuration to exclude tests related to these issues. Basically there are two TestNG configuration files. The first one is included in TCK implementation and defines CDI spec and TCK issues only (e.g. invalid test or specification bug): cdi-tck/impl/src/main/resources/tck-tests.xml. The second one is included in TCK runner (is specific for each implementation) and contains exludes from the first one plus exludes related to implementation issues.

    3. Logging

    Sometimes it's useful to log some tracing information during complex test debugging. Do no use System.out.println() though but use org.jboss.cdi.tck.SimpleLogger instead. It's a very simple JDK logger (java.util.logging) wrapper that logs all messages with Level.FINE. JDK logging is generaly easily configurable in target containers.

    4. Asynchronous tests

    Testing asynchronous method invocation (Session beans asynchronous methods, EJB timers, JMS, etc.) is not always easy. If there is no synchronization available out-of-box (e.g. java.util.concurrent.Future) you can either simply wait for a constant time (probably using Thread.sleep()) or implement your own simple synchronization (e.g. testing static variables). org.jboss.cdi.tck.Timer is intended to help you with the latter. Search usage in TCK tests for more details.

    5. Testing sequences

    When writing a test for chronological sequences (e.g. event A must be fired before B and B must be fired before C) consider using a simple util class org.jboss.cdi.tck.util.ActionSequence. Check it's javadoc and referencing tests for more information.

    1202 Views Categories: Tags: