This page is about the Bean Validation 1.0 (JSR 303) TCK. This testharness is obsolete and succeeded by the Bean Vaildation 1.1 (JSR 349) TCK. If you want information about the latter, you should start reading here. If you are really interested in the JSR 303 TCK keep on reading.
Resources
Here are links to the most important pieces for the JSR 303 TCK:
- The full TCK distribution bundle can be found on SourceForge. The latest version is 1.0.6.GA
<dependency> <groupId>org.hibernate.jsr303.tck</groupId> <artifactId>jsr303-tck</artifactId> <version>1.0.6.GA</version> </dependency>
- If you want to build from source you have to checkout the 1.0.6.GA tag from GitHub - https://github.com/beanvalidation/beanvalidation-tck/tree/1.0.6.GA
- The online documentation can be found here.
- The reference implenetation is Hibernate Validator 4.2.0.Final - https://github.com/hibernate/hibernate-validator/tree/4.2.0.Final
<dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-validator</artifactId> <version>4.2.0.Final</version> </dependency>
- The online documentation for Hibernate Validator 4.2 can be found here.
Running the test harness
The actual test for the TCK are defined in the jsr303-tck jar file. You can checkout the source code for the TCK from GitHub as well. The tests are an excellent source for Bean Validation usage examples. Each test is linked to an assertions in the specification via an @SpecAssert annotation. All tested assertions are listed in tck-audit.xml which is part of the TCK.
The easiest way to run the TCK is configured as a maven project. We recommend to setup a separate maven module to run the TCK. This module won't contain any source code itself. Instead it will run the tests defined in the Bean Validation TCK against the Bean Validation implementation implementation specified in the POM.
You'll need:
- a pom.xml
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <parent> <artifactId>hibernate-validator-parent</artifactId> <groupId>org.hibernate</groupId> <version>4.2.0.Final</version> <relativePath>../pom.xml</relativePath> </parent> <artifactId>hibernate-validator-tck-runner</artifactId> <name>Hibernate Validator TCK Runner</name> <description>Aggregates dependencies and runs the JSR-303 TCK</description> <dependencies> <dependency> <groupId>javax.validation</groupId> <artifactId>validation-api</artifactId> </dependency> <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-validator</artifactId> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.hibernate.javax.persistence</groupId> <artifactId>hibernate-jpa-2.0-api</artifactId> <scope>test</scope> </dependency> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> </dependency> <dependency> <groupId>org.hibernate.jsr303.tck</groupId> <artifactId>jsr303-tck</artifactId> </dependency> <dependency> <groupId>org.jboss.jbossas.as7-cdi-tck</groupId> <artifactId>jbossas-container</artifactId> </dependency> </dependencies> <properties> <jboss.home>${env.JBOSS_HOME}</jboss.home> <validation.provider>org.hibernate.validator.HibernateValidator</validation.provider> <remote.debug /> </properties> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>copy</id> <phase>generate-test-sources</phase> <goals> <goal>copy</goal> </goals> <configuration> <stripVersion>true</stripVersion> <artifactItems> <artifactItem> <groupId>org.hibernate.jsr303.tck</groupId> <artifactId>jsr303-tck</artifactId> <type>xml</type> <classifier>suite</classifier> <overWrite>false</overWrite> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <configuration> <suiteXmlFiles> <suiteXmlFile>${project.build.directory}/dependency/jsr303-tck-suite.xml</suiteXmlFile> </suiteXmlFiles> <argLine>-Xmx128m</argLine> <forkMode>once</forkMode> <systemProperties> <property> <name>validation.provider</name> <value>${validation.provider}</value> </property> <property> <name>org.jboss.testharness.spi.StandaloneContainers</name> <value>org.hibernate.jsr303.tck.util.StandaloneContainersImpl</value> </property> </systemProperties> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-report-plugin</artifactId> <executions> <execution> <id>generate-test-report</id> <phase>test</phase> <goals> <goal>report-only</goal> </goals> </execution> </executions> <configuration> <outputDirectory>${project.build.directory}/surefire-reports</outputDirectory> <outputName>test-report</outputName> </configuration> </plugin> </plugins> </build> <profiles> <profile> <id>incontainer-debug</id> <activation> <property> <name>debug</name> </property> </activation> <properties> <remote.debug>-Xnoagent -Djava.compiler=NONE -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005 </remote.debug> </properties> </profile> <profile> <id>incontainer</id> <activation> <property> <name>incontainer</name> </property> </activation> <dependencies> <dependency> <groupId>org.jboss.as</groupId> <artifactId>jboss-as-arquillian-container-managed</artifactId> <scope>test</scope> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <configuration> <suiteXmlFiles> <suiteXmlFile>${project.build.directory}/dependency/jsr303-tck-suite.xml</suiteXmlFile> </suiteXmlFiles> <systemProperties> <property> <name>java.io.tmpdir</name> <value>${project.build.outputDirectory}</value> </property> <property> <name>org.jboss.testharness.standalone</name> <value>false</value> </property> <property> <name>org.jboss.testharness.container.forceRestart</name> <value>true</value> </property> <property> <name>org.jboss.testharness.runIntegrationTests</name> <value>true</value> </property> <property> <name>org.jboss.testharness.libraryDirectory</name> <value>${project.build.directory}/dependency/lib</value> </property> <property> <name>org.jboss.testharness.outputDirectory</name> <value>${project.build.directory}</value> </property> <property> <name>jboss.options</name> <value>-Xms128m -Xmx512m -XX:MaxPermSize=512m -enableassertions -Dvalidation.provider=${validation.provider} ${remote.debug} </value> </property> <property> <name>org.jboss.testharness.spi.Containers</name> <value>org.jboss.jbossas.cdiTck.container.ArquillianContainerAdaptor</value> </property> <property> <name>org.jboss.har2arq.container</name> <value>org.jboss.as.arquillian.container.managed.JBossAsManagedContainer</value> </property> <property> <name>jboss.home</name> <value>${jboss.home}</value> </property> <property> <name>validation.provider</name> <value>${validation.provider}</value> </property> </systemProperties> </configuration> </plugin> </plugins> </build> </profile> <profile> <!-- Enabling this profile will bundle the BV API and Hibernate Validator w/ the deployed war files instead of testing against the AS provided dependencies --> <id>bundled-dependencies</id> <activation> <property> <name>bundled-dependencies</name> </property> </activation> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <executions> <execution> <id>copy</id> <phase>generate-test-sources</phase> <goals> <goal>copy</goal> </goals> <configuration> <stripVersion>true</stripVersion> <artifactItems> <artifactItem> <groupId>org.hibernate.jsr303.tck</groupId> <artifactId>jsr303-tck</artifactId> <type>xml</type> <classifier>suite</classifier> <overWrite>false</overWrite> </artifactItem> <artifactItem> <groupId>javax.validation</groupId> <artifactId>validation-api</artifactId> <overWrite>true</overWrite> <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory> </artifactItem> <artifactItem> <groupId>org.hibernate</groupId> <artifactId>hibernate-validator</artifactId> <overWrite>true</overWrite> <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory> </artifactItem> <artifactItem> <groupId>org.slf4j</groupId> <artifactId>slf4j-simple</artifactId> <overWrite>true</overWrite> <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory> </artifactItem> <artifactItem> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <overWrite>true</overWrite> <outputDirectory>${project.build.directory}/dependency/lib</outputDirectory> </artifactItem> </artifactItems> </configuration> </execution> </executions> </plugin> </plugins> </build> </profile> <profile> <id>write-artifacts-to-disk</id> <activation> <property> <name>dumpArtifacts</name> </property> </activation> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <executions> <execution> <id>generate-jsr-303-artifacts</id> <phase>test-compile</phase> <goals> <goal>java</goal> </goals> </execution> </executions> <configuration> <classpathScope>test</classpathScope> <mainClass>org.jboss.testharness.api.TCK</mainClass> <systemProperties> <systemProperty> <key>dumpArtifacts</key> <value>true</value> </systemProperty> <systemProperty> <key>org.jboss.testharness.outputDirectory</key> <value>target/jsr303-artifacts</value> </systemProperty> <systemProperty> <key>org.jboss.testharness.libraryDirectory</key> <value>target/jsr303-artifacts/dependencies</value> </systemProperty> <systemProperty> <key>org.jboss.testharness.libraryDirectory</key> <value>${project.build.directory}/dependency/lib</value> </systemProperty> <systemProperty> <key>org.jboss.testharness.spi.StandaloneContainers</key> <value>org.hibernate.jsr303.tck.util.StandaloneContainersImpl</value> </systemProperty> </systemProperties> </configuration> </plugin> </plugins> </build> </profile> </profiles> </project>
- and src/main/resources/META-INf/jboss-test-harness.properties
org.jboss.testharness.api.TestLauncher=org.jboss.testharness.impl.runner.servlet.ServletTestLauncherorg.jboss.testharness.testPackage=org.hibernate.jsr303.tck.tests
You can see the full setup for the Hibernate Validator TCK runner here: https://github.com/hibernate/hibernate-validator/blob/4.2.0.Final/hibernate-validator-tck-runner
Once you have the right setup you can choose between running the tests standalone or in a container. The example above assumes that you run the TCK against JBoss 7.
Standalone
This is the easiest way to run the test harness.
> mvn clean test
In this case the tests are run with the default surefire classpath. All tests of the TCK which are not marked as @IntegrationTest are run.
In container
In order to run against JBoss 7 you need:
- A JBoss 7 installation
- Define jboss.home in your POM
If your setup is complete it is enough to run:
> mvn clean test -Dincontainer
Dumping artifacts
In case you are running against a container the tests are deployed as war/ear files. In order to debug the tests it can be useful to actually inspect the deployed artifacts. To do this just specify the dumpArtifacts property at the command line. This will create all the artifacts which gets deployed in target/jsr303-artifacts.
> mvn clean test -DdumpArtifacts
TCK Audit
Part of the source code of the jsr303-tck is an audit tool which allows to determine the coverage of tests in the TCK in relation to the actual specification. The assertions of the TCK are captured in a file called tck-audit.xml which can be found under /src/main/resources of the source code. More information about the format and the usage of this xml file can be found JSR 299 TCK.
In order to run the audit tool you have to checkout the jsr303-tck project and run:
> mvn clean test -P tck-audit
After the build completes there should be a coverage.html in your target directory. Note, this is no measurements whether an actual implementation passes a TCK. It just indicates how well the TCK tests cover the assertions in the specification. More information about the audit tool can be found here.
What do all the annotations mean?
// $Id: BootstrapCustomProviderDefinedInServiceFileTest.java 17345 2009-08-18 09:35:56Z hardy.ferentschik $
/*
* JBoss, Home of Professional Open Source
* Copyright 2008, Red Hat Middleware LLC, and individual contributors
* by the @authors tag. See the copyright.txt in the distribution for a
* full listing of individual contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.hibernate.jsr303.tck.tests.bootstrap.customprovider;
import javax.validation.Configuration;
import javax.validation.Validation;
import javax.validation.ValidatorFactory;
import org.jboss.test.audit.annotations.SpecAssertion;
import org.jboss.test.audit.annotations.SpecAssertions;
import org.jboss.testharness.AbstractTest;
import org.jboss.testharness.impl.packaging.Artifact;
import org.jboss.testharness.impl.packaging.ArtifactType;
import org.jboss.testharness.impl.packaging.Classes;
import org.jboss.testharness.impl.packaging.IntegrationTest;
import org.jboss.testharness.impl.packaging.Resource;
import static org.testng.Assert.assertNotNull;
import static org.testng.Assert.assertTrue;
import org.testng.annotations.Test;
import org.hibernate.jsr303.tck.common.TCKValidationProvider;
import org.hibernate.jsr303.tck.common.TCKValidatorConfiguration;
import org.hibernate.jsr303.tck.util.TestUtil;
/**
* @author Hardy Ferentschik
*/
@Artifact(artifactType = ArtifactType.JSR303)
@Classes({
TestUtil.class,
TestUtil.PathImpl.class,
TestUtil.NodeImpl.class,
TCKValidationProvider.class,
TCKValidatorConfiguration.class,
TCKValidationProvider.DummyValidatorFactory.class
})
@Resource(source = "javax.validation.spi.ValidationProvider",
destination = "WEB-INF/classes/META-INF/services/javax.validation.spi.ValidationProvider")
@IntegrationTest
public class BootstrapCustomProviderDefinedInServiceFileTest extends AbstractTest {
@Test
@SpecAssertions({
@SpecAssertion(section = "4.4", id = "a"),
@SpecAssertion(section = "4.4.4.2", id = "a")
})
public void testGetFactoryByProviderSpecifiedProgrammatically() {
TCKValidatorConfiguration configuration = Validation.byProvider( TCKValidationProvider.class ).configure();
ValidatorFactory factory = configuration.buildValidatorFactory();
assertNotNull( factory );
assertTrue( factory instanceof TCKValidationProvider.DummyValidatorFactory );
}
}
@Artifact
Annotation is part of the jboss-test-harness framework and decides which type of artifact should be build when deploying the test into a container. This annotation is irrelevant for when tests are run in standalone mode.
@Classes
Per default only classes from the same package as the test class are included in the artifacts. In order to include other classes the @Classes annotation is used. This annotation is irrelevant for when tests are run in standalone mode.
@Resource(s)
Used to include resource files into artifacts. In case of ArtifactType.JSR303 war files are getting build and the resource files should go somewhere into WEB-INF/classes. This annotation is irrelevant for when tests are run in standalone mode.
@IntegrationTest
Per default tests will be run in standalone as well as in incontainer mode. Marking a test as @IntegrationTest will skip this tests in standalone mode.
@Test
TestNG annotation to mark a method as test.
@SpecAssertion(s)
Used to link a test to the BV specification via the tck-audit.xml file.
Further reading
The harness to run the TCK tests is based on and reuses the infrastructure provided by the JSR 299 TCK.
Comments