This should be a concise list of requirements for the AS 7 testsuite.
There are some documents already, plus discussions on mailing list
This should be a shorter aggregation of all of them, kind of check-list.
It's possible this will then be moved elsewhere (AS docs?).
- The default build (i.e. mvn install) must run the smoke tests
- There must be a way to run all tests
- There must be a way to run a group of tests
- There must be a way to run individual tests
- Client side test debuging must work
- It should be possible to run individual tests in the IDE
- It should be possible to run indivudual tests againt an already running server
- It must be possible to skip the test run
- mvn install
- mvn -DallTests install
- mvn -DskipTests to skip all tests incl. unit tests of all modules and whole testsuite.
- ./build.sh clean install
- ./build.sh install
- ./build.sh install -DskipTests
- ./build.sh clean install -DskipTests
- ./build.sh clean install -DallTests
With these priorities:
- AS7-2541 TS: Clutering tests - figure out how they will be run.
- AS7-2485 Propagate property node0 to system properties
- AS7-2086 TS: Tests grouping
- AS7-2540 TS: Move *-arquillian.xml from testsuite.integration/src/test/resources/** to some better place.
- AS7-2210 TS: Move Arquillian configs from src/test/resources/X to some special dir.
- AS7-2448 TS: Test output is not stored in ...-output.txt
- AS7-2176 TS: Check tests workdirs
- AS7-2166 Alternative JDKs for building and running - OpenJDK 6, 7, Sun JDK 7, IcedTea
Feel free to add additional requirements.
Tests should also fail if their initialisation or clean up risks leaving the server in an inconsistent state - as an example we currently have tests that will silently ignore errors adding or removing security realms despite the fact that using the correct realm is critical to the correct execution of the test.
Would also suggest a unique naming of all resources used so if one test leaves a server in an inconsistent state it will not conflict with a different test.