A flaky test is a test which does not give a reliable pass fail result. For example, if we had a test which, under the same architecture-OS-JVM combination, returns pass sometimes and fail other times, then this test is flaky. Tests can be flaky if they directly involve some form of non-determinism, and the non-determinsism is affecting the result. Tests can also appear to be flaky if they depend upon a component which is itself flaky. For example, if your test depended on the correct deployment of certain resources, and the deployment of the resources was flaky (does not reliably occur), then the test may appear to be flaky.

The aim of this page is to provide a place to record tests which we have found to be flaky, so that they may later be investigated and made deterministic. Please list the architcure, OS and JVM combination used to run the test.