Some tests, e.g. those depending on external systems, may fail through no fault of the code under test.
Such tests make a suite fragile and it makes sense to try and avoid them, but if that’s infeasible, it may help to retry a number of times before eventually assuming actual failure.
RetryingTest
provides that functionality.
Basic Use
@RetryingTest(n)
is used instead of @Test
or other such annotations (e.g. @RepeatedTest
).
The attribute n
specifies how often the test is executed before giving up.
@RetryingTest(3)
void failsNever() {
// passing test code
}
The test failsNever
is executed once (which succeeds) and marked as passed.
@RetryingTest(3)
void failsOnlyOnFirstInvocation() {
// test code that fails on first execution
// but passes on the second
}
The test failsOnlyOnFirstInvocation
is executed once (which fails) and then once more (which succeeds).
To allow the entire test suite to pass, the first execution is marked as ignored/aborted, which includes the underlying exception - the second is of course marked as passing.
@RetryingTest(3)
void failsAlways() {
// test code that always fails
}
The test failsAlways
is executed three times (all of which fail).
The first two executions are marked as ignored/aborted, while the last as failed - each contains the underlying exception.
@RetryingTest(3)
void aborted() {
// test code that is aborted,
// e.g. because of an `Assumption`.
}
If a test is aborted (e.g.: because of a failed Assumption) @RetryingTest
won’t try again after that.
The test aborted
is aborted before (or during) its first execution.
The first execution is marked as aborted/skipped, containing the underlying cause.
The test suite as a whole is also marked as aborted/skipped.
Combining @RetryingTest
with @Test
et al
If @RetryingTest
is combined with @Test
or TestTemplate
-based mechanisms (like @RepeatedTest
or @ParameterizedTest
), the test engine will execute it according to each annotation (i.e. more than once).
This is most likely unwanted and thus the engine warns:
Possible configuration error: method […] resulted in multiple TestDescriptors […]. This is typically the result of annotating a method with multiple competing annotations such as @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, etc.
Thread-Safety
This extension is thread-safe.
During parallel test execution, all repetitions of a @RepeatFailedTest
are executed sequentially.