Some tests, e.g. those depending on external systems, may fail through no fault of the code under test. Such tests make a suite fragile and it makes sense to try and avoid them, but if that’s infeasible, it may help to retry a number of times before eventually assuming actual failure. RetryingTest provides that functionality.

Basic Use

@RetryingTest(n) is used instead of @Test or other such annotations (e.g. @RepeatedTest). The attribute n specifies how often the test is executed before giving up.

@RetryingTest(3)
void failsNever() {
    // passing test code
}

The test failsNever is executed once (which succeeds) and marked as passed.

@RetryingTest(3)
void failsOnlyOnFirstInvocation() {
    // test code that fails on first execution
    // but passes on the second
}

The test failsOnlyOnFirstInvocation is executed once (which fails) and then once more (which succeeds). To allow the entire test suite to pass, the first execution is marked as ignored/aborted, which includes the underlying exception - the second is of course marked as passing.

@RetryingTest(3)
void failsAlways() {
    // test code that always fails
}

The test failsAlways is executed three times (all of which fail). The first two executions are marked as ignored/aborted, while the last as failed - each contains the underlying exception.

@RetryingTest(3)
void aborted() {
    // test code that is aborted,
    // e.g. because of an `Assumption`.
}

If a test is aborted (e.g.: because of a failed Assumption) @RetryingTest won’t try again after that. The test aborted is aborted before (or during) its first execution. The first execution is marked as aborted/skipped, containing the underlying cause. The test suite as a whole is also marked as aborted/skipped.

Combining @RetryingTest with @Test et al

If @RetryingTest is combined with @Test or TestTemplate-based mechanisms (like @RepeatedTest or @ParameterizedTest), the test engine will execute it according to each annotation (i.e. more than once). This is most likely unwanted and thus the engine warns:

Possible configuration error: method […​] resulted in multiple TestDescriptors […​]. This is typically the result of annotating a method with multiple competing annotations such as @Test, @RepeatedTest, @ParameterizedTest, @TestFactory, etc.

Thread-Safety

This extension is thread-safe. During parallel test execution, all repetitions of a @RepeatFailedTest are executed sequentially.

You may also enjoy