Often tests fail due to a bug in the tested application or in dependencies.
Traditionally such a test method would be annotated with JUnit’s @Disabled
.
However, this has disadvantages when the bug that causes the test failure is fixed:
-
the developer might not notice the existing test method and create a new one
-
the existing test method might not be noticed and remains disabled for a long time after the bug has been fixed, adding no value for the project
@ExpectedToFail
solves these issues.
Unlike @Disabled
it still executes the annotated test method but aborts the test if a test failure or error occurs.
However, if the test is executed successfully it will cause a test failure because the test is working.
This lets the developer know that they have fixed the bug (possibly by accident) and that they can now remove the @ExpectedToFail
annotation from the test method.
The annotation can only be used on methods and as meta-annotation on other annotation types.
Similar to @Disabled
, it has to be used in addition to a "testable" annotation, such as @Test
.
Otherwise the annotation has no effect.
Important
|
This annotation is not intended as a way to mark test methods which intentionally cause exceptions.
Such test methods should use JUnit’s |
Basic Use
The test is aborted because the tested method brokenMethod()
returns an incorrect result.
@Test
@ExpectedToFail
void test() {
int actual = brokenMethod();
assertThat(actual).isEqualTo(10);
}
An aborted test is no failure and so the test suite passes (if all other tests pass, of course).
Should brokenMethod()
start returning the correct value, the test invocation passes, but @ExpectedToFail
marks the test as failed to draw attention to that change in behavior.
A custom message can be provided, explaining why the tested code is not working as intended at the moment.
@Test
@ExpectedToFail("Implementation bug in brokenMethod()")
void doSomething() {
int actual = brokenMethod();
assertThat(actual).isEqualTo(10);
}
Only Abort on Expected Exceptions
A test that is @ExpectedToFail
will change its behavior by starting to actually fail, once the code under test behaves correctly.
If the underlying failure changes, though, for example from an assertion error to an exception or from an UnsupportedOperationException
of a formerly missing implementation to a runtime exception of buggy implementation, the @ExpectedToFail
-test will keep passing.
To better react to such changes, @ExpectedToFail
has an attribute withExceptions
that can be used to enumerate the exceptions which when thrown will result in an aborted (and thus passing) test.
Any other exception thrown by the code under test will result in a failing test.
In the following example a test case for productionFeature()
has been implemented.
While the test is fully implemented, the production code has been stubbed and throws an UnsupportedOperationException
to indicate this.
@Test
@ExpectedToFail(withExceptions = UnsupportedOperationException.class)
void testProductionFeature() {
int actual = productionFeature();
assertThat(actual).isEqualTo(10);
}
private int productionFeature() {
throw new UnsupportedOperationException("productionFeature() is not yet implemented");
}
Once productionFeature()
is implemented, @ExpectedToFail
will fail the test, as no UnsupportedOperationException
is thrown anymore.
By using withExceptions
you can thus prevent "masking" a faulty implementation (e.g. when a value other rather than 10
is returned) with an aborted test (which would be the result when no withExceptions
is set).
Thread-Safety
This extension is safe to use during parallel test execution.