From Wikipedia:

In mathematics, specifically set theory, the Cartesian product of two sets A and B, denoted A × B, is the set of all ordered pairs (a, b) where a is in A and b is in B. In terms of set-builder notation, that is A × B = {(a,b) | a ∈ A and b ∈ B} [...] One can similarly define the Cartesian product of n sets, also known as an n-fold Cartesian product, which can be represented by an n-dimensional array, where each element is an n-tuple.

What does all this mean?

The Cartesian product of sets is all the possible combinations where you take a single element from each set. If you have two sets, { 1, 2 } and { 3, 4 }, their cartesian product is { { 1, 3 }, { 1, 4 }, { 2, 3 }, { 2, 4 } }.

Sometimes it’s useful to test all possible combinations of parameter sets. Normally, this results in a lot of written test data parameters. For a more comfortable way you may use the @CartesianTest extension. The extension takes the test data parameter values and runs the test for every possible combination of them.

Note
The CartesianTest extension has undergone significant changes in 1.5.0. This included moving it into a new package. The old variant is deprecated and will be removed in the 2.0 release (tentatively scheduled for December 2021).

Basic Use

@CartesianTest is used instead of @Test or other such annotations (e.g. @RepeatedTest).

You can supply test parameters to @CartesianTest in two ways:

  • You can annotate your test method, providing all parameter values in a single annotation (Not yet implemented).

  • The test parameters can be annotated with @CartesianTest.Values, @CartesianTest.Enum, or range source annotations (see Annotating your test parameters)

Specifying more than one kind of parameter source (i.e.: annotating your test parameters and the test method itself) does not work and will throw an ExtensionConfigurationException.

Our earlier example with { 1, 2 } and { 3, 4 }, would look like this:

import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;

class MyCartesianTestClass {
    @CartesianTest
    void myCartesianTestMethod(
            @Values(ints = { 1, 2 }) int x,
            @Values(ints = { 3, 4 }) int y) {
        // passing test code
    }
}

@CartesianTest works with parameters injected by JUnit automatically (e.g.: TestReporter). You can read about auto-injected parameters here.

Just like the mathematical Cartesian product, @CartesianTest works with sets. Duplicate elements get removed automatically. If your input is { 1, 1, 3 } and { 2, 2 } the extension will consider their Cartesian product { { 1, 2 }, { 3, 2 } }. Otherwise, the test would run with the same parameters multiple times. If you need to pass the same parameters multiple times, you might want to look into repeated tests.

Annotating your test parameters

You can annotate the parameters of your @CartesianTest, to provide values to that specific parameter. Parameter annotations are "self-contained", they only provide values to the parameter they are on and do not interfere with each other. You can mix and match parameter annotations as you need.

import java.time.temporal.ChronoUnit;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.params.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

class MyCartesianTestClass {
    @CartesianTest
    void testIntChars(
            @ShortRangeSource(from = 1, to = 3, step = 1) short s,
            @Values(strings = { "A", "B" }) String character,
            @Enum ChronoUnit unit) {
        // passing test code
    }
}

@CartesianTest.Values

@CartesianTest.Values is used to define the possible inputs of a single test parameter. The test will try every combination those values can have.

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;

class MyCartesianTestClass {
    @CartesianTest
    void testIntChars(
            @Values(ints = { 1, 2, 4 }) int number,
            @Values(strings = { "A", "B" }) String character) {
        // passing test code
    }
}

This annotation might look familiar - it mimics JUnit’s @ValueSource, except @CartesianTest.Values must be put on the parameter. It does NOT work with @ParameterizedTest.

The test testIntChars is executed exactly six times. The first parameter can have any of the three values 1, 2 or 4. The second parameter can have any of the two values "A" or "B". @CartesianTest tests for all input combinations, that’s 3 × 2, so six tests in total.

To demonstrate with a table:

# of test

value of number

value of character

1st test

1

"A"

2nd test

1

"B"

3rd test

2

"A"

4th test

2

"B"

5th test

4

"A"

6th test

4

"B"

@CartesianTest.Enum

@CartesianTest.Enum provides a convenient way to use Enum constants.

import java.time.temporal.ChronoUnit;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

class MyCartesianTestClass {
    @CartesianTest
    void testWithEnum(@Enum ChronoUnit unit) {
        assertThat(unit).isNotNull();
    }
}

Like @CartesianTest.Values, @CartesianTest.Enum is an annotation that might look familiar - it mimics JUnit’s @EnumSource, except @CartesianTest.Enum must be put on the parameter. It does NOT work with @ParameterizedTest.

The annotation has an optional value attribute. When omitted, the declared type of the parameter is used. The test will fail if it is not an enum type. The value attribute is required in the following example because the method parameter is declared as TemporalUnit, i.e. the interface implemented by ChronoUnit, which isn’t an enum type.

import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

class MyCartesianTestClass {
    @CartesianTest
    void testExplicitEnum(@Enum(ChronoUnit.class) TemporalUnit unit) {
        assertThat(unit).isNotNull();
    }
}

The annotation provides an optional names attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used.

import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

class MyCartesianTestClass {
    @CartesianTest
    void testEnumNames(
            @Enum(names = { "DAYS", "HOURS" }) ChronoUnit unit) {
        assertThat(EnumSet.of(ChronoUnit.DAYS, ChronoUnit.HOURS)).contains(unit);
    }
}

The annotation also provides an optional mode attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples.

import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

class MyCartesianTestClass {
    @CartesianTest
    void testWithEnumModes(
            @Enum(mode = EXCLUDE, names = { "ERAS", "FOREVER" }) ChronoUnit unit) {
        assertThat(EnumSet.of(ChronoUnit.ERAS, ChronoUnit.FOREVER)).doesNotContain(unit);
    }
}
import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

class MyCartesianTestClass {
    @CartesianTest
    void testWithEnumRegex(
            @Enum(mode = MATCH_ALL, names = "^.*DAYS$") ChronoUnit unit) {
        assertThat(unit.name()).endsWith("DAYS");
    }
}

The example below shows how to use @CartesianTest.Enum with two Enum types.

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

enum MyEnum {
    ONE, TWO, THREE
}

enum AnotherEnum {
    ALPHA, BETA, GAMMA, DELTA
}

class MyCartesianTestClass {
    @CartesianTest
    void testEnumValues(
            @Enum MyEnum myEnum,
        @Enum(names = { "ALPHA", "DELTA" }, mode = Mode.EXCLUDE) AnotherEnum anotherEnum) {
        // passing test code
    }
}

The test testEnumValues is executed exactly six times. The first parameter can have any of the three constants ONE, TWO or THREE. The second parameter can have any of the two constants BETA or GAMMA (note the EXCLUDE mode applied to the other two constants). @CartesianTest tests for all input combinations, that’s 3 × 2, so six tests in total.

To demonstrate with a table:

# of test

value of myEnum

value of anotherEnum

1st test

ONE

BETA

2nd test

ONE

GAMMA

3rd test

TWO

BETA

4th test

TWO

GAMMA

5th test

THREE

BETA

6th test

THREE

GAMMA

Range Source annotations

You can annotate your test parameters with range source annotations. For this purpose only, range sources can be used on parameters.

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.params.*;

class MyCartesianTestClass {
    @CartesianTest
    void testShortAndLong(
            @ShortRangeSource(from = 1, to = 3, step = 1) short s,
            @LongRangeSource(from = 0L, to = 2L, step = 1, closed = true) long l) {
        // passing test code
    }
}

The test testShortAndLong is executed exactly six times. The first parameter can have any of the two values 1, 2. The second parameter can have any of the three values 0L, 1L or 2L. @CartesianTest tests for all input combinations, that’s 2 × 3, so six tests in total.

To demonstrate with a table:

# of test

value of s

value of l

1st test

1

0L

2nd test

1

1L

3rd test

1

2L

4th test

2

0L

5th test

2

1L

6th test

2

2L

For more information, please see the separate documentation about range sources.

Writing your own @ArgumentsSource for @CartesianTest

You might find that the available @ArgumentsSource annotations do not fit your need. In that case, you can write a custom @ArgumentsSource and a corresponding ArgumentsProvider for @CartesianTest. Because CartesianTest works with parameters, you will have to use CartesianArgumentsProvider instead of ArgumentsProvider. This will let you initialize your argument source with it’s corresponding parameter.

Let’s demonstrate with an example.

For the sake of simplicity, let’s imagine that @CartesianTest.Values does not exist. We would like to have an @ArgumentsSource, where we can specify integers. Let’s create an annotation for it.

import java.lang.annotation.*;

@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.RUNTIME)
@ArgumentsSource(IntArgumentsProvider.class)
public @interface Ints {

    int[] value();

}

The annotation has to have RUNTIME retention, so JUnit (and Pioneer) can discover it on your test via reflection. It has to be annotated with @ArgumentsSource, so Pioneer knows which CartesianArgumentsProvider it should invoke.

Next, we need to have a class that takes these values and passes them to our test.

import java.lang.reflect.Parameter;
import java.util.Arrays;
import java.util.Objects;
import java.util.stream.Stream;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.provider.Arguments;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsProvider;

class IntArgumentsProvider implements CartesianArgumentsProvider {

    @Override
    public Stream<Integer> provideArguments(ExtensionContext context, Parameter parameter) {
        Ints source = Objects.requireNonNull(parameter.getAnnotation(Ints.class));
        return Arrays.stream(source.value());
    }

}

The class has to implement CartesianArgumentsProvider. Note that an implementation of CartesianArgumentsProvider must be declared as either a top-level class or as a static nested class. It must also provide a default (no argument) constructor. The IntArgumentsProvider class first initializes itself via the accept method, then provides the arguments to JUnit via the provideArguments method. The order of these operations is guaranteed, so accept is always first and provideArguments is always second.

In our case, we don’t have to process the values from our annotation, so we just return the values as is, but you could do additional processing, for example:

import java.lang.annotation.*;
import java.util.*;
import java.lang.reflect.Parameter;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.provider.Arguments;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsProvider;

@Target(ElementType.PARAMETER)
@Retention(RetentionPolicy.RUNTIME)
@ArgumentsSource(PeopleProvider.class)
@interface People {

    String[] names();

    int ages();

}

class PeopleProvider implements CartesianArgumentsProvider {

    @Override
public Stream<Person> provideArguments(ExtensionContext context, Parameter parameter) {
        People source = Objects.requireNonNull(parameter.getAnnotation(People.class));
    return IntStream.range(0, source.names().length)
        .mapToObj(i -> new Person(source.names()[i], source.ages()[i]));
    }
}

Customizing Display Names

By default, the display name of a CartesianTest invocation contains the invocation index and the String representation of all arguments for that specific invocation. You can customize invocation display names via the name attribute of the @CartesianTest annotation. For example:

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;

class MyCartesianTest {

    @CartesianTest(name = "{index} => first bit: {0} second bit: {1}")
    @DisplayName("Basic bit test")
    void testWithCustomDisplayName(
            @Values({"0", "1"}) String a, @Values({"0", "1"}) String b) {
        // passing test code
    }
}

When executing the above test, you should see output similar to the following:

Basic bit test
├─ 1 => first bit: 0 second bit: 0
├─ 2 => first bit: 0 second bit: 1
├─ 3 => first bit: 1 second bit: 0
└─ 4 => first bit: 1 second bit: 1

Please note that name is a MessageFormat pattern. A single quote (') needs to be represented as a doubled single quote ('') in order to be displayed.

CartesianTest supports the following placeholders in custom display names:

Placeholder Description

{displayName}

the display name of the method

{index}

the current invocation index, starting with 1

{arguments}

the complete, comma-separated arguments list

{0}, {1}, …​

an individual argument

Do not use @CartesianTest with @Test

If @CartesianTest is combined with @Test or TestTemplate-based mechanisms (like @RepeatedTest or @ParameterizedTest), the test engine will execute it according to each annotation (i.e. more than once). This is most likely unwanted and will probably lead to the following exception/failure message:

org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter […​]

This is because @Test does not know what to do with the parameter(s) of the @CartesianTest.

Thread-Safety

This extension is safe to use during parallel test execution.