From Wikipedia:

In mathematics, specifically set theory, the Cartesian product of two sets A and B, denoted A × B, is the set of all ordered pairs (a, b) where a is in A and b is in B. In terms of set-builder notation, that is A × B = {(a,b) | a ∈ A and b ∈ B} [...] One can similarly define the Cartesian product of n sets, also known as an n-fold Cartesian product, which can be represented by an n-dimensional array, where each element is an n-tuple.

What does all this mean?

The Cartesian product of sets is all the possible combinations where you take a single element from each set. If you have two sets, { 1, 2 } and { 3, 4 }, their cartesian product is { { 1, 3 }, { 1, 4 }, { 2, 3 }, { 2, 4 } }.

Sometimes it’s useful to test all possible combinations of parameter sets. Normally, this results in a lot of written test data parameters. For a more comfortable way you may use the @CartesianTest extension. The extension takes the test data parameter values and runs the test for every possible combination of them.

Note
The CartesianTest extension has undergone significant changes in 1.6.0. This included moving it into a new package. The old variant is deprecated and will be removed in the 2.0 release (tentatively scheduled for 2022).

Basic Use

@CartesianTest is used instead of @Test or other such annotations (e.g. @RepeatedTest).

You can supply test parameters to @CartesianTest in two ways:

  • You can annotate your test method, providing all parameter values in a single annotation.

  • The test parameters can be annotated with @CartesianTest.Values, @CartesianTest.Enum, or range source annotations (see Annotating your test parameters)

Specifying more than one kind of parameter source (i.e.: annotating your test parameters and the test method itself) does not work and will throw an ExtensionConfigurationException.

Our earlier example with { 1, 2 } and { 3, 4 }, would look like this:

import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;

    class MyCartesianTestClass {

        @CartesianTest
        void myCartesianTestMethod(@Values(ints = { 1, 2 }) int x, @Values(ints = { 3, 4 }) int y) {
            // passing test code
        }

    }

@CartesianTest works with parameters injected by JUnit automatically (e.g.: TestReporter). You can read about auto-injected parameters here.

Just like the mathematical Cartesian product, @CartesianTest works with sets. Duplicate elements get removed automatically. If your input is { 1, 1, 3 } and { 2, 2 } the extension will consider their Cartesian product { { 1, 2 }, { 3, 2 } }. Otherwise, the test would run with the same parameters multiple times. If you need to pass the same parameters multiple times, you might want to look into repeated tests.

Annotating your test parameters

You can annotate the parameters of your @CartesianTest, to provide values to that specific parameter. Parameter annotations are "self-contained", they only provide values to the parameter they are on and do not interfere with each other. You can mix and match parameter annotations as you need.

import java.time.temporal.ChronoUnit;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.params.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    class SecondCartesianTestClass {

        @CartesianTest
        void testIntChars(@ShortRangeSource(from = 1, to = 3, step = 1) short s,
                @Values(strings = { "A", "B" }) String character, @Enum ChronoUnit unit) {
            // passing test code
        }

    }

@CartesianTest.Values

@CartesianTest.Values is used to define the possible inputs of a single test parameter. The test will try every combination those values can have.

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;

    class ThirdCartesianTestClass {

        @CartesianTest
        void testIntChars(@Values(ints = { 1, 2, 4 }) int number, @Values(strings = { "A", "B" }) String character) {
            // passing test code
        }

    }

This annotation might look familiar - it mimics JUnit’s @ValueSource, except @CartesianTest.Values must be put on the parameter. It does NOT work with @ParameterizedTest.

The test testIntChars is executed exactly six times. The first parameter can have any of the three values 1, 2 or 4. The second parameter can have any of the two values "A" or "B". @CartesianTest tests for all input combinations, that’s 3 × 2, so six tests in total.

To demonstrate with a table:

# of test

value of number

value of character

1st test

1

"A"

2nd test

1

"B"

3rd test

2

"A"

4th test

2

"B"

5th test

4

"A"

6th test

4

"B"

@CartesianTest.Enum

@CartesianTest.Enum provides a convenient way to use Enum constants.

import java.time.temporal.ChronoUnit;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    class CartesianTestWithEnumClass {

        @CartesianTest
        void testWithEnum(@Enum ChronoUnit unit) {
            assertThat(unit).isNotNull();
        }

    }

Like @CartesianTest.Values, @CartesianTest.Enum is an annotation that might look familiar - it mimics JUnit’s @EnumSource, except @CartesianTest.Enum must be put on the parameter. It does NOT work with @ParameterizedTest.

The annotation has an optional value attribute. When omitted, the declared type of the parameter is used. The test will fail if it is not an enum type. The value attribute is required in the following example because the method parameter is declared as TemporalUnit, i.e. the interface implemented by ChronoUnit, which isn’t an enum type.

import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    class CartesianTestEnumWithTypeClass {

        @CartesianTest
        void testExplicitEnum(@Enum(ChronoUnit.class) TemporalUnit unit) {
            assertThat(unit).isNotNull();
        }

    }

The annotation provides an optional names attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used.

import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    class CartesianTestEnumWithTypeAndNamesClass {

        @CartesianTest
        void testEnumNames(@Enum(names = { "DAYS", "HOURS" }) ChronoUnit unit) {
            assertThat(EnumSet.of(ChronoUnit.DAYS, ChronoUnit.HOURS)).contains(unit);
        }

    }

The annotation also provides an optional mode attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples.

import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    class CartesianTestEnumWithModeClass {

        @CartesianTest
        void testWithEnumModes(@Enum(mode = EXCLUDE, names = { "ERAS", "FOREVER" }) ChronoUnit unit) {
            assertThat(EnumSet.of(ChronoUnit.ERAS, ChronoUnit.FOREVER)).doesNotContain(unit);
        }

    }
import java.time.temporal.*;

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    class CartesianTestEnumWithRegExClass {

        @CartesianTest
        void testWithEnumRegex(@Enum(mode = MATCH_ALL, names = "^.*DAYS$") ChronoUnit unit) {
            assertThat(unit.name()).endsWith("DAYS");
        }

    }

The example below shows how to use @CartesianTest.Enum with two Enum types.

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Enum;

    enum MyEnum {
        ONE, TWO, THREE
    }

    enum AnotherEnum {
        ALPHA, BETA, GAMMA, DELTA
    }

    class CartesianTestEnumWithEnumTypesClass {

        @CartesianTest
        void testEnumValues(@Enum MyEnum myEnum,
                @Enum(names = { "ALPHA", "DELTA" }, mode = Enum.Mode.EXCLUDE) AnotherEnum anotherEnum) {
            // passing test code
        }

    }

The test testEnumValues is executed exactly six times. The first parameter can have any of the three constants ONE, TWO or THREE. The second parameter can have any of the two constants BETA or GAMMA (note the EXCLUDE mode applied to the other two constants). @CartesianTest tests for all input combinations, that’s 3 × 2, so six tests in total.

To demonstrate with a table:

# of test

value of myEnum

value of anotherEnum

1st test

ONE

BETA

2nd test

ONE

GAMMA

3rd test

TWO

BETA

4th test

TWO

GAMMA

5th test

THREE

BETA

6th test

THREE

GAMMA

Range Source annotations

You can annotate your test parameters with range source annotations. For this purpose only, range sources can be used on parameters.

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.params.*;

    class CartesianTestEnumWithRangeSourcesClass {

        @CartesianTest
        void testShortAndLong(@ShortRangeSource(from = 1, to = 3, step = 1) short s,
                @LongRangeSource(from = 0L, to = 2L, step = 1, closed = true) long l) {
            // passing test code
        }

    }

The test testShortAndLong is executed exactly six times. The first parameter can have any of the two values 1, 2. The second parameter can have any of the three values 0L, 1L or 2L. @CartesianTest tests for all input combinations, that’s 2 × 3, so six tests in total.

To demonstrate with a table:

# of test

value of s

value of l

1st test

1

0L

2nd test

1

1L

3rd test

1

2L

4th test

2

0L

5th test

2

1L

6th test

2

2L

For more information, please see the separate documentation about range sources.

Annotating your test method

You can annotate your test method to supply arguments to all parameters simultaneously.

@CartesianTest.MethodFactory

@CartesianTest.MethodFactory can be used to name a static factory method that supplies your arguments. The value annotation parameter is mandatory. Just like with JUnit’s @MethodSource, you can specify the factory method with its fully-qualified name (including the class), e.g. com.example.Class#factory. This method must return ArgumentSets.

ArgumentSets is a helper class, specifically for creating sets for @CartesianTest. To create the test data, instantiate with the static factory method argumentsForFirstParameter, then call the addValuesForNextParameter method once per additional parameter in the order in which the they appear in the test method. In each call, pass in all values for the corresponding parameter. For convenience, all methods return with your ArgumentSets instance, so you can chain add() calls. If you want to create an initially-empty ArgumentSets, call the static factory method create().

Let’s look at an example.

class CartesianTestWithArgumentSetsClass {

    @CartesianTest
    @CartesianTest.MethodFactory("setFactory")
    void testMethod(String string, Class<?> clazz, TimeUnit unit) {
        // passing test code
    }

    ArgumentSets setFactory() {
        return ArgumentSets
                .argumentsForFirstParameter("Alpha", "Omega")
                .argumentsForNextParameter(Runnable.class, Cloneable.class, Predicate.class)
                .argumentsForNextParameter(TimeUnit.DAYS, TimeUnit.HOURS);
    }

}

The test testMethod is executed exactly twelve times. The first parameter can have any of the two values "Alpha" or "Omega". The second parameter can have any of the three values Runnable.class, Cloneable.class or Predicate.class. The third parameter can have any of the two values TimeUnit.DAYS or TimeUnit.HOURS. @CartesianTest tests for all input combinations, that’s 2 × 3 × 2, so twelve tests in total.

To demonstrate with a table:

# of test

value of string

value of clazz

value of unit

1st test

"Alpha"

Runnable.class

TimeUnit.DAYS

2nd test

"Alpha"

Runnable.class

TimeUnit.HOURS

3rd test

"Alpha"

Cloneable.class

TimeUnit.DAYS

4th test

"Alpha"

Cloneable.class

TimeUnit.HOURS

5th test

"Alpha"

Predicate.class

TimeUnit.DAYS

6th test

"Alpha"

Predicate.class

TimeUnit.HOURS

7th test

"Omega"

Runnable.class

TimeUnit.DAYS

8th test

"Omega"

Runnable.class

TimeUnit.HOURS

9th test

"Omega"

Cloneable.class

TimeUnit.DAYS

10th test

"Omega"

Cloneable.class

TimeUnit.HOURS

11th test

"Omega"

Predicate.class

TimeUnit.DAYS

12th test

"Omega"

Predicate.class

TimeUnit.HOURS

You can reuse the same argument provider method multiple times.

class CartesianTestWithReusedArgumentSetsClass {

    @CartesianTest
    @CartesianTest.MethodFactory("provideArguments")
    void testNeedingArguments(String string, int i) {
        // passing test code
    }

    @CartesianTest
    @CartesianTest.MethodFactory("provideArguments")
    void testNeedingSameArguments(String string, int i) {
        // different passing test code
    }

    ArgumentSets provideArguments() {
        return ArgumentSets
                .argumentsForFirstParameter("Mercury", "Earth", "Venus")
                .argumentsForNextParameter(1, 12, 144);
    }

}

Requirements for the static factory method

There are multiple requirements the static factory method has to fulfill to qualify:

  • must have the same name as the test method (or its name must be specified via the factory attribute)

  • must be static

  • must have no parameters

  • must return ArgumentSets

  • must register values for every parameter exactly once

  • must register values in order

Returning wrong ArgumentSets in the static factory method

If you register too few, too many, or conflicting parameters, you will get an ParameterResolutionException. "Conflicting parameters" means your test method has a parameter that should be injected by JUnit (e.g.: TestReporter) but you also try to inject it.

Examples of badly configured tests/static factory method:

class BadExamples {

    @CartesianTest
    @CartesianTest.MethodFactory("resolveParameters")
    void tooFewParameters(String string, int i, boolean b) {
        // fails because the boolean parameter is not resolved
    }

    @CartesianTest
    @CartesianTest.MethodFactory("resolveParameters")
    void tooManyParameters(String string) {
        // fails because we try to supply a non-existent integer parameter
    }

    @CartesianTest
    @CartesianTest.MethodFactory("resolveParameters")
    void wrongOrderParameters(int i, String string) {
        // fails because the static factory method declared parameter sets in the wrong order
    }

    @CartesianTest
    @CartesianTest.MethodFactory("resolveTestReporterParam")
    void conflictingParameters(String string, TestReporter info) {
        // fails because both the factory method and JUnit tries to inject TestReporter
    }

    ArgumentSets resolveParameters() {
        return ArgumentSets.argumentsForFirstParameter("A", "B", "C").argumentsForNextParameter(1, 2, 3);
    }

    ArgumentSets resolveTestReporterParam() {
        return ArgumentSets
                .argumentsForFirstParameter("A", "B", "C")
                // in this case MyTestReporter implements TestReporter
                .argumentsForNextParameter(new MyTestReporter());
    }

}

Writing your own @ArgumentsSource for @CartesianTest

You might find that the available annotations do not fit your need. For a @ParameterizedTest, you can write a custom @ArgumentsSource and a corresponding ArgumentsProvider. You can do the same thing for @CartesianTest, with the following caveats:

  • Instead of @ArgumentsSource, you have to use @CartesianArgumentsSource

  • Instead of ArgumentsProvider, you have to use one of two interfaces, each explored in their own section.

Implementing CartesianParameterArgumentsProvider

Let’s see how you can provide arguments to a single parameter with a custom annotation and provider. You will have to use CartesianParameterArgumentsProvider instead of ArgumentsProvider. This will let you initialize your argument source with its corresponding parameter.

Let’s demonstrate with an example.

For the sake of simplicity, let’s imagine that @CartesianTest.Values does not exist. We would like to have an argument source, where we can specify integers. Let’s create an annotation for it.

import java.lang.annotation.*;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsSource;

The annotation targets parameters because we want to use it directly on a parameter. The annotation has RUNTIME retention, so JUnit (and Pioneer) can discover it on your test via reflection. It is annotated with @CartesianArgumentsSource, so Pioneer knows which CartesianParameterArgumentsProvider it should invoke.

Next, we need to implement IntArgumentsProvider, that takes these values and passes them to our test.

import java.lang.reflect.Parameter;
import java.util.Arrays;
import java.util.Objects;
import java.util.stream.Stream;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junitpioneer.jupiter.cartesian.CartesianParameterArgumentsProvider;

    @Target(ElementType.PARAMETER)
    @Retention(RetentionPolicy.RUNTIME)
    @CartesianArgumentsSource(IntArgumentsProvider.class)
    public @interface Ints {

        int[] value();

    }

    class IntArgumentsProvider implements CartesianParameterArgumentsProvider {

        @Override
        public Stream<Integer> provideArguments(ExtensionContext context, Parameter parameter) {
            Ints source = Objects.requireNonNull(parameter.getAnnotation(Ints.class));
            return Arrays.stream(source.value()).boxed();
        }

    }

    class MyCartesianWithDisplayNameTest {

        @CartesianTest(name = "{index} => first bit: {0} second bit: {1}")
        @DisplayName("Basic bit test")
        void testWithCustomDisplayName(@Values(strings = { "0", "1" }) String a,
                @Values(strings = { "0", "1" }) String b) {
            // passing test code
        }

    }

}

The class has to implement CartesianParameterArgumentsProvider. Note that an implementation of CartesianParameterArgumentsProvider must be declared as either a top-level class or as a static nested class. It must also provide a default (no argument) constructor. The second parameter of the provideArguments method is the Parameter object that represents the parameter for which you are supplying arguments.

In our case, we don’t have to process the values from our annotation, so we just return the values as is. You could do additional processing, for example:

import java.lang.annotation.*;
import java.util.*;
import java.lang.reflect.Parameter;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsSource;
import org.junitpioneer.jupiter.cartesian.CartesianParameterArgumentsProvider;

    @Target(ElementType.PARAMETER)
    @Retention(RetentionPolicy.RUNTIME)
    @CartesianArgumentsSource(PeopleProvider.class)
    @interface People {

        String[] names();

        int[] ages();

    }

    class PeopleProvider implements CartesianParameterArgumentsProvider {

        @Override
        public Stream<Person> provideArguments(ExtensionContext context, Parameter parameter) {
            People source = Objects.requireNonNull(parameter.getAnnotation(People.class));
            return IntStream
                    .range(0, source.names().length)
                    .mapToObj(i -> new Person(source.names()[i], source.ages()[i]));
        }

    }

Alternatively, @CartesianTest supports it if you want to implement the AnnotationConsumer interface. The previous example would look like the following:

import java.lang.annotation.*;
import java.util.*;
import java.lang.reflect.Parameter;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.params.support.AnnotationConsumer;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsSource;
import org.junitpioneer.jupiter.cartesian.CartesianParameterArgumentsProvider;

    class PeopleProviderWithAnnotationConsumer
            implements CartesianParameterArgumentsProvider, AnnotationConsumer<People> {

        private People source;

        @Override
        public Stream<Person> provideArguments(ExtensionContext context, Parameter parameter) {
            return IntStream
                    .range(0, source.names().length)
                    .mapToObj(i -> new Person(source.names()[i], source.ages()[i]));
        }

        @Override
        public void accept(People source) {
            this.source = source;
        }

    }

Implementing CartesianMethodArgumentsProvider

Let’s see how you can provide arguments to every parameter with a custom annotation and provider. You will have to use CartesianMethodArgumentsProvider instead of ArgumentsProvider.

Let’s demonstrate with an example.

We would like to have an argument that supplies 'bits' to every parameter. Let’s create an annotation for it.

import java.lang.annotation.*;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsSource;

    @Target(ElementType.METHOD)
    @Retention(RetentionPolicy.RUNTIME)
    @CartesianArgumentsSource(BitArgumentsProvider.class)
    public @interface BitSource {
    }

The annotation targets methods because we want to use it on the test method itself. The annotation has RUNTIME retention, so JUnit (and Pioneer) can discover it on your test via reflection. It is annotated with @CartesianArgumentsSource, so Pioneer knows which CartesianMethodArgumentsProvider it should invoke.

Next, we need to implement BitArgumentsProvider.

import java.lang.reflect.Parameter;
import java.util.Arrays;
import java.util.Objects;
import java.util.stream.Stream;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junitpioneer.jupiter.cartesian.CartesianMethodArgumentsProvider;

    class BitArgumentsProvider implements CartesianMethodArgumentsProvider {

        @Override
        public ArgumentSets provideArguments(ExtensionContext context) {
            int paramCount = context.getRequiredTestMethod().getParameters().length;
            ArgumentSets sets = ArgumentSets.create();
            for (int i = 0; i < paramCount; i++) {
                sets.argumentsForNextParameter(0, 1);
            }
            return sets;
        }

    }

The class has to implement CartesianMethodArgumentsProvider. Note that an implementation of CartesianMethodArgumentsProvider must be declared as either a top-level class or as a static nested class. It must also provide a default (no argument) constructor. The method provideArguments returns an ArgumentSets object. This object should contain the arguments list for every parameter, except if an argument is supplied by JUnit (e.g.: a TestInfo).

Warning
This example does not work with parameters that are supplied by JUnit, you have to add custom logic for that scenario.

@CartesianTest supports it if you want to implement the AnnotationConsumer interface. For example:

import java.lang.annotation.*;
import java.lang.reflect.Parameter;
import java.util.Arrays;
import java.util.Objects;
import java.util.stream.Stream;

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junitpioneer.jupiter.cartesian.CartesianArgumentsSource;
import org.junitpioneer.jupiter.cartesian.CartesianMethodArgumentsProvider;

    @Target(ElementType.METHOD)
    @Retention(RetentionPolicy.RUNTIME)
    @CartesianArgumentsSource(NumberArgumentsProvider.class)
    public @interface NumberSource {

        int[] value();

    }

    class NumberArgumentsProvider implements CartesianMethodArgumentsProvider, AnnotationConsumer<NumberSource> {

        private int[] numbers;

        @Override
        public ArgumentSets provideArguments(ExtensionContext context) {
            int paramCount = context.getRequiredTestMethod().getParameters().length;
            ArgumentSets sets = ArgumentSets.create();
            for (int i = 0; i < paramCount; i++) {
                sets.argumentsForNextParameter(Arrays.stream(numbers).boxed());
            }
            return sets;
        }

        @Override
        public void accept(NumberSource source) {
            this.numbers = source.value();
        }

    }

Customizing Display Names

By default, the display name of a CartesianTest invocation contains the invocation index and the String representation of all arguments for that specific invocation. You can customize invocation display names via the name attribute of the @CartesianTest annotation. For example:

import org.junitpioneer.jupiter.cartesian.*;
import org.junitpioneer.jupiter.cartesian.CartesianTest.Values;

    class MyCartesianWithDisplayNameTest {

        @CartesianTest(name = "{index} => first bit: {0} second bit: {1}")
        @DisplayName("Basic bit test")
        void testWithCustomDisplayName(@Values(strings = { "0", "1" }) String a,
                @Values(strings = { "0", "1" }) String b) {
            // passing test code
        }

    }

When executing the above test, you should see output similar to the following:

Basic bit test
├─ 1 => first bit: 0 second bit: 0
├─ 2 => first bit: 0 second bit: 1
├─ 3 => first bit: 1 second bit: 0
└─ 4 => first bit: 1 second bit: 1

Please note that name is a MessageFormat pattern. A single quote (') needs to be represented as a doubled single quote ('') in order to be displayed.

CartesianTest supports the following placeholders in custom display names:

Placeholder Description

{displayName}

the display name of the method

{index}

the current invocation index, starting with 1

{arguments}

the complete, comma-separated arguments list

{0}, {1}, …​

an individual argument

Do not use @CartesianTest with @Test

If @CartesianTest is combined with @Test or TestTemplate-based mechanisms (like @RepeatedTest or @ParameterizedTest), the test engine will execute it according to each annotation (i.e. more than once). This is most likely unwanted and will probably lead to the following exception/failure message:

org.junit.jupiter.api.extension.ParameterResolutionException: No ParameterResolver registered for parameter […​]

This is because @Test does not know what to do with the parameter(s) of the @CartesianTest.

Thread-Safety

This extension is safe to use during parallel test execution.