[[writing-tests]] == Writing Tests The following example provides a glimpse at the minimum requirements for writing a test in JUnit Jupiter. Subsequent sections of this chapter will provide further details on all available features. [source,java,indent=0] .A first test case ---- include::{testDir}/example/MyFirstJUnitJupiterTests.java[tags=user_guide] ---- [[writing-tests-annotations]] === Annotations JUnit Jupiter supports the following annotations for configuring tests and extending the framework. Unless otherwise stated, all core annotations are located in the `{api-package}` package in the `junit-jupiter-api` module. [cols="20,80"] |=== | Annotation | Description | `@Test` | Denotes that a method is a test method. Unlike JUnit 4's `@Test` annotation, this annotation does not declare any attributes, since test extensions in JUnit Jupiter operate based on their own dedicated annotations. Such methods are _inherited_ unless they are _overridden_. | `@ParameterizedTest` | Denotes that a method is a <>. Such methods are _inherited_ unless they are _overridden_. | `@RepeatedTest` | Denotes that a method is a test template for a <>. Such methods are _inherited_ unless they are _overridden_. | `@TestFactory` | Denotes that a method is a test factory for <>. Such methods are _inherited_ unless they are _overridden_. | `@TestTemplate` | Denotes that a method is a <> designed to be invoked multiple times depending on the number of invocation contexts returned by the registered <>. Such methods are _inherited_ unless they are _overridden_. | `@TestClassOrder` | Used to configure the <> for `@Nested` test classes in the annotated test class. Such annotations are _inherited_. | `@TestMethodOrder` | Used to configure the <> for the annotated test class; similar to JUnit 4's `@FixMethodOrder`. Such annotations are _inherited_. | `@TestInstance` | Used to configure the <> for the annotated test class. Such annotations are _inherited_. | `@DisplayName` | Declares a custom <> for the test class or test method. Such annotations are not _inherited_. | `@DisplayNameGeneration` | Declares a custom <> for the test class. Such annotations are _inherited_. | `@BeforeEach` | Denotes that the annotated method should be executed _before_ *each* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, or `@TestFactory` method in the current class; analogous to JUnit 4's `@Before`. Such methods are _inherited_ – unless they are _overridden_ or _superseded_ (i.e., replaced based on signature only, irrespective of Java's visibility rules). | `@AfterEach` | Denotes that the annotated method should be executed _after_ *each* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, or `@TestFactory` method in the current class; analogous to JUnit 4's `@After`. Such methods are _inherited_ – unless they are _overridden_ or _superseded_ (i.e., replaced based on signature only, irrespective of Java's visibility rules). | `@BeforeAll` | Denotes that the annotated method should be executed _before_ *all* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, and `@TestFactory` methods in the current class; analogous to JUnit 4's `@BeforeClass`. Such methods are _inherited_ – unless they are _hidden_, _overridden_, or _superseded_, (i.e., replaced based on signature only, irrespective of Java's visibility rules) – and must be `static` unless the "per-class" <> is used. | `@AfterAll` | Denotes that the annotated method should be executed _after_ *all* `@Test`, `@RepeatedTest`, `@ParameterizedTest`, and `@TestFactory` methods in the current class; analogous to JUnit 4's `@AfterClass`. Such methods are _inherited_ – unless they are _hidden_, _overridden_, or _superseded_, (i.e., replaced based on signature only, irrespective of Java's visibility rules) – and must be `static` unless the "per-class" <> is used. | `@Nested` | Denotes that the annotated class is a non-static <>. On Java 8 through Java 15, `@BeforeAll` and `@AfterAll` methods cannot be used directly in a `@Nested` test class unless the "per-class" <> is used. Beginning with Java 16, `@BeforeAll` and `@AfterAll` methods can be declared as `static` in a `@Nested` test class with either test instance lifecycle mode. Such annotations are not _inherited_. | `@Tag` | Used to declare <>, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4. Such annotations are _inherited_ at the class level but not at the method level. | `@Disabled` | Used to <> a test class or test method; analogous to JUnit 4's `@Ignore`. Such annotations are not _inherited_. | `@Timeout` | Used to fail a test, test factory, test template, or lifecycle method if its execution exceeds a given duration. Such annotations are _inherited_. | `@ExtendWith` | Used to <>. Such annotations are _inherited_. | `@RegisterExtension` | Used to <> via fields. Such fields are _inherited_ unless they are _shadowed_. | `@TempDir` | Used to supply a <> via field injection or parameter injection in a lifecycle method or test method; located in the `org.junit.jupiter.api.io` package. |=== WARNING: Some annotations may currently be _experimental_. Consult the table in <> for details. [[writing-tests-meta-annotations]] ==== Meta-Annotations and Composed Annotations JUnit Jupiter annotations can be used as _meta-annotations_. That means that you can define your own _composed annotation_ that will automatically _inherit_ the semantics of its meta-annotations. For example, instead of copying and pasting `@Tag("fast")` throughout your code base (see <>), you can create a custom _composed annotation_ named `@Fast` as follows. `@Fast` can then be used as a drop-in replacement for `@Tag("fast")`. [source,java,indent=0] ---- include::{testDir}/example/Fast.java[tags=user_guide] ---- The following `@Test` method demonstrates usage of the `@Fast` annotation. [source,java,indent=0] ---- @Fast @Test void myFastTest() { // ... } ---- You can even take that one step further by introducing a custom `@FastTest` annotation that can be used as a drop-in replacement for `@Tag("fast")` _and_ `@Test`. [source,java,indent=0] ---- include::{testDir}/example/FastTest.java[tags=user_guide] ---- JUnit automatically recognizes the following as a `@Test` method that is tagged with "fast". [source,java,indent=0] ---- @FastTest void myFastTest() { // ... } ---- [[writing-tests-definitions]] === Definitions .Platform Concepts **** Container:: a node in the test tree that contains other containers or tests as its children (e.g. a _test class_). Test:: a node in the test tree that verifies expected behavior when executed (e.g. a `@Test` method). **** .Jupiter Concepts **** Lifecycle Method:: any method that is directly annotated or meta-annotated with `@BeforeAll`, `@AfterAll`, `@BeforeEach`, or `@AfterEach`. Test Class:: any top-level class, `static` member class, or <> that contains at least one _test method_, i.e. a _container_. Test classes must not be `abstract` and must have a single constructor. Test Method:: any instance method that is directly annotated or meta-annotated with `@Test`, `@RepeatedTest`, `@ParameterizedTest`, `@TestFactory`, or `@TestTemplate`. With the exception of `@Test`, these create a _container_ in the test tree that groups _tests_ or, potentially (for `@TestFactory`), other _containers_. **** [[writing-tests-classes-and-methods]] === Test Classes and Methods Test methods and lifecycle methods may be declared locally within the current test class, inherited from superclasses, or inherited from interfaces (see <>). In addition, test methods and lifecycle methods must not be `abstract` and must not return a value (except `@TestFactory` methods which are required to return a value). [NOTE] .Class and method visibility ==== Test classes, test methods, and lifecycle methods are not required to be `public`, but they must _not_ be `private`. It is generally recommended to omit the `public` modifier for test classes, test methods, and lifecycle methods unless there is a technical reason for doing so – for example, when a test class is extended by a test class in another package. Another technical reason for making classes and methods `public` is to simplify testing on the module path when using the Java Module System. ==== The following test class demonstrates the use of `@Test` methods and all supported lifecycle methods. For further information on runtime semantics, see <> and <>. [source,java,indent=0] .A standard test class ---- include::{testDir}/example/StandardTests.java[tags=user_guide] ---- [[writing-tests-display-names]] === Display Names Test classes and test methods can declare custom display names via `@DisplayName` -- with spaces, special characters, and even emojis -- that will be displayed in test reports and by test runners and IDEs. [source,java,indent=0] ---- include::{testDir}/example/DisplayNameDemo.java[tags=user_guide] ---- [[writing-tests-display-name-generator]] ==== Display Name Generators JUnit Jupiter supports custom display name generators that can be configured via the `@DisplayNameGeneration` annotation. Values provided via `@DisplayName` annotations always take precedence over display names generated by a `DisplayNameGenerator`. Generators can be created by implementing `DisplayNameGenerator`. Here are some default ones available in Jupiter: [cols="20,80"] |=== | DisplayNameGenerator | Behavior | `Standard` | Matches the standard display name generation behavior in place since JUnit Jupiter 5.0 was released. | `Simple` | Removes trailing parentheses for methods with no parameters. | `ReplaceUnderscores` | Replaces underscores with spaces. | `IndicativeSentences` | Generates complete sentences by concatenating the names of the test and the enclosing classes. |=== Note that for `IndicativeSentences`, you can customize the separator and the underlying generator by using `@IndicativeSentencesGeneration` as shown in the following example. [source,java,indent=0] ---- include::{testDir}/example/DisplayNameGeneratorDemo.java[tags=user_guide] ---- ``` +-- DisplayNameGeneratorDemo [OK] +-- A year is not supported [OK] | +-- A negative value for year is not supported by the leap year computation. [OK] | | +-- For example, year -1 is not supported. [OK] | | '-- For example, year -4 is not supported. [OK] | '-- if it is zero() [OK] '-- A year is a leap year [OK] +-- A year is a leap year -> if it is divisible by 4 but not by 100. [OK] '-- A year is a leap year -> if it is one of the following years. [OK] +-- Year 2016 is a leap year. [OK] +-- Year 2020 is a leap year. [OK] '-- Year 2048 is a leap year. [OK] ``` [[writing-tests-display-name-generator-default]] ==== Setting the Default Display Name Generator You can use the `junit.jupiter.displayname.generator.default` <> to specify the fully qualified class name of the `DisplayNameGenerator` you would like to use by default. Just like for display name generators configured via the `@DisplayNameGeneration` annotation, the supplied class has to implement the `DisplayNameGenerator` interface. The default display name generator will be used for all tests unless the `@DisplayNameGeneration` annotation is present on an enclosing test class or test interface. Values provided via `@DisplayName` annotations always take precedence over display names generated by a `DisplayNameGenerator`. For example, to use the `ReplaceUnderscores` display name generator by default, you should set the configuration parameter to the corresponding fully qualified class name (e.g., in `src/test/resources/junit-platform.properties`): [source,properties,indent=0] ---- junit.jupiter.displayname.generator.default = \ org.junit.jupiter.api.DisplayNameGenerator$ReplaceUnderscores ---- Similarly, you can specify the fully qualified name of any custom class that implements `DisplayNameGenerator`. [[writing-tests-display-name-generator-precedence-rules]] In summary, the display name for a test class or method is determined according to the following precedence rules: 1. value of the `@DisplayName` annotation, if present 2. by calling the `DisplayNameGenerator` specified in the `@DisplayNameGeneration` annotation, if present 3. by calling the default `DisplayNameGenerator` configured via the configuration parameter, if present 4. by calling `org.junit.jupiter.api.DisplayNameGenerator.Standard` [[writing-tests-assertions]] === Assertions JUnit Jupiter comes with many of the assertion methods that JUnit 4 has and adds a few that lend themselves well to being used with Java 8 lambdas. All JUnit Jupiter assertions are `static` methods in the `{Assertions}` class. [source,java,indent=0] ---- include::{testDir}/example/AssertionsDemo.java[tags=user_guide] ---- [[writing-tests-assertions-preemptive-timeouts]] [WARNING] .Preemptive Timeouts with `assertTimeoutPreemptively()` ==== The various `assertTimeoutPreemptively()` methods in the `Assertions` class execute the provided `executable` or `supplier` in a different thread than that of the calling code. This behavior can lead to undesirable side effects if the code that is executed within the `executable` or `supplier` relies on `java.lang.ThreadLocal` storage. One common example of this is the transactional testing support in the Spring Framework. Specifically, Spring's testing support binds transaction state to the current thread (via a `ThreadLocal`) before a test method is invoked. Consequently, if an `executable` or `supplier` provided to `assertTimeoutPreemptively()` invokes Spring-managed components that participate in transactions, any actions taken by those components will not be rolled back with the test-managed transaction. On the contrary, such actions will be committed to the persistent store (e.g., relational database) even though the test-managed transaction is rolled back. Similar side effects may be encountered with other frameworks that rely on `ThreadLocal` storage. ==== [[writing-tests-assertions-kotlin]] ==== Kotlin Assertion Support JUnit Jupiter also comes with a few assertion methods that lend themselves well to being used in https://kotlinlang.org/[Kotlin]. All JUnit Jupiter Kotlin assertions are top-level functions in the `org.junit.jupiter.api` package. [source,kotlin,indent=0] ---- include::{kotlinTestDir}/example/KotlinAssertionsDemo.kt[tags=user_guide] ---- [[writing-tests-assertions-third-party]] ==== Third-party Assertion Libraries Even though the assertion facilities provided by JUnit Jupiter are sufficient for many testing scenarios, there are times when more power and additional functionality such as _matchers_ are desired or required. In such cases, the JUnit team recommends the use of third-party assertion libraries such as {AssertJ}, {Hamcrest}, {Truth}, etc. Developers are therefore free to use the assertion library of their choice. For example, the combination of _matchers_ and a fluent API can be used to make assertions more descriptive and readable. However, JUnit Jupiter's `{Assertions}` class does not provide an https://junit.org/junit4/javadoc/latest/org/junit/Assert.html#assertThat[`assertThat()`] method like the one found in JUnit 4's `org.junit.Assert` class which accepts a Hamcrest https://junit.org/junit4/javadoc/latest/org/hamcrest/Matcher.html[`Matcher`]. Instead, developers are encouraged to use the built-in support for matchers provided by third-party assertion libraries. The following example demonstrates how to use the `assertThat()` support from Hamcrest in a JUnit Jupiter test. As long as the Hamcrest library has been added to the classpath, you can statically import methods such as `assertThat()`, `is()`, and `equalTo()` and then use them in tests like in the `assertWithHamcrestMatcher()` method below. [source,java,indent=0] ---- include::{testDir}/example/HamcrestAssertionsDemo.java[tags=user_guide] ---- Naturally, legacy tests based on the JUnit 4 programming model can continue using `org.junit.Assert#assertThat`. [[writing-tests-assumptions]] === Assumptions JUnit Jupiter comes with a subset of the assumption methods that JUnit 4 provides and adds a few that lend themselves well to being used with Java 8 lambda expressions and method references. All JUnit Jupiter assumptions are static methods in the `{Assumptions}` class. [source,java,indent=0] ---- include::{testDir}/example/AssumptionsDemo.java[tags=user_guide] ---- NOTE: As of JUnit Jupiter 5.4, it is also possible to use methods from JUnit 4's `org.junit.Assume` class for assumptions. Specifically, JUnit Jupiter supports JUnit 4's `AssumptionViolatedException` to signal that a test should be aborted instead of marked as a failure. [[writing-tests-disabling]] === Disabling Tests Entire test classes or individual test methods may be _disabled_ via the `{Disabled}` annotation, via one of the annotations discussed in <>, or via a custom <>. Here's a `@Disabled` test class. [source,java,indent=0] ---- include::{testDir}/example/DisabledClassDemo.java[tags=user_guide] ---- And here's a test class that contains a `@Disabled` test method. [source,java,indent=0] ---- include::{testDir}/example/DisabledTestsDemo.java[tags=user_guide] ---- [TIP] ==== `@Disabled` may be declared without providing a _reason_; however, the JUnit team recommends that developers provide a short explanation for why a test class or test method has been disabled. Consequently, the above examples both show the use of a reason -- for example, `@Disabled("Disabled until bug #42 has been resolved")`. Some development teams even require the presence of issue tracking numbers in the _reason_ for automated traceability, etc. ==== [NOTE] ==== `@Disabled` is not `@Inherited`. Consequently, if you wish to disable a class whose superclass is `@Disabled`, you must redeclare `@Disabled` on the subclass. ==== [[writing-tests-conditional-execution]] === Conditional Test Execution The <> extension API in JUnit Jupiter allows developers to either _enable_ or _disable_ a container or test based on certain conditions _programmatically_. The simplest example of such a condition is the built-in `{DisabledCondition}` which supports the `{Disabled}` annotation (see <>). In addition to `@Disabled`, JUnit Jupiter also supports several other annotation-based conditions in the `org.junit.jupiter.api.condition` package that allow developers to enable or disable containers and tests _declaratively_. When multiple `ExecutionCondition` extensions are registered, a container or test is disabled as soon as one of the conditions returns _disabled_. If you wish to provide details about why they might be disabled, every annotation associated with these built-in conditions has a `disabledReason` attribute available for that purpose. See <> and the following sections for details. [TIP] .Composed Annotations ==== Note that any of the _conditional_ annotations listed in the following sections may also be used as a meta-annotation in order to create a custom _composed annotation_. For example, the `@TestOnMac` annotation in the <> shows how you can combine `@Test` and `@EnabledOnOs` in a single, reusable annotation. ==== [NOTE] ==== _Conditional_ annotations in JUnit Jupiter are not `@Inherited`. Consequently, if you wish to apply the same semantics to subclasses, each conditional annotation must be redeclared on each subclass. ==== [WARNING] ==== Unless otherwise stated, each of the _conditional_ annotations listed in the following sections can only be declared once on a given test interface, test class, or test method. If a conditional annotation is directly present, indirectly present, or meta-present multiple times on a given element, only the first such annotation discovered by JUnit will be used; any additional declarations will be silently ignored. Note, however, that each conditional annotation may be used in conjunction with other conditional annotations in the `org.junit.jupiter.api.condition` package. ==== [[writing-tests-conditional-execution-os]] ==== Operating System and Architecture Conditions A container or test may be enabled or disabled on a particular operating system, architecture, or combination of both via the `{EnabledOnOs}` and `{DisabledOnOs}` annotations. [[writing-tests-conditional-execution-os-demo]] [source,java,indent=0] .Conditional execution based on operating system ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_os] ---- [[writing-tests-conditional-execution-architectures-demo]] [source,java,indent=0] .Conditional execution based on architecture ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_architecture] ---- [[writing-tests-conditional-execution-jre]] ==== Java Runtime Environment Conditions A container or test may be enabled or disabled on particular versions of the Java Runtime Environment (JRE) via the `{EnabledOnJre}` and `{DisabledOnJre}` annotations or on a particular range of versions of the JRE via the `{EnabledForJreRange}` and `{DisabledForJreRange}` annotations. The range defaults to `{JRE}.JAVA_8` as the lower border (`min`) and `{JRE}.OTHER` as the higher border (`max`), which allows usage of half open ranges. [source,java,indent=0] ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_jre] ---- [[writing-tests-conditional-execution-native]] ==== Native Image Conditions A container or test may be enabled or disabled within a https://www.graalvm.org/reference-manual/native-image/[GraalVM native image] via the `{EnabledInNativeImage}` and `{DisabledInNativeImage}` annotations. These annotations are typically used when running tests within a native image using the Gradle and Maven plug-ins from the GraalVM https://graalvm.github.io/native-build-tools/latest/[Native Build Tools] project. [source,java,indent=0] ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_native] ---- [[writing-tests-conditional-execution-system-properties]] ==== System Property Conditions A container or test may be enabled or disabled based on the value of the `named` JVM system property via the `{EnabledIfSystemProperty}` and `{DisabledIfSystemProperty}` annotations. The value supplied via the `matches` attribute will be interpreted as a regular expression. [source,java,indent=0] ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_system_property] ---- [TIP] ==== As of JUnit Jupiter 5.6, `{EnabledIfSystemProperty}` and `{DisabledIfSystemProperty}` are _repeatable annotations_. Consequently, these annotations may be declared multiple times on a test interface, test class, or test method. Specifically, these annotations will be found if they are directly present, indirectly present, or meta-present on a given element. ==== [[writing-tests-conditional-execution-environment-variables]] ==== Environment Variable Conditions A container or test may be enabled or disabled based on the value of the `named` environment variable from the underlying operating system via the `{EnabledIfEnvironmentVariable}` and `{DisabledIfEnvironmentVariable}` annotations. The value supplied via the `matches` attribute will be interpreted as a regular expression. [source,java,indent=0] ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_environment_variable] ---- [TIP] ==== As of JUnit Jupiter 5.6, `{EnabledIfEnvironmentVariable}` and `{DisabledIfEnvironmentVariable}` are _repeatable annotations_. Consequently, these annotations may be declared multiple times on a test interface, test class, or test method. Specifically, these annotations will be found if they are directly present, indirectly present, or meta-present on a given element. ==== [[writing-tests-conditional-execution-custom]] ==== Custom Conditions As an alternative to implementing an <>, a container or test may be enabled or disabled based on a _condition method_ configured via the `{EnabledIf}` and `{DisabledIf}` annotations. A condition method must have a `boolean` return type and may accept either no arguments or a single `ExtensionContext` argument. The following test class demonstrates how to configure a local method named `customCondition` via `@EnabledIf` and `@DisabledIf`. [source,java,indent=0] ---- include::{testDir}/example/ConditionalTestExecutionDemo.java[tags=user_guide_custom] ---- Alternatively, the condition method can be located outside the test class. In this case, it must be referenced by its _fully qualified name_ as demonstrated in the following example. [source,java,indent=0] ---- package example; include::{testDir}/example/ExternalCustomConditionDemo.java[tags=user_guide_external_custom_condition] ---- [NOTE] ==== There are several cases where a condition method would need to be `static`: - when `@EnabledIf` or `@DisabledIf` is used at class level - when `@EnabledIf` or `@DisabledIf` is used on a `@ParameterizedTest` or a `@TestTemplate` method - when the condition method is located in an external class In any other case, you can use either static methods or instance methods as condition methods. ==== [TIP] ==== It is often the case that you can use an existing static method in a utility class as a custom condition. For example, `java.awt.GraphicsEnvironment` provides a `public static boolean isHeadless()` method that can be used to determine if the current environment does not support a graphical display. Thus, if you have a test that depends on graphical support you can disable it when such support is unavailable as follows. [source,java,indent=0] ---- @DisabledIf(value = "java.awt.GraphicsEnvironment#isHeadless", disabledReason = "headless environment") ---- ==== [[writing-tests-tagging-and-filtering]] === Tagging and Filtering Test classes and methods can be tagged via the `@Tag` annotation. Those tags can later be used to filter <>. Please refer to the <> section for more information about tag support in the JUnit Platform. [source,java,indent=0] ---- include::{testDir}/example/TaggingDemo.java[tags=user_guide] ---- TIP: See <> for examples demonstrating how to create custom annotations for tags. [[writing-tests-test-execution-order]] === Test Execution Order By default, test classes and methods will be ordered using an algorithm that is deterministic but intentionally nonobvious. This ensures that subsequent runs of a test suite execute test classes and test methods in the same order, thereby allowing for repeatable builds. NOTE: See <> for a definition of _test method_ and _test class_. [[writing-tests-test-execution-order-methods]] ==== Method Order Although true _unit tests_ typically should not rely on the order in which they are executed, there are times when it is necessary to enforce a specific test method execution order -- for example, when writing _integration tests_ or _functional tests_ where the sequence of the tests is important, especially in conjunction with `@TestInstance(Lifecycle.PER_CLASS)`. To control the order in which test methods are executed, annotate your test class or test interface with `{TestMethodOrder}` and specify the desired `{MethodOrderer}` implementation. You can implement your own custom `MethodOrderer` or use one of the following built-in `MethodOrderer` implementations. * `{MethodOrderer_DisplayName}`: sorts test methods _alphanumerically_ based on their display names (see <>) * `{MethodOrderer_MethodName}`: sorts test methods _alphanumerically_ based on their names and formal parameter lists * `{MethodOrderer_OrderAnnotation}`: sorts test methods _numerically_ based on values specified via the `{Order}` annotation * `{MethodOrderer_Random}`: orders test methods _pseudo-randomly_ and supports configuration of a custom _seed_ * `{MethodOrderer_Alphanumeric}`: sorts test methods _alphanumerically_ based on their names and formal parameter lists; **deprecated in favor of `{MethodOrderer_MethodName}`, to be removed in 6.0** NOTE: See also: <> The following example demonstrates how to guarantee that test methods are executed in the order specified via the `@Order` annotation. [source,java,indent=0] ---- include::{testDir}/example/OrderedTestsDemo.java[tags=user_guide] ---- [[writing-tests-test-execution-order-methods-default]] ===== Setting the Default Method Orderer You can use the `junit.jupiter.testmethod.order.default` <> to specify the fully qualified class name of the `{MethodOrderer}` you would like to use by default. Just like for the orderer configured via the `{TestMethodOrder}` annotation, the supplied class has to implement the `MethodOrderer` interface. The default orderer will be used for all tests unless the `@TestMethodOrder` annotation is present on an enclosing test class or test interface. For example, to use the `{MethodOrderer_OrderAnnotation}` method orderer by default, you should set the configuration parameter to the corresponding fully qualified class name (e.g., in `src/test/resources/junit-platform.properties`): [source,properties,indent=0] ---- junit.jupiter.testmethod.order.default = \ org.junit.jupiter.api.MethodOrderer$OrderAnnotation ---- Similarly, you can specify the fully qualified name of any custom class that implements `MethodOrderer`. [[writing-tests-test-execution-order-classes]] ==== Class Order Although test classes typically should not rely on the order in which they are executed, there are times when it is desirable to enforce a specific test class execution order. You may wish to execute test classes in a random order to ensure there are no accidental dependencies between test classes, or you may wish to order test classes to optimize build time as outlined in the following scenarios. * Run previously failing tests and faster tests first: "fail fast" mode * With parallel execution enabled, schedule longer tests first: "shortest test plan execution duration" mode * Various other use cases To configure test class execution order _globally_ for the entire test suite, use the `junit.jupiter.testclass.order.default` <> to specify the fully qualified class name of the `{ClassOrderer}` you would like to use. The supplied class must implement the `ClassOrderer` interface. You can implement your own custom `ClassOrderer` or use one of the following built-in `ClassOrderer` implementations. * `{ClassOrderer_ClassName}`: sorts test classes _alphanumerically_ based on their fully qualified class names * `{ClassOrderer_DisplayName}`: sorts test classes _alphanumerically_ based on their display names (see <>) * `{ClassOrderer_OrderAnnotation}`: sorts test classes _numerically_ based on values specified via the `{Order}` annotation * `{ClassOrderer_Random}`: orders test classes _pseudo-randomly_ and supports configuration of a custom _seed_ For example, for the `@Order` annotation to be honored on _test classes_, you should configure the `{ClassOrderer_OrderAnnotation}` class orderer using the configuration parameter with the corresponding fully qualified class name (e.g., in `src/test/resources/junit-platform.properties`): [source,properties,indent=0] ---- junit.jupiter.testclass.order.default = \ org.junit.jupiter.api.ClassOrderer$OrderAnnotation ---- The configured `ClassOrderer` will be applied to all top-level test classes (including `static` nested test classes) and `@Nested` test classes. NOTE: Top-level test classes will be ordered relative to each other; whereas, `@Nested` test classes will be ordered relative to other `@Nested` test classes sharing the same _enclosing class_. To configure test class execution order _locally_ for `@Nested` test classes, declare the `{TestClassOrder}` annotation on the enclosing class for the `@Nested` test classes you want to order, and supply a class reference to the `ClassOrderer` implementation you would like to use directly in the `@TestClassOrder` annotation. The configured `ClassOrderer` will be applied recursively to `@Nested` test classes and their `@Nested` test classes. Note that a local `@TestClassOrder` declaration always overrides an inherited `@TestClassOrder` declaration or a `ClassOrderer` configured globally via the `junit.jupiter.testclass.order.default` configuration parameter. The following example demonstrates how to guarantee that `@Nested` test classes are executed in the order specified via the `@Order` annotation. [source,java,indent=0] ---- include::{testDir}/example/OrderedNestedTestClassesDemo.java[tags=user_guide] ---- [[writing-tests-test-instance-lifecycle]] === Test Instance Lifecycle In order to allow individual test methods to be executed in isolation and to avoid unexpected side effects due to mutable test instance state, JUnit creates a new instance of each test class before executing each _test method_ (see <>). This "per-method" test instance lifecycle is the default behavior in JUnit Jupiter and is analogous to all previous versions of JUnit. NOTE: Please note that the test class will still be instantiated if a given _test method_ is _disabled_ via a <> (e.g., `@Disabled`, `@DisabledOnOs`, etc.) even when the "per-method" test instance lifecycle mode is active. If you would prefer that JUnit Jupiter execute all test methods on the same test instance, annotate your test class with `@TestInstance(Lifecycle.PER_CLASS)`. When using this mode, a new test instance will be created once per test class. Thus, if your test methods rely on state stored in instance variables, you may need to reset that state in `@BeforeEach` or `@AfterEach` methods. The "per-class" mode has some additional benefits over the default "per-method" mode. Specifically, with the "per-class" mode it becomes possible to declare `@BeforeAll` and `@AfterAll` on non-static methods as well as on interface `default` methods. The "per-class" mode therefore also makes it possible to use `@BeforeAll` and `@AfterAll` methods in `@Nested` test classes. NOTE: Beginning with Java 16, `@BeforeAll` and `@AfterAll` methods can be declared as `static` in `@Nested` test classes. If you are authoring tests using the Kotlin programming language, you may also find it easier to implement non-static `@BeforeAll` and `@AfterAll` lifecycle methods as well as `@MethodSource` factory methods by switching to the "per-class" test instance lifecycle mode. [[writing-tests-test-instance-lifecycle-changing-default]] ==== Changing the Default Test Instance Lifecycle If a test class or test interface is not annotated with `@TestInstance`, JUnit Jupiter will use a _default_ lifecycle mode. The standard _default_ mode is `PER_METHOD`; however, it is possible to change the _default_ for the execution of an entire test plan. To change the default test instance lifecycle mode, set the `junit.jupiter.testinstance.lifecycle.default` _configuration parameter_ to the name of an enum constant defined in `TestInstance.Lifecycle`, ignoring case. This can be supplied as a JVM system property, as a _configuration parameter_ in the `LauncherDiscoveryRequest` that is passed to the `Launcher`, or via the JUnit Platform configuration file (see <> for details). For example, to set the default test instance lifecycle mode to `Lifecycle.PER_CLASS`, you can start your JVM with the following system property. `-Djunit.jupiter.testinstance.lifecycle.default=per_class` Note, however, that setting the default test instance lifecycle mode via the JUnit Platform configuration file is a more robust solution since the configuration file can be checked into a version control system along with your project and can therefore be used within IDEs and your build software. To set the default test instance lifecycle mode to `Lifecycle.PER_CLASS` via the JUnit Platform configuration file, create a file named `junit-platform.properties` in the root of the class path (e.g., `src/test/resources`) with the following content. `junit.jupiter.testinstance.lifecycle.default = per_class` WARNING: Changing the _default_ test instance lifecycle mode can lead to unpredictable results and fragile builds if not applied consistently. For example, if the build configures "per-class" semantics as the default but tests in the IDE are executed using "per-method" semantics, that can make it difficult to debug errors that occur on the build server. It is therefore recommended to change the default in the JUnit Platform configuration file instead of via a JVM system property. [[writing-tests-nested]] === Nested Tests `@Nested` tests give the test writer more capabilities to express the relationship among several groups of tests. Such nested tests make use of Java's nested classes and facilitate hierarchical thinking about the test structure. Here's an elaborate example, both as source code and as a screenshot of the execution within an IDE. [source,java,indent=0] .Nested test suite for testing a stack ---- include::{testDir}/example/TestingAStackDemo.java[tags=user_guide] ---- When executing this example in an IDE, the test execution tree in the GUI will look similar to the following image. image::writing-tests_nested_test_ide.png[caption='',title='Executing a nested test in an IDE'] In this example, preconditions from outer tests are used in inner tests by defining hierarchical lifecycle methods for the setup code. For example, `createNewStack()` is a `@BeforeEach` lifecycle method that is used in the test class in which it is defined and in all levels in the nesting tree below the class in which it is defined. The fact that setup code from outer tests is run before inner tests are executed gives you the ability to run all tests independently. You can even run inner tests alone without running the outer tests, because the setup code from the outer tests is always executed. NOTE: _Only non-static nested classes_ (i.e. _inner classes_) can serve as `@Nested` test classes. Nesting can be arbitrarily deep, and those inner classes are subject to full lifecycle support with one exception: `@BeforeAll` and `@AfterAll` methods do not work _by default_. The reason is that Java does not allow `static` members in inner classes prior to Java 16. However, this restriction can be circumvented by annotating a `@Nested` test class with `@TestInstance(Lifecycle.PER_CLASS)` (see <>). If you are using Java 16 or higher, `@BeforeAll` and `@AfterAll` methods can be declared as `static` in `@Nested` test classes, and this restriction no longer applies. [[writing-tests-dependency-injection]] === Dependency Injection for Constructors and Methods In all prior JUnit versions, test constructors or methods were not allowed to have parameters (at least not with the standard `Runner` implementations). As one of the major changes in JUnit Jupiter, both test constructors and methods are now permitted to have parameters. This allows for greater flexibility and enables _Dependency Injection_ for constructors and methods. `{ParameterResolver}` defines the API for test extensions that wish to _dynamically_ resolve parameters at runtime. If a _test class_ constructor, a _test method_, or a _lifecycle method_ (see <>) accepts a parameter, the parameter must be resolved at runtime by a registered `ParameterResolver`. There are currently three built-in resolvers that are registered automatically. * `{TestInfoParameterResolver}`: if a constructor or method parameter is of type `{TestInfo}`, the `TestInfoParameterResolver` will supply an instance of `TestInfo` corresponding to the current container or test as the value for the parameter. The `TestInfo` can then be used to retrieve information about the current container or test such as the display name, the test class, the test method, and associated tags. The display name is either a technical name, such as the name of the test class or test method, or a custom name configured via `@DisplayName`. + `{TestInfo}` acts as a drop-in replacement for the `TestName` rule from JUnit 4. The following demonstrates how to have `TestInfo` injected into a test constructor, `@BeforeEach` method, and `@Test` method. [source,java,indent=0] ---- include::{testDir}/example/TestInfoDemo.java[tags=user_guide] ---- * `{RepetitionExtension}`: if a method parameter in a `@RepeatedTest`, `@BeforeEach`, or `@AfterEach` method is of type `{RepetitionInfo}`, the `RepetitionExtension` will supply an instance of `RepetitionInfo`. `RepetitionInfo` can then be used to retrieve information about the current repetition, the total number of repetitions, the number of repetitions that have failed, and the failure threshold for the corresponding `@RepeatedTest`. Note, however, that `RepetitionExtension` is not registered outside the context of a `@RepeatedTest`. See <>. * `{TestReporterParameterResolver}`: if a constructor or method parameter is of type `{TestReporter}`, the `TestReporterParameterResolver` will supply an instance of `TestReporter`. The `TestReporter` can be used to publish additional data about the current test run. The data can be consumed via the `reportingEntryPublished()` method in a `{TestExecutionListener}`, allowing it to be viewed in IDEs or included in reports. + In JUnit Jupiter you should use `TestReporter` where you used to print information to `stdout` or `stderr` in JUnit 4. Using `@RunWith(JUnitPlatform.class)` will output all reported entries to `stdout`. In addition, some IDEs print report entries to `stdout` or display them in the user interface for test results. [source,java,indent=0] ---- include::{testDir}/example/TestReporterDemo.java[tags=user_guide] ---- NOTE: Other parameter resolvers must be explicitly enabled by registering appropriate <> via `@ExtendWith`. Check out the `{RandomParametersExtension}` for an example of a custom `{ParameterResolver}`. While not intended to be production-ready, it demonstrates the simplicity and expressiveness of both the extension model and the parameter resolution process. `MyRandomParametersTest` demonstrates how to inject random values into `@Test` methods. [source,java,indent=0] ---- @ExtendWith(RandomParametersExtension.class) class MyRandomParametersTest { @Test void injectsInteger(@Random int i, @Random int j) { assertNotEquals(i, j); } @Test void injectsDouble(@Random double d) { assertEquals(0.0, d, 1.0); } } ---- For real-world use cases, check out the source code for the `{MockitoExtension}` and the `{SpringExtension}`. When the type of the parameter to inject is the only condition for your `{ParameterResolver}`, you can use the generic `{TypeBasedParameterResolver}` base class. The `supportsParameters` method is implemented behind the scenes and supports parameterized types. [[writing-tests-test-interfaces-and-default-methods]] === Test Interfaces and Default Methods JUnit Jupiter allows `@Test`, `@RepeatedTest`, `@ParameterizedTest`, `@TestFactory`, `@TestTemplate`, `@BeforeEach`, and `@AfterEach` to be declared on interface `default` methods. `@BeforeAll` and `@AfterAll` can either be declared on `static` methods in a test interface or on interface `default` methods _if_ the test interface or test class is annotated with `@TestInstance(Lifecycle.PER_CLASS)` (see <>). Here are some examples. [source,java] ---- include::{testDir}/example/testinterface/TestLifecycleLogger.java[tags=user_guide] ---- [source,java] ---- include::{testDir}/example/testinterface/TestInterfaceDynamicTestsDemo.java[tags=user_guide] ---- `@ExtendWith` and `@Tag` can be declared on a test interface so that classes that implement the interface automatically inherit its tags and extensions. See <> for the source code of the <>. [source,java] ---- include::{testDir}/example/testinterface/TimeExecutionLogger.java[tags=user_guide] ---- In your test class you can then implement these test interfaces to have them applied. [source,java] ---- include::{testDir}/example/testinterface/TestInterfaceDemo.java[tags=user_guide] ---- Running the `TestInterfaceDemo` results in output similar to the following: .... INFO example.TestLifecycleLogger - Before all tests INFO example.TestLifecycleLogger - About to execute [dynamicTestsForPalindromes()] INFO example.TimingExtension - Method [dynamicTestsForPalindromes] took 19 ms. INFO example.TestLifecycleLogger - Finished executing [dynamicTestsForPalindromes()] INFO example.TestLifecycleLogger - About to execute [isEqualValue()] INFO example.TimingExtension - Method [isEqualValue] took 1 ms. INFO example.TestLifecycleLogger - Finished executing [isEqualValue()] INFO example.TestLifecycleLogger - After all tests .... Another possible application of this feature is to write tests for interface contracts. For example, you can write tests for how implementations of `Object.equals` or `Comparable.compareTo` should behave as follows. [source,java] ---- include::{testDir}/example/defaultmethods/Testable.java[tags=user_guide] ---- [source,java] ---- include::{testDir}/example/defaultmethods/EqualsContract.java[tags=user_guide] ---- [source,java] ---- include::{testDir}/example/defaultmethods/ComparableContract.java[tags=user_guide] ---- In your test class you can then implement both contract interfaces thereby inheriting the corresponding tests. Of course you'll have to implement the abstract methods. [source,java] ---- include::{testDir}/example/defaultmethods/StringTests.java[tags=user_guide] ---- NOTE: The above tests are merely meant as examples and therefore not complete. [[writing-tests-repeated-tests]] === Repeated Tests JUnit Jupiter provides the ability to repeat a test a specified number of times by annotating a method with `@RepeatedTest` and specifying the total number of repetitions desired. Each invocation of a repeated test behaves like the execution of a regular `@Test` method with full support for the same lifecycle callbacks and extensions. The following example demonstrates how to declare a test named `repeatedTest()` that will be automatically repeated 10 times. [source,java] ---- @RepeatedTest(10) void repeatedTest() { // ... } ---- Since JUnit Jupiter 5.10, `@RepeatedTest` can be configured with a failure threshold which signifies the number of failures after which remaining repetitions will be automatically skipped. Set the `failureThreshold` attribute to a positive number less than the total number of repetitions in order to skip the invocations of remaining repetitions after the specified number of failures has been encountered. For example, if you are using `@RepeatedTest` to repeatedly invoke a test that you suspect to be _flaky_, a single failure is sufficient to demonstrate that the test is flaky, and there is no need to invoke the remaining repetitions. To support that specific use case, set `failureThreshold = 1`. You can alternatively set the threshold to a number greater than 1 depending on your use case. By default, the `failureThreshold` attribute is set to `Integer.MAX_VALUE`, signaling that no failure threshold will be applied, which effectively means that the specified number of repetitions will be invoked regardless of whether any repetitions fail. WARNING: If the repetitions of a `@RepeatedTest` method are executed in parallel, no guarantees can be made regarding the failure threshold. It is therefore recommended that a `@RepeatedTest` method be annotated with `@Execution(SAME_THREAD)` when parallel execution is configured. See <> for further details. In addition to specifying the number of repetitions and failure threshold, a custom display name can be configured for each repetition via the `name` attribute of the `@RepeatedTest` annotation. Furthermore, the display name can be a pattern composed of a combination of static text and dynamic placeholders. The following placeholders are currently supported. - `{displayName}`: display name of the `@RepeatedTest` method - `{currentRepetition}`: the current repetition count - `{totalRepetitions}`: the total number of repetitions The default display name for a given repetition is generated based on the following pattern: `"repetition {currentRepetition} of {totalRepetitions}"`. Thus, the display names for individual repetitions of the previous `repeatedTest()` example would be: `repetition 1 of 10`, `repetition 2 of 10`, etc. If you would like the display name of the `@RepeatedTest` method included in the name of each repetition, you can define your own custom pattern or use the predefined `RepeatedTest.LONG_DISPLAY_NAME` pattern. The latter is equal to `"{displayName} :: repetition {currentRepetition} of {totalRepetitions}"` which results in display names for individual repetitions like `repeatedTest() :: repetition 1 of 10`, `repeatedTest() :: repetition 2 of 10`, etc. In order to retrieve information about the current repetition, the total number of repetitions, the number of repetitions that have failed, and the failure threshold, a developer can choose to have an instance of `{RepetitionInfo}` injected into a `@RepeatedTest`, `@BeforeEach`, or `@AfterEach` method. [[writing-tests-repeated-tests-examples]] ==== Repeated Test Examples The `RepeatedTestsDemo` class at the end of this section demonstrates several examples of repeated tests. The `repeatedTest()` method is identical to the example from the previous section; whereas, `repeatedTestWithRepetitionInfo()` demonstrates how to have an instance of `RepetitionInfo` injected into a test to access the total number of repetitions for the current repeated test. `repeatedTestWithFailureThreshold()` demonstrates how to set a failure threshold and simulates an unexpected failure for every second repetition. The resulting behavior can be viewed in the `ConsoleLauncher` output at the end of this section. The next two methods demonstrate how to include a custom `@DisplayName` for the `@RepeatedTest` method in the display name of each repetition. `customDisplayName()` combines a custom display name with a custom pattern and then uses `TestInfo` to verify the format of the generated display name. `Repeat!` is the `{displayName}` which comes from the `@DisplayName` declaration, and `1/1` comes from `{currentRepetition}/{totalRepetitions}`. In contrast, `customDisplayNameWithLongPattern()` uses the aforementioned predefined `RepeatedTest.LONG_DISPLAY_NAME` pattern. `repeatedTestInGerman()` demonstrates the ability to translate display names of repeated tests into foreign languages -- in this case German, resulting in names for individual repetitions such as: `Wiederholung 1 von 5`, `Wiederholung 2 von 5`, etc. Since the `beforeEach()` method is annotated with `@BeforeEach` it will get executed before each repetition of each repeated test. By having the `TestInfo` and `RepetitionInfo` injected into the method, we see that it's possible to obtain information about the currently executing repeated test. Executing `RepeatedTestsDemo` with the `INFO` log level enabled results in the following output. .... INFO: About to execute repetition 1 of 10 for repeatedTest INFO: About to execute repetition 2 of 10 for repeatedTest INFO: About to execute repetition 3 of 10 for repeatedTest INFO: About to execute repetition 4 of 10 for repeatedTest INFO: About to execute repetition 5 of 10 for repeatedTest INFO: About to execute repetition 6 of 10 for repeatedTest INFO: About to execute repetition 7 of 10 for repeatedTest INFO: About to execute repetition 8 of 10 for repeatedTest INFO: About to execute repetition 9 of 10 for repeatedTest INFO: About to execute repetition 10 of 10 for repeatedTest INFO: About to execute repetition 1 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 2 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 3 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 4 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 5 of 5 for repeatedTestWithRepetitionInfo INFO: About to execute repetition 1 of 8 for repeatedTestWithFailureThreshold INFO: About to execute repetition 2 of 8 for repeatedTestWithFailureThreshold INFO: About to execute repetition 3 of 8 for repeatedTestWithFailureThreshold INFO: About to execute repetition 4 of 8 for repeatedTestWithFailureThreshold INFO: About to execute repetition 1 of 1 for customDisplayName INFO: About to execute repetition 1 of 1 for customDisplayNameWithLongPattern INFO: About to execute repetition 1 of 5 for repeatedTestInGerman INFO: About to execute repetition 2 of 5 for repeatedTestInGerman INFO: About to execute repetition 3 of 5 for repeatedTestInGerman INFO: About to execute repetition 4 of 5 for repeatedTestInGerman INFO: About to execute repetition 5 of 5 for repeatedTestInGerman .... [source,java] ---- include::{testDir}/example/RepeatedTestsDemo.java[tags=user_guide] ---- When using the `ConsoleLauncher` with the unicode theme enabled, execution of `RepeatedTestsDemo` results in the following output to the console. .... ├─ RepeatedTestsDemo ✔ │ ├─ repeatedTest() ✔ │ │ ├─ repetition 1 of 10 ✔ │ │ ├─ repetition 2 of 10 ✔ │ │ ├─ repetition 3 of 10 ✔ │ │ ├─ repetition 4 of 10 ✔ │ │ ├─ repetition 5 of 10 ✔ │ │ ├─ repetition 6 of 10 ✔ │ │ ├─ repetition 7 of 10 ✔ │ │ ├─ repetition 8 of 10 ✔ │ │ ├─ repetition 9 of 10 ✔ │ │ └─ repetition 10 of 10 ✔ │ ├─ repeatedTestWithRepetitionInfo(RepetitionInfo) ✔ │ │ ├─ repetition 1 of 5 ✔ │ │ ├─ repetition 2 of 5 ✔ │ │ ├─ repetition 3 of 5 ✔ │ │ ├─ repetition 4 of 5 ✔ │ │ └─ repetition 5 of 5 ✔ │ ├─ repeatedTestWithFailureThreshold(RepetitionInfo) ✔ │ │ ├─ repetition 1 of 8 ✔ │ │ ├─ repetition 2 of 8 ✘ Boom! │ │ ├─ repetition 3 of 8 ✔ │ │ ├─ repetition 4 of 8 ✘ Boom! │ │ ├─ repetition 5 of 8 ↷ Failure threshold [2] exceeded │ │ ├─ repetition 6 of 8 ↷ Failure threshold [2] exceeded │ │ ├─ repetition 7 of 8 ↷ Failure threshold [2] exceeded │ │ └─ repetition 8 of 8 ↷ Failure threshold [2] exceeded │ ├─ Repeat! ✔ │ │ └─ Repeat! 1/1 ✔ │ ├─ Details... ✔ │ │ └─ Details... :: repetition 1 of 1 ✔ │ └─ repeatedTestInGerman() ✔ │ ├─ Wiederholung 1 von 5 ✔ │ ├─ Wiederholung 2 von 5 ✔ │ ├─ Wiederholung 3 von 5 ✔ │ ├─ Wiederholung 4 von 5 ✔ │ └─ Wiederholung 5 von 5 ✔ .... [[writing-tests-parameterized-tests]] === Parameterized Tests Parameterized tests make it possible to run a test multiple times with different arguments. They are declared just like regular `@Test` methods but use the `{ParameterizedTest}` annotation instead. In addition, you must declare at least one _source_ that will provide the arguments for each invocation and then _consume_ the arguments in the test method. The following example demonstrates a parameterized test that uses the `@ValueSource` annotation to specify a `String` array as the source of arguments. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=first_example] ---- When executing the above parameterized test method, each invocation will be reported separately. For instance, the `ConsoleLauncher` will print output similar to the following. .... palindromes(String) ✔ ├─ [1] candidate=racecar ✔ ├─ [2] candidate=radar ✔ └─ [3] candidate=able was I ere I saw elba ✔ .... [[writing-tests-parameterized-tests-setup]] ==== Required Setup In order to use parameterized tests you need to add a dependency on the `junit-jupiter-params` artifact. Please refer to <> for details. [[writing-tests-parameterized-tests-consuming-arguments]] ==== Consuming Arguments Parameterized test methods typically _consume_ arguments directly from the configured source (see <>) following a one-to-one correlation between argument source index and method parameter index (see examples in <>). However, a parameterized test method may also choose to _aggregate_ arguments from the source into a single object passed to the method (see <>). Additional arguments may also be provided by a `ParameterResolver` (e.g., to obtain an instance of `TestInfo`, `TestReporter`, etc.). Specifically, a parameterized test method must declare formal parameters according to the following rules. * Zero or more _indexed arguments_ must be declared first. * Zero or more _aggregators_ must be declared next. * Zero or more arguments supplied by a `ParameterResolver` must be declared last. In this context, an _indexed argument_ is an argument for a given index in the `Arguments` provided by an `ArgumentsProvider` that is passed as an argument to the parameterized method at the same index in the method's formal parameter list. An _aggregator_ is any parameter of type `ArgumentsAccessor` or any parameter annotated with `@AggregateWith`. [NOTE] .AutoCloseable arguments ==== Arguments that implement `java.lang.AutoCloseable` (or `java.io.Closeable` which extends `java.lang.AutoCloseable`) will be automatically closed after `@AfterEach` methods and `AfterEachCallback` extensions have been called for the current parameterized test invocation. To prevent this from happening, set the `autoCloseArguments` attribute in `@ParameterizedTest` to `false`. Specifically, if an argument that implements `AutoCloseable` is reused for multiple invocations of the same parameterized test method, you must annotate the method with `@ParameterizedTest(autoCloseArguments = false)` to ensure that the argument is not closed between invocations. ==== [[writing-tests-parameterized-tests-sources]] ==== Sources of Arguments Out of the box, JUnit Jupiter provides quite a few _source_ annotations. Each of the following subsections provides a brief overview and an example for each of them. Please refer to the Javadoc in the `{params-provider-package}` package for additional information. [[writing-tests-parameterized-tests-sources-ValueSource]] ===== @ValueSource `@ValueSource` is one of the simplest possible sources. It lets you specify a single array of literal values and can only be used for providing a single argument per parameterized test invocation. The following types of literal values are supported by `@ValueSource`. - `short` - `byte` - `int` - `long` - `float` - `double` - `char` - `boolean` - `java.lang.String` - `java.lang.Class` For example, the following `@ParameterizedTest` method will be invoked three times, with the values `1`, `2`, and `3` respectively. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ValueSource_example] ---- [[writing-tests-parameterized-tests-sources-null-and-empty]] ===== Null and Empty Sources In order to check corner cases and verify proper behavior of our software when it is supplied _bad input_, it can be useful to have `null` and _empty_ values supplied to our parameterized tests. The following annotations serve as sources of `null` and empty values for parameterized tests that accept a single argument. * `{NullSource}`: provides a single `null` argument to the annotated `@ParameterizedTest` method. - `@NullSource` cannot be used for a parameter that has a primitive type. * `{EmptySource}`: provides a single _empty_ argument to the annotated `@ParameterizedTest` method for parameters of the following types: `java.lang.String`, `java.util.Collection` (and concrete subtypes with a `public` no-arg constructor), `java.util.List`, `java.util.Set`, `java.util.SortedSet`, `java.util.NavigableSet`, `java.util.Map` (and concrete subtypes with a `public` no-arg constructor), `java.util.SortedMap`, `java.util.NavigableMap`, primitive arrays (e.g., `int[]`, `char[][]`, etc.), object arrays (e.g., `String[]`, `Integer[][]`, etc.). * `{NullAndEmptySource}`: a _composed annotation_ that combines the functionality of `@NullSource` and `@EmptySource`. If you need to supply multiple varying types of _blank_ strings to a parameterized test, you can achieve that using <> -- for example, `@ValueSource(strings = {"{nbsp}", "{nbsp}{nbsp}{nbsp}", "\t", "\n"})`. You can also combine `@NullSource`, `@EmptySource`, and `@ValueSource` to test a wider range of `null`, _empty_, and _blank_ input. The following example demonstrates how to achieve this for strings. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=NullAndEmptySource_example1] ---- Making use of the composed `@NullAndEmptySource` annotation simplifies the above as follows. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=NullAndEmptySource_example2] ---- NOTE: Both variants of the `nullEmptyAndBlankStrings(String)` parameterized test method result in six invocations: 1 for `null`, 1 for the empty string, and 4 for the explicit blank strings supplied via `@ValueSource`. [[writing-tests-parameterized-tests-sources-EnumSource]] ===== @EnumSource `@EnumSource` provides a convenient way to use `Enum` constants. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_example] ---- The annotation's `value` attribute is optional. When omitted, the declared type of the first method parameter is used. The test will fail if it does not reference an enum type. Thus, the `value` attribute is required in the above example because the method parameter is declared as `TemporalUnit`, i.e. the interface implemented by `ChronoUnit`, which isn't an enum type. Changing the method parameter type to `ChronoUnit` allows you to omit the explicit enum type from the annotation as follows. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_example_autodetection] ---- The annotation provides an optional `names` attribute that lets you specify which constants shall be used, like in the following example. If omitted, all constants will be used. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_include_example] ---- The `@EnumSource` annotation also provides an optional `mode` attribute that enables fine-grained control over which constants are passed to the test method. For example, you can exclude names from the enum constant pool or specify regular expressions as in the following examples. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_exclude_example] ---- [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=EnumSource_regex_example] ---- [[writing-tests-parameterized-tests-sources-MethodSource]] ===== @MethodSource `{MethodSource}` allows you to refer to one or more _factory_ methods of the test class or external classes. Factory methods within the test class must be `static` unless the test class is annotated with `@TestInstance(Lifecycle.PER_CLASS)`; whereas, factory methods in external classes must always be `static`. Each factory method must generate a _stream_ of _arguments_, and each set of arguments within the stream will be provided as the physical arguments for individual invocations of the annotated `@ParameterizedTest` method. Generally speaking this translates to a `Stream` of `Arguments` (i.e., `Stream`); however, the actual concrete return type can take on many forms. In this context, a "stream" is anything that JUnit can reliably convert into a `Stream`, such as `Stream`, `DoubleStream`, `LongStream`, `IntStream`, `Collection`, `Iterator`, `Iterable`, an array of objects, or an array of primitives. The "arguments" within the stream can be supplied as an instance of `Arguments`, an array of objects (e.g., `Object[]`), or a single value if the parameterized test method accepts a single argument. If you only need a single parameter, you can return a `Stream` of instances of the parameter type as demonstrated in the following example. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=simple_MethodSource_example] ---- If you do not explicitly provide a factory method name via `@MethodSource`, JUnit Jupiter will search for a _factory_ method that has the same name as the current `@ParameterizedTest` method by convention. This is demonstrated in the following example. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=simple_MethodSource_without_value_example] ---- Streams for primitive types (`DoubleStream`, `IntStream`, and `LongStream`) are also supported as demonstrated by the following example. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=primitive_MethodSource_example] ---- If a parameterized test method declares multiple parameters, you need to return a collection, stream, or array of `Arguments` instances or object arrays as shown below (see the Javadoc for `{MethodSource}` for further details on supported return types). Note that `arguments(Object...)` is a static factory method defined in the `Arguments` interface. In addition, `Arguments.of(Object...)` may be used as an alternative to `arguments(Object...)`. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=multi_arg_MethodSource_example] ---- An external, `static` _factory_ method can be referenced by providing its _fully qualified method name_ as demonstrated in the following example. [source,java,indent=0] ---- package example; include::{testDir}/example/ExternalMethodSourceDemo.java[tags=external_MethodSource_example] ---- Factory methods can declare parameters, which will be provided by registered implementations of the `ParameterResolver` extension API. In the following example, the factory method is referenced by its name since there is only one such method in the test class. If there are several local methods with the same name, parameters can also be provided to differentiate them – for example, `@MethodSource("factoryMethod()")` or `@MethodSource("factoryMethod(java.lang.String)")`. Alternatively, the factory method can be referenced by its fully qualified method name, e.g. `@MethodSource("example.MyTests#factoryMethod(java.lang.String)")`. [source,java,indent=0] ---- include::{testDir}/example/MethodSourceParameterResolutionDemo.java[tags=parameter_resolution_MethodSource_example] ---- [[writing-tests-parameterized-tests-sources-CsvSource]] ===== @CsvSource `@CsvSource` allows you to express argument lists as comma-separated values (i.e., CSV `String` literals). Each string provided via the `value` attribute in `@CsvSource` represents a CSV record and results in one invocation of the parameterized test. The first record may optionally be used to supply CSV headers (see the Javadoc for the `useHeadersInDisplayName` attribute for details and an example). [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=CsvSource_example] ---- The default delimiter is a comma (`,`), but you can use another character by setting the `delimiter` attribute. Alternatively, the `delimiterString` attribute allows you to use a `String` delimiter instead of a single character. However, both delimiter attributes cannot be set simultaneously. By default, `@CsvSource` uses a single quote (`'`) as its quote character, but this can be changed via the `quoteCharacter` attribute. See the `'lemon, lime'` value in the example above and in the table below. An empty, quoted value (`''`) results in an empty `String` unless the `emptyValue` attribute is set; whereas, an entirely _empty_ value is interpreted as a `null` reference. By specifying one or more `nullValues`, a custom value can be interpreted as a `null` reference (see the `NIL` example in the table below). An `ArgumentConversionException` is thrown if the target type of a `null` reference is a primitive type. NOTE: An _unquoted_ empty value will always be converted to a `null` reference regardless of any custom values configured via the `nullValues` attribute. Except within a quoted string, leading and trailing whitespace in a CSV column is trimmed by default. This behavior can be changed by setting the `ignoreLeadingAndTrailingWhitespace` attribute to `true`. [cols="50,50"] |=== | Example Input | Resulting Argument List | `@CsvSource({ "apple, banana" })` | `"apple"`, `"banana"` | `@CsvSource({ "apple, 'lemon, lime'" })` | `"apple"`, `"lemon, lime"` | `@CsvSource({ "apple, ''" })` | `"apple"`, `""` | `@CsvSource({ "apple, " })` | `"apple"`, `null` | `@CsvSource(value = { "apple, banana, NIL" }, nullValues = "NIL")` | `"apple"`, `"banana"`, `null` | `@CsvSource(value = { " apple , banana" }, ignoreLeadingAndTrailingWhitespace = false)` | `" apple "`, `" banana"` |=== If the programming language you are using supports _text blocks_ -- for example, Java SE 15 or higher -- you can alternatively use the `textBlock` attribute of `@CsvSource`. Each record within a text block represents a CSV record and results in one invocation of the parameterized test. The first record may optionally be used to supply CSV headers by setting the `useHeadersInDisplayName` attribute to `true` as in the example below. Using a text block, the previous example can be implemented as follows. [source,java,indent=0] ---- @ParameterizedTest(name = "[{index}] {arguments}") @CsvSource(useHeadersInDisplayName = true, textBlock = """ FRUIT, RANK apple, 1 banana, 2 'lemon, lime', 0xF1 strawberry, 700_000 """) void testWithCsvSource(String fruit, int rank) { // ... } ---- The generated display names for the previous example include the CSV header names. ---- [1] FRUIT = apple, RANK = 1 [2] FRUIT = banana, RANK = 2 [3] FRUIT = lemon, lime, RANK = 0xF1 [4] FRUIT = strawberry, RANK = 700_000 ---- In contrast to CSV records supplied via the `value` attribute, a text block can contain comments. Any line beginning with a `+++#+++` symbol will be treated as a comment and ignored. Note, however, that the `+++#+++` symbol must be the first character on the line without any leading whitespace. It is therefore recommended that the closing text block delimiter (`"""`) be placed either at the end of the last line of input or on the following line, left aligned with the rest of the input (as can be seen in the example below which demonstrates formatting similar to a table). [source,java,indent=0] ---- @ParameterizedTest @CsvSource(delimiter = '|', quoteCharacter = '"', textBlock = """ #----------------------------- # FRUIT | RANK #----------------------------- apple | 1 #----------------------------- banana | 2 #----------------------------- "lemon lime" | 0xF1 #----------------------------- strawberry | 700_000 #----------------------------- """) void testWithCsvSource(String fruit, int rank) { // ... } ---- [NOTE] ==== Java's https://docs.oracle.com/en/java/javase/15/text-blocks/index.html[text block] feature automatically removes _incidental whitespace_ when the code is compiled. However other JVM languages such as Groovy and Kotlin do not. Thus, if you are using a programming language other than Java and your text block contains comments or new lines within quoted strings, you will need to ensure that there is no leading whitespace within your text block. ==== [[writing-tests-parameterized-tests-sources-CsvFileSource]] ===== @CsvFileSource `@CsvFileSource` lets you use comma-separated value (CSV) files from the classpath or the local file system. Each record from a CSV file results in one invocation of the parameterized test. The first record may optionally be used to supply CSV headers. You can instruct JUnit to ignore the headers via the `numLinesToSkip` attribute. If you would like for the headers to be used in the display names, you can set the `useHeadersInDisplayName` attribute to `true`. The examples below demonstrate the use of `numLinesToSkip` and `useHeadersInDisplayName`. The default delimiter is a comma (`,`), but you can use another character by setting the `delimiter` attribute. Alternatively, the `delimiterString` attribute allows you to use a `String` delimiter instead of a single character. However, both delimiter attributes cannot be set simultaneously. .Comments in CSV files NOTE: Any line beginning with a `+++#+++` symbol will be interpreted as a comment and will be ignored. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=CsvFileSource_example] ---- [source,csv,indent=0] .two-column.csv ---- include::{testResourcesDir}/two-column.csv[] ---- The following listing shows the generated display names for the first two parameterized test methods above. ---- [1] country=Sweden, reference=1 [2] country=Poland, reference=2 [3] country=United States of America, reference=3 [4] country=France, reference=700_000 ---- The following listing shows the generated display names for the last parameterized test method above that uses CSV header names. ---- [1] COUNTRY = Sweden, REFERENCE = 1 [2] COUNTRY = Poland, REFERENCE = 2 [3] COUNTRY = United States of America, REFERENCE = 3 [4] COUNTRY = France, REFERENCE = 700_000 ---- In contrast to the default syntax used in `@CsvSource`, `@CsvFileSource` uses a double quote (`+++"+++`) as the quote character by default, but this can be changed via the `quoteCharacter` attribute. See the `"United States of America"` value in the example above. An empty, quoted value (`+++""+++`) results in an empty `String` unless the `emptyValue` attribute is set; whereas, an entirely _empty_ value is interpreted as a `null` reference. By specifying one or more `nullValues`, a custom value can be interpreted as a `null` reference. An `ArgumentConversionException` is thrown if the target type of a `null` reference is a primitive type. NOTE: An _unquoted_ empty value will always be converted to a `null` reference regardless of any custom values configured via the `nullValues` attribute. Except within a quoted string, leading and trailing whitespace in a CSV column is trimmed by default. This behavior can be changed by setting the `ignoreLeadingAndTrailingWhitespace` attribute to `true`. [[writing-tests-parameterized-tests-sources-ArgumentsSource]] ===== @ArgumentsSource `@ArgumentsSource` can be used to specify a custom, reusable `ArgumentsProvider`. Note that an implementation of `ArgumentsProvider` must be declared as either a top-level class or as a `static` nested class. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsSource_example] ---- [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsProvider_example] ---- If you wish to implement a custom `ArgumentsProvider` that also consumes an annotation (like built-in providers such as `{ValueArgumentsProvider}` or `{CsvArgumentsProvider}`), you have the possibility to extend the `{AnnotationBasedArgumentsProvider}` class. [[writing-tests-parameterized-tests-argument-conversion]] ==== Argument Conversion [[writing-tests-parameterized-tests-argument-conversion-widening]] ===== Widening Conversion JUnit Jupiter supports https://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#jls-5.1.2[Widening Primitive Conversion] for arguments supplied to a `@ParameterizedTest`. For example, a parameterized test annotated with `@ValueSource(ints = { 1, 2, 3 })` can be declared to accept not only an argument of type `int` but also an argument of type `long`, `float`, or `double`. [[writing-tests-parameterized-tests-argument-conversion-implicit]] ===== Implicit Conversion To support use cases like `@CsvSource`, JUnit Jupiter provides a number of built-in implicit type converters. The conversion process depends on the declared type of each method parameter. For example, if a `@ParameterizedTest` declares a parameter of type `TimeUnit` and the actual type supplied by the declared source is a `String`, the string will be automatically converted into the corresponding `TimeUnit` enum constant. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=implicit_conversion_example] ---- `String` instances are implicitly converted to the following target types. NOTE: Decimal, hexadecimal, and octal `String` literals will be converted to their integral types: `byte`, `short`, `int`, `long`, and their boxed counterparts. [[writing-tests-parameterized-tests-argument-conversion-implicit-table]] [cols="10,90"] |=== | Target Type | Example | `boolean`/`Boolean` | `"true"` -> `true` _(only accepts values 'true' or 'false', case-insensitive)_ | `byte`/`Byte` | `"15"`, `"0xF"`, or `"017"` -> `(byte) 15` | `char`/`Character` | `"o"` -> `'o'` | `short`/`Short` | `"15"`, `"0xF"`, or `"017"` -> `(short) 15` | `int`/`Integer` | `"15"`, `"0xF"`, or `"017"` -> `15` | `long`/`Long` | `"15"`, `"0xF"`, or `"017"` -> `15L` | `float`/`Float` | `"1.0"` -> `1.0f` | `double`/`Double` | `"1.0"` -> `1.0d` | `Enum` subclass | `"SECONDS"` -> `TimeUnit.SECONDS` | `java.io.File` | `"/path/to/file"` -> `new File("/path/to/file")` | `java.lang.Class` | `"java.lang.Integer"` -> `java.lang.Integer.class` _(use `$` for nested classes, e.g. `"java.lang.Thread$State"`)_ | `java.lang.Class` | `"byte"` -> `byte.class` _(primitive types are supported)_ | `java.lang.Class` | `"char[]"` -> `char[].class` _(array types are supported)_ | `java.math.BigDecimal` | `"123.456e789"` -> `new BigDecimal("123.456e789")` | `java.math.BigInteger` | `"1234567890123456789"` -> `new BigInteger("1234567890123456789")` | `java.net.URI` | `"https://junit.org/"` -> `URI.create("https://junit.org/")` | `java.net.URL` | `"https://junit.org/"` -> `URI.create("https://junit.org/").toURL()` | `java.nio.charset.Charset` | `"UTF-8"` -> `Charset.forName("UTF-8")` | `java.nio.file.Path` | `"/path/to/file"` -> `Paths.get("/path/to/file")` | `java.time.Duration` | `"PT3S"` -> `Duration.ofSeconds(3)` | `java.time.Instant` | `"1970-01-01T00:00:00Z"` -> `Instant.ofEpochMilli(0)` | `java.time.LocalDateTime` | `"2017-03-14T12:34:56.789"` -> `LocalDateTime.of(2017, 3, 14, 12, 34, 56, 789_000_000)` | `java.time.LocalDate` | `"2017-03-14"` -> `LocalDate.of(2017, 3, 14)` | `java.time.LocalTime` | `"12:34:56.789"` -> `LocalTime.of(12, 34, 56, 789_000_000)` | `java.time.MonthDay` | `"--03-14"` -> `MonthDay.of(3, 14)` | `java.time.OffsetDateTime` | `"2017-03-14T12:34:56.789Z"` -> `OffsetDateTime.of(2017, 3, 14, 12, 34, 56, 789_000_000, ZoneOffset.UTC)` | `java.time.OffsetTime` | `"12:34:56.789Z"` -> `OffsetTime.of(12, 34, 56, 789_000_000, ZoneOffset.UTC)` | `java.time.Period` | `"P2M6D"` -> `Period.of(0, 2, 6)` | `java.time.YearMonth` | `"2017-03"` -> `YearMonth.of(2017, 3)` | `java.time.Year` | `"2017"` -> `Year.of(2017)` | `java.time.ZonedDateTime` | `"2017-03-14T12:34:56.789Z"` -> `ZonedDateTime.of(2017, 3, 14, 12, 34, 56, 789_000_000, ZoneOffset.UTC)` | `java.time.ZoneId` | `"Europe/Berlin"` -> `ZoneId.of("Europe/Berlin")` | `java.time.ZoneOffset` | `"+02:30"` -> `ZoneOffset.ofHoursMinutes(2, 30)` | `java.util.Currency` | `"JPY"` -> `Currency.getInstance("JPY")` | `java.util.Locale` | `"en"` -> `new Locale("en")` | `java.util.UUID` | `"d043e930-7b3b-48e3-bdbe-5a3ccfb833db"` -> `UUID.fromString("d043e930-7b3b-48e3-bdbe-5a3ccfb833db")` |=== [[writing-tests-parameterized-tests-argument-conversion-implicit-fallback]] ====== Fallback String-to-Object Conversion In addition to implicit conversion from strings to the target types listed in the above table, JUnit Jupiter also provides a fallback mechanism for automatic conversion from a `String` to a given target type if the target type declares exactly one suitable _factory method_ or a _factory constructor_ as defined below. - __factory method__: a non-private, `static` method declared in the target type that accepts a single `String` argument and returns an instance of the target type. The name of the method can be arbitrary and need not follow any particular convention. - __factory constructor__: a non-private constructor in the target type that accepts a single `String` argument. Note that the target type must be declared as either a top-level class or as a `static` nested class. NOTE: If multiple _factory methods_ are discovered, they will be ignored. If a _factory method_ and a _factory constructor_ are discovered, the factory method will be used instead of the constructor. For example, in the following `@ParameterizedTest` method, the `Book` argument will be created by invoking the `Book.fromTitle(String)` factory method and passing `"42 Cats"` as the title of the book. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=implicit_fallback_conversion_example] ---- [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=implicit_fallback_conversion_example_Book] ---- [[writing-tests-parameterized-tests-argument-conversion-explicit]] ===== Explicit Conversion Instead of relying on implicit argument conversion you may explicitly specify an `ArgumentConverter` to use for a certain parameter using the `@ConvertWith` annotation like in the following example. Note that an implementation of `ArgumentConverter` must be declared as either a top-level class or as a `static` nested class. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_conversion_example] ---- [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_conversion_example_ToStringArgumentConverter] ---- If the converter is only meant to convert one type to another, you can extend `TypedArgumentConverter` to avoid boilerplate type checks. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_conversion_example_TypedArgumentConverter] ---- Explicit argument converters are meant to be implemented by test and extension authors. Thus, `junit-jupiter-params` only provides a single explicit argument converter that may also serve as a reference implementation: `JavaTimeArgumentConverter`. It is used via the composed annotation `JavaTimeConversionPattern`. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=explicit_java_time_converter] ---- If you wish to implement a custom `ArgumentConverter` that also consumes an annotation (like `JavaTimeArgumentConverter`), you have the possibility to extend the `{AnnotationBasedArgumentConverter}` class. [[writing-tests-parameterized-tests-argument-aggregation]] ==== Argument Aggregation By default, each _argument_ provided to a `@ParameterizedTest` method corresponds to a single method parameter. Consequently, argument sources which are expected to supply a large number of arguments can lead to large method signatures. In such cases, an `{ArgumentsAccessor}` can be used instead of multiple parameters. Using this API, you can access the provided arguments through a single argument passed to your test method. In addition, type conversion is supported as discussed in <>. Besides, you can retrieve the current test invocation index with `ArgumentsAccessor.getInvocationIndex()`. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAccessor_example] ---- _An instance of `ArgumentsAccessor` is automatically injected into any parameter of type `ArgumentsAccessor`._ [[writing-tests-parameterized-tests-argument-aggregation-custom]] ===== Custom Aggregators Apart from direct access to a `@ParameterizedTest` method's arguments using an `ArgumentsAccessor`, JUnit Jupiter also supports the usage of custom, reusable _aggregators_. To use a custom aggregator, implement the `{ArgumentsAggregator}` interface and register it via the `@AggregateWith` annotation on a compatible parameter in the `@ParameterizedTest` method. The result of the aggregation will then be provided as an argument for the corresponding parameter when the parameterized test is invoked. Note that an implementation of `ArgumentsAggregator` must be declared as either a top-level class or as a `static` nested class. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_example] ---- [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_example_PersonAggregator] ---- If you find yourself repeatedly declaring `@AggregateWith(MyTypeAggregator.class)` for multiple parameterized test methods across your codebase, you may wish to create a custom _composed annotation_ such as `@CsvToMyType` that is meta-annotated with `@AggregateWith(MyTypeAggregator.class)`. The following example demonstrates this in action with a custom `@CsvToPerson` annotation. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_with_custom_annotation_example] ---- [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ArgumentsAggregator_with_custom_annotation_example_CsvToPerson] ---- [[writing-tests-parameterized-tests-display-names]] ==== Customizing Display Names By default, the display name of a parameterized test invocation contains the invocation index and the `String` representation of all arguments for that specific invocation. Each of them is preceded by the parameter name (unless the argument is only available via an `ArgumentsAccessor` or `ArgumentAggregator`), if present in the bytecode (for Java, test code must be compiled with the `-parameters` compiler flag). However, you can customize invocation display names via the `name` attribute of the `@ParameterizedTest` annotation like in the following example. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=custom_display_names] ---- When executing the above method using the `ConsoleLauncher` you will see output similar to the following. .... Display name of container ✔ ├─ 1 ==> the rank of 'apple' is 1 ✔ ├─ 2 ==> the rank of 'banana' is 2 ✔ └─ 3 ==> the rank of 'lemon, lime' is 3 ✔ .... Please note that `name` is a `MessageFormat` pattern. Thus, a single quote (`'`) needs to be represented as a doubled single quote (`''`) in order to be displayed. The following placeholders are supported within custom display names. [cols="20,80"] |=== | Placeholder | Description | `{displayName}` | the display name of the method | `{index}` | the current invocation index (1-based) | `{arguments}` | the complete, comma-separated arguments list | `{argumentsWithNames}` | the complete, comma-separated arguments list with parameter names | `{0}`, `{1}`, ... | an individual argument |=== NOTE: When including arguments in display names, their string representations are truncated if they exceed the configured maximum length. The limit is configurable via the `junit.jupiter.params.displayname.argument.maxlength` configuration parameter and defaults to 512 characters. When using `@MethodSource` or `@ArgumentsSource`, you can provide custom names for arguments using the `{Named}` API. A custom name will be used if the argument is included in the invocation display name, like in the example below. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=named_arguments] ---- .... A parameterized test with named arguments ✔ ├─ 1: An important file ✔ └─ 2: Another file ✔ .... If you'd like to set a default name pattern for all parameterized tests in your project, you can declare the `junit.jupiter.params.displayname.default` configuration parameter in the `junit-platform.properties` file as demonstrated in the following example (see <> for other options). [source,properties,indent=0] ---- junit.jupiter.params.displayname.default = {index} ---- The display name for a parameterized test is determined according to the following precedence rules: 1. `name` attribute in `@ParameterizedTest`, if present 2. value of the `junit.jupiter.params.displayname.default` configuration parameter, if present 3. `DEFAULT_DISPLAY_NAME` constant defined in `@ParameterizedTest` [[writing-tests-parameterized-tests-lifecycle-interop]] ==== Lifecycle and Interoperability Each invocation of a parameterized test has the same lifecycle as a regular `@Test` method. For example, `@BeforeEach` methods will be executed before each invocation. Similar to <>, invocations will appear one by one in the test tree of an IDE. You may at will mix regular `@Test` methods and `@ParameterizedTest` methods within the same test class. You may use `ParameterResolver` extensions with `@ParameterizedTest` methods. However, method parameters that are resolved by argument sources need to come first in the argument list. Since a test class may contain regular tests as well as parameterized tests with different parameter lists, values from argument sources are not resolved for lifecycle methods (e.g. `@BeforeEach`) and test class constructors. [source,java,indent=0] ---- include::{testDir}/example/ParameterizedTestDemo.java[tags=ParameterResolver_example] ---- [[writing-tests-test-templates]] === Test Templates A `{TestTemplate}` method is not a regular test case but rather a template for test cases. As such, it is designed to be invoked multiple times depending on the number of invocation contexts returned by the registered providers. Thus, it must be used in conjunction with a registered `{TestTemplateInvocationContextProvider}` extension. Each invocation of a test template method behaves like the execution of a regular `@Test` method with full support for the same lifecycle callbacks and extensions. Please refer to <> for usage examples. NOTE: <> and <> are built-in specializations of test templates. [[writing-tests-dynamic-tests]] === Dynamic Tests The standard `@Test` annotation in JUnit Jupiter described in <> is very similar to the `@Test` annotation in JUnit 4. Both describe methods that implement test cases. These test cases are static in the sense that they are fully specified at compile time, and their behavior cannot be changed by anything happening at runtime. _Assumptions provide a basic form of dynamic behavior but are intentionally rather limited in their expressiveness._ In addition to these standard tests a completely new kind of test programming model has been introduced in JUnit Jupiter. This new kind of test is a _dynamic test_ which is generated at runtime by a factory method that is annotated with `@TestFactory`. In contrast to `@Test` methods, a `@TestFactory` method is not itself a test case but rather a factory for test cases. Thus, a dynamic test is the product of a factory. Technically speaking, a `@TestFactory` method must return a single `DynamicNode` or a `Stream`, `Collection`, `Iterable`, `Iterator`, or array of `DynamicNode` instances. Instantiable subclasses of `DynamicNode` are `DynamicContainer` and `DynamicTest`. `DynamicContainer` instances are composed of a _display name_ and a list of dynamic child nodes, enabling the creation of arbitrarily nested hierarchies of dynamic nodes. `DynamicTest` instances will be executed lazily, enabling dynamic and even non-deterministic generation of test cases. Any `Stream` returned by a `@TestFactory` will be properly closed by calling `stream.close()`, making it safe to use a resource such as `Files.lines()`. As with `@Test` methods, `@TestFactory` methods must not be `private` or `static` and may optionally declare parameters to be resolved by `ParameterResolvers`. A `DynamicTest` is a test case generated at runtime. It is composed of a _display name_ and an `Executable`. `Executable` is a `@FunctionalInterface` which means that the implementations of dynamic tests can be provided as _lambda expressions_ or _method references_. .Dynamic Test Lifecycle WARNING: The execution lifecycle of a dynamic test is quite different than it is for a standard `@Test` case. Specifically, there are no lifecycle callbacks for individual dynamic tests. This means that `@BeforeEach` and `@AfterEach` methods and their corresponding extension callbacks are executed for the `@TestFactory` method but not for each _dynamic test_. In other words, if you access fields from the test instance within a lambda expression for a dynamic test, those fields will not be reset by callback methods or extensions between the execution of individual dynamic tests generated by the same `@TestFactory` method. As of JUnit Jupiter {jupiter-version}, dynamic tests must always be created by factory methods; however, this might be complemented by a registration facility in a later release. [[writing-tests-dynamic-tests-examples]] ==== Dynamic Test Examples The following `DynamicTestsDemo` class demonstrates several examples of test factories and dynamic tests. The first method returns an invalid return type. Since an invalid return type cannot be detected at compile time, a `JUnitException` is thrown when it is detected at runtime. The next six methods demonstrate the generation of a `Collection`, `Iterable`, `Iterator`, array, or `Stream` of `DynamicTest` instances. Most of these examples do not really exhibit dynamic behavior but merely demonstrate the supported return types in principle. However, `dynamicTestsFromStream()` and `dynamicTestsFromIntStream()` demonstrate how to generate dynamic tests for a given set of strings or a range of input numbers. The next method is truly dynamic in nature. `generateRandomNumberOfTests()` implements an `Iterator` that generates random numbers, a display name generator, and a test executor and then provides all three to `DynamicTest.stream()`. Although the non-deterministic behavior of `generateRandomNumberOfTests()` is of course in conflict with test repeatability and should thus be used with care, it serves to demonstrate the expressiveness and power of dynamic tests. The next method is similar to `generateRandomNumberOfTests()` in terms of flexibility; however, `dynamicTestsFromStreamFactoryMethod()` generates a stream of dynamic tests from an existing `Stream` via the `DynamicTest.stream()` factory method. For demonstration purposes, the `dynamicNodeSingleTest()` method generates a single `DynamicTest` instead of a stream, and the `dynamicNodeSingleContainer()` method generates a nested hierarchy of dynamic tests utilizing `DynamicContainer`. [source,java] ---- include::{testDir}/example/DynamicTestsDemo.java[tags=user_guide] ---- [[writing-tests-dynamic-tests-uri-test-source]] ==== URI Test Sources for Dynamic Tests The JUnit Platform provides `TestSource`, a representation of the source of a test or container used to navigate to its location by IDEs and build tools. The `TestSource` for a dynamic test or dynamic container can be constructed from a `java.net.URI` which can be supplied via the `DynamicTest.dynamicTest(String, URI, Executable)` or `DynamicContainer.dynamicContainer(String, URI, Stream)` factory method, respectively. The `URI` will be converted to one of the following `TestSource` implementations. `ClasspathResourceSource` :: If the `URI` contains the `classpath` scheme -- for example, `classpath:/test/foo.xml?line=20,column=2`. `DirectorySource` :: If the `URI` represents a directory present in the file system. `FileSource` :: If the `URI` represents a file present in the file system. `MethodSource` :: If the `URI` contains the `method` scheme and the fully qualified method name (FQMN) -- for example, `method:org.junit.Foo#bar(java.lang.String, java.lang.String[])`. Please refer to the Javadoc for `DiscoverySelectors.selectMethod(String)` for the supported formats for a FQMN. `ClassSource` :: If the `URI` contains the `class` scheme and the fully qualified class name -- for example, `class:org.junit.Foo?line=42`. `UriSource` :: If none of the above `TestSource` implementations are applicable. [[writing-tests-declarative-timeouts]] === Timeouts The `@Timeout` annotation allows one to declare that a test, test factory, test template, or lifecycle method should fail if its execution time exceeds a given duration. The time unit for the duration defaults to seconds but is configurable. The following example shows how `@Timeout` is applied to lifecycle and test methods. [source,java] ---- include::{testDir}/example/TimeoutDemo.java[tags=user_guide] ---- To apply the same timeout to all test methods within a test class and all of its `@Nested` classes, you can declare the `@Timeout` annotation at the class level. It will then be applied to all test, test factory, and test template methods within that class and its `@Nested` classes unless overridden by a `@Timeout` annotation on a specific method or `@Nested` class. Please note that `@Timeout` annotations declared at the class level are not applied to lifecycle methods. Declaring `@Timeout` on a `@TestFactory` method checks that the factory method returns within the specified duration but does not verify the execution time of each individual `DynamicTest` generated by the factory. Please use `assertTimeout()` or `assertTimeoutPreemptively()` for that purpose. If `@Timeout` is present on a `@TestTemplate` method — for example, a `@RepeatedTest` or `@ParameterizedTest` — each invocation will have the given timeout applied to it. [[writing-tests-declarative-timeouts-thread-mode]] ==== Thread mode The timeout can be applied using one of the following three thread modes: `SAME_THREAD`, `SEPARATE_THREAD`, or `INFERRED`. When `SAME_THREAD` is used, the execution of the annotated method proceeds in the main thread of the test. If the timeout is exceeded, the main thread is interrupted from another thread. This is done to ensure interoperability with frameworks such as Spring that make use of mechanisms that are sensitive to the currently running thread — for example, `ThreadLocal` transaction management. On the contrary when `SEPARATE_THREAD` is used, like the `assertTimeoutPreemptively()` assertion, the execution of the annotated method proceeds in a separate thread, this can lead to undesirable side effects, see <>. When `INFERRED` (default) thread mode is used, the thread mode is resolved via the `junit.jupiter.execution.timeout.thread.mode.default` configuration parameter. If the provided configuration parameter is invalid or not present then `SAME_THREAD` is used as fallback. [[writing-tests-declarative-timeouts-default-timeouts]] ==== Default Timeouts The following <> can be used to specify default timeouts for all methods of a certain category unless they or an enclosing test class is annotated with `@Timeout`: `junit.jupiter.execution.timeout.default`:: Default timeout for all testable and lifecycle methods `junit.jupiter.execution.timeout.testable.method.default`:: Default timeout for all testable methods `junit.jupiter.execution.timeout.test.method.default`:: Default timeout for `@Test` methods `junit.jupiter.execution.timeout.testtemplate.method.default`:: Default timeout for `@TestTemplate` methods `junit.jupiter.execution.timeout.testfactory.method.default`:: Default timeout for `@TestFactory` methods `junit.jupiter.execution.timeout.lifecycle.method.default`:: Default timeout for all lifecycle methods `junit.jupiter.execution.timeout.beforeall.method.default`:: Default timeout for `@BeforeAll` methods `junit.jupiter.execution.timeout.beforeeach.method.default`:: Default timeout for `@BeforeEach` methods `junit.jupiter.execution.timeout.aftereach.method.default`:: Default timeout for `@AfterEach` methods `junit.jupiter.execution.timeout.afterall.method.default`:: Default timeout for `@AfterAll` methods More specific configuration parameters override less specific ones. For example, `junit.jupiter.execution.timeout.test.method.default` overrides `junit.jupiter.execution.timeout.testable.method.default` which overrides `junit.jupiter.execution.timeout.default`. The values of such configuration parameters must be in the following, case-insensitive format: ` [ns|μs|ms|s|m|h|d]`. The space between the number and the unit may be omitted. Specifying no unit is equivalent to using seconds. .Example timeout configuration parameter values [cols="20,80"] |=== | Parameter value | Equivalent annotation | `42` | `@Timeout(42)` | `42 ns` | `@Timeout(value = 42, unit = NANOSECONDS)` | `42 μs` | `@Timeout(value = 42, unit = MICROSECONDS)` | `42 ms` | `@Timeout(value = 42, unit = MILLISECONDS)` | `42 s` | `@Timeout(value = 42, unit = SECONDS)` | `42 m` | `@Timeout(value = 42, unit = MINUTES)` | `42 h` | `@Timeout(value = 42, unit = HOURS)` | `42 d` | `@Timeout(value = 42, unit = DAYS)` |=== [[writing-tests-declarative-timeouts-polling]] ==== Using @Timeout for Polling Tests When dealing with asynchronous code, it is common to write tests that poll while waiting for something to happen before performing any assertions. In some cases you can rewrite the logic to use a `CountDownLatch` or another synchronization mechanism, but sometimes that is not possible — for example, if the subject under test sends a message to a channel in an external message broker and assertions cannot be performed until the message has been successfully sent through the channel. Asynchronous tests like these require some form of timeout to ensure they don't hang the test suite by executing indefinitely, as would be the case if an asynchronous message never gets successfully delivered. By configuring a timeout for an asynchronous test that polls, you can ensure that the test does not execute indefinitely. The following example demonstrates how to achieve this with JUnit Jupiter's `@Timeout` annotation. This technique can be used to implement "poll until" logic very easily. [source,java] ---- include::{testDir}/example/PollingTimeoutDemo.java[tags=user_guide,indent=0] ---- NOTE: If you need more control over polling intervals and greater flexibility with asynchronous tests, consider using a dedicated library such as link:https://github.com/awaitility/awaitility[Awaitility]. [[writing-tests-declarative-timeouts-mode]] ==== Disable @Timeout Globally When stepping through your code in a debug session, a fixed timeout limit may influence the result of the test, e.g. mark the test as failed although all assertions were met. JUnit Jupiter supports the `junit.jupiter.execution.timeout.mode` configuration parameter to configure when timeouts are applied. There are three modes: `enabled`, `disabled`, and `disabled_on_debug`. The default mode is `enabled`. A VM runtime is considered to run in debug mode when one of its input parameters starts with `-agentlib:jdwp` or `-Xrunjdwp`. This heuristic is queried by the `disabled_on_debug` mode. [[writing-tests-parallel-execution]] === Parallel Execution By default, JUnit Jupiter tests are run sequentially in a single thread. Running tests in parallel -- for example, to speed up execution -- is available as an opt-in feature since version 5.3. To enable parallel execution, set the `junit.jupiter.execution.parallel.enabled` configuration parameter to `true` -- for example, in `junit-platform.properties` (see <> for other options). Please note that enabling this property is only the first step required to execute tests in parallel. If enabled, test classes and methods will still be executed sequentially by default. Whether or not a node in the test tree is executed concurrently is controlled by its execution mode. The following two modes are available. `SAME_THREAD`:: Force execution in the same thread used by the parent. For example, when used on a test method, the test method will be executed in the same thread as any `@BeforeAll` or `@AfterAll` methods of the containing test class. `CONCURRENT`:: Execute concurrently unless a resource lock forces execution in the same thread. By default, nodes in the test tree use the `SAME_THREAD` execution mode. You can change the default by setting the `junit.jupiter.execution.parallel.mode.default` configuration parameter. Alternatively, you can use the `{Execution}` annotation to change the execution mode for the annotated element and its subelements (if any) which allows you to activate parallel execution for individual test classes, one by one. [source,properties] .Configuration parameters to execute all tests in parallel ---- junit.jupiter.execution.parallel.enabled = true junit.jupiter.execution.parallel.mode.default = concurrent ---- The default execution mode is applied to all nodes of the test tree with a few notable exceptions, namely test classes that use the `Lifecycle.PER_CLASS` mode or a `{MethodOrderer}` (except for `{MethodOrderer_Random}`). In the former case, test authors have to ensure that the test class is thread-safe; in the latter, concurrent execution might conflict with the configured execution order. Thus, in both cases, test methods in such test classes are only executed concurrently if the `@Execution(CONCURRENT)` annotation is present on the test class or method. When parallel execution is enabled and a default `{ClassOrderer}` is registered (see <> for details), top-level test classes will initially be sorted accordingly and scheduled in that order. However, they are not guaranteed to be started in exactly that order since the threads they are executed on are not controlled directly by JUnit. All nodes of the test tree that are configured with the `CONCURRENT` execution mode will be executed fully in parallel according to the provided <> while observing the declarative <> mechanism. Please note that <> needs to be enabled separately. In addition, you can configure the default execution mode for top-level classes by setting the `junit.jupiter.execution.parallel.mode.classes.default` configuration parameter. By combining both configuration parameters, you can configure classes to run in parallel but their methods in the same thread: [source,properties] .Configuration parameters to execute top-level classes in parallel but methods in same thread ---- junit.jupiter.execution.parallel.enabled = true junit.jupiter.execution.parallel.mode.default = same_thread junit.jupiter.execution.parallel.mode.classes.default = concurrent ---- The opposite combination will run all methods within one class in parallel, but top-level classes will run sequentially: [source,properties] .Configuration parameters to execute top-level classes sequentially but their methods in parallel ---- junit.jupiter.execution.parallel.enabled = true junit.jupiter.execution.parallel.mode.default = concurrent junit.jupiter.execution.parallel.mode.classes.default = same_thread ---- The following diagram illustrates how the execution of two top-level test classes `A` and `B` with two test methods per class behaves for all four combinations of `junit.jupiter.execution.parallel.mode.default` and `junit.jupiter.execution.parallel.mode.classes.default` (see labels in first column). //// Source: https://mermaidjs.github.io/mermaid-live-editor/#/view/eyJjb2RlIjoiZ2FudHRcbiAgICBkYXRlRm9ybWF0ICBZWVlZLU1NLUREXG5cbiAgICBzZWN0aW9uIChzYW1lX3RocmVhZCwgc2FtZV90aHJlYWQpXG4gICAgQS50ZXN0MSgpIDphc3MxLCAyMDE5LTAxLTAxLCAxZFxuICAgIEEudGVzdDIoKSA6YXNzMiwgYWZ0ZXIgYXNzMSwgMWRcbiAgICBCLnRlc3QxKCkgOmJzczEsIGFmdGVyIGFzczIsIDFkXG4gICAgQi50ZXN0MigpIDpic3MyLCBhZnRlciBic3MxLCAxZFxuXG4gICAgc2VjdGlvbiAoc2FtZV90aHJlYWQsIGNvbmN1cnJlbnQpXG4gICAgQS50ZXN0MSgpIDphc2MxLCAyMDE5LTAxLTAxLCAxZFxuICAgIEEudGVzdDIoKSA6YXNjMiwgYWZ0ZXIgYXNjMSwgMWRcbiAgICBCLnRlc3QxKCkgOmJzYzEsIDIwMTktMDEtMDEsIDFkXG4gICAgQi50ZXN0MigpIDpic2MyLCBhZnRlciBic2MxLCAxZFxuXG4gICAgc2VjdGlvbiAoY29uY3VycmVudCwgc2FtZV90aHJlYWQpXG4gICAgQS50ZXN0MSgpIDphY3MxLCAyMDE5LTAxLTAxLCAxZFxuICAgIEEudGVzdDIoKSA6YWNzMiwgMjAxOS0wMS0wMSwgMWRcbiAgICBCLnRlc3QxKCkgOmJjczEsIGFmdGVyIGFjczIsIDFkXG4gICAgQi50ZXN0MigpIDpiY3MyLCBhZnRlciBhY3MyLCAxZFxuXG4gICAgc2VjdGlvbiAoY29uY3VycmVudCwgY29uY3VycmVudClcbiAgICBBLnRlc3QxKCkgOmFjYzEsIDIwMTktMDEtMDEsIDFkXG4gICAgQS50ZXN0MigpIDphY2MyLCAyMDE5LTAxLTAxLCAxZFxuICAgIEIudGVzdDEoKSA6YmNjMSwgMjAxOS0wMS0wMSwgMWRcbiAgICBCLnRlc3QyKCkgOmJjYzIsIDIwMTktMDEtMDEsIDFkXG4iLCJtZXJtYWlkIjp7InRoZW1lIjoibmV1dHJhbCIsImdhbnR0Ijp7ImxlZnRQYWRkaW5nIjoyMjUsImJhckdhcCI6NSwiZ3JpZExpbmVTdGFydFBhZGRpbmciOjEwLCJiYXJIZWlnaHQiOjMwLCJmb250U2l6ZSI6MTV9LCJ0aGVtZUNTUyI6Ii50YXNrVGV4dCwgLnNlY3Rpb25UaXRsZSB7IGZvbnQtZmFtaWx5OiAnT3BlbiBTYW5zJzsgZm9udC1zaXplOjE1cHggfSAuZ3JpZCAudGljayB0ZXh0IHsgZGlzcGxheTpub25lIH0gLmdyaWQgLnRpY2s6bnRoLWNoaWxkKDJuKzEpIHsgZGlzcGxheTpub25lIH0ifX0 gantt dateFormat YYYY-MM-DD section (same_thread, same_thread) A.test1() :ass1, 2019-01-01, 1d A.test2() :ass2, after ass1, 1d B.test1() :bss1, after ass2, 1d B.test2() :bss2, after bss1, 1d section (same_thread, concurrent) A.test1() :asc1, 2019-01-01, 1d A.test2() :asc2, after asc1, 1d B.test1() :bsc1, 2019-01-01, 1d B.test2() :bsc2, after bsc1, 1d section (concurrent, same_thread) A.test1() :acs1, 2019-01-01, 1d A.test2() :acs2, 2019-01-01, 1d B.test1() :bcs1, after acs2, 1d B.test2() :bcs2, after acs2, 1d section (concurrent, concurrent) A.test1() :acc1, 2019-01-01, 1d A.test2() :acc2, 2019-01-01, 1d B.test1() :bcc1, 2019-01-01, 1d B.test2() :bcc2, 2019-01-01, 1d //// image::writing-tests_execution_mode.svg[caption='',title='Default execution mode configuration combinations'] If the `junit.jupiter.execution.parallel.mode.classes.default` configuration parameter is not explicitly set, the value for `junit.jupiter.execution.parallel.mode.default` will be used instead. [[writing-tests-parallel-execution-config]] ==== Configuration Properties such as the desired parallelism and the maximum pool size can be configured using a `{ParallelExecutionConfigurationStrategy}`. The JUnit Platform provides two implementations out of the box: `dynamic` and `fixed`. Alternatively, you may implement a `custom` strategy. To select a strategy, set the `junit.jupiter.execution.parallel.config.strategy` configuration parameter to one of the following options. `dynamic`:: Computes the desired parallelism based on the number of available processors/cores multiplied by the `junit.jupiter.execution.parallel.config.dynamic.factor` configuration parameter (defaults to `1`). The optional `junit.jupiter.execution.parallel.config.dynamic.max-pool-size-factor` configuration parameter can be used to limit the maximum number of threads. `fixed`:: Uses the mandatory `junit.jupiter.execution.parallel.config.fixed.parallelism` configuration parameter as the desired parallelism. The optional `junit.jupiter.execution.parallel.config.fixed.max-pool-size` configuration parameter can be used to limit the maximum number of threads. `custom`:: Allows you to specify a custom `{ParallelExecutionConfigurationStrategy}` implementation via the mandatory `junit.jupiter.execution.parallel.config.custom.class` configuration parameter to determine the desired configuration. If no configuration strategy is set, JUnit Jupiter uses the `dynamic` configuration strategy with a factor of `1`. Consequently, the desired parallelism will be equal to the number of available processors/cores. .Parallelism alone does not imply maximum number of concurrent threads NOTE: By default JUnit Jupiter does not guarantee that the number of concurrently executing tests will not exceed the configured parallelism. For example, when using one of the synchronization mechanisms described in the next section, the `ForkJoinPool` that is used behind the scenes may spawn additional threads to ensure execution continues with sufficient parallelism. If you require such guarantees, with Java 9+, it is possible to limit the maximum number of concurrent threads by controlling the maximum pool size of the `dynamic`, `fixed` and `custom` strategies. [[writing-tests-parallel-execution-config-properties]] ===== Relevant properties The following table lists relevant properties for configuring parallel execution. See <> for details on how to set such properties. [cols="d,d,a,d"] |=== |Property |Description |Supported Values |Default Value | ```junit.jupiter.execution.parallel.enabled``` | Enable parallel test execution | * `true` * `false` | ```false``` | ```junit.jupiter.execution.parallel.mode.default``` | Default execution mode of nodes in the test tree | * `concurrent` * `same_thread` | ```same_thread``` | ```junit.jupiter.execution.parallel.mode.classes.default``` | Default execution mode of top-level classes | * `concurrent` * `same_thread` | ```same_thread``` | ```junit.jupiter.execution.parallel.config.strategy``` | Execution strategy for desired parallelism and maximum pool size | * `dynamic` * `fixed` * `custom` | ```dynamic``` | ```junit.jupiter.execution.parallel.config.dynamic.factor``` | Factor to be multiplied by the number of available processors/cores to determine the desired parallelism for the ```dynamic``` configuration strategy | a positive decimal number | ```1.0``` | ```junit.jupiter.execution.parallel.config.dynamic.max-pool-size-factor``` | Factor to be multiplied by the number of available processors/cores and the value of `junit.jupiter.execution.parallel.config.dynamic.factor` to determine the desired parallelism for the ```dynamic``` configuration strategy | a positive decimal number, must be greater than or equal to `1.0` | 256 + the value of `junit.jupiter.execution.parallel.config.dynamic.factor` multiplied by the number of available processors/cores | ```junit.jupiter.execution.parallel.config.dynamic.saturate``` | Disable saturation of the underlying fork-join pool for the ```dynamic``` configuration strategy | * `true` * `false` | ```true``` | ```junit.jupiter.execution.parallel.config.fixed.parallelism``` | Desired parallelism for the ```fixed``` configuration strategy | a positive integer | no default value | ```junit.jupiter.execution.parallel.config.fixed.max-pool-size``` | Desired maximum pool size of the underlying fork-join pool for the ```fixed``` configuration strategy | a positive integer, must be greater than or equal to `junit.jupiter.execution.parallel.config.fixed.parallelism` | 256 + the value of `junit.jupiter.execution.parallel.config.fixed.parallelism` | ```junit.jupiter.execution.parallel.config.fixed.saturate``` | Disable saturation of the underlying fork-join pool for the ```fixed``` configuration strategy | * `true` * `false` | ```true``` | ```junit.jupiter.execution.parallel.config.custom.class``` | Fully qualified class name of the _ParallelExecutionConfigurationStrategy_ to be used for the ```custom``` configuration strategy | for example, _org.example.CustomStrategy_ | no default value |=== [[writing-tests-parallel-execution-synchronization]] ==== Synchronization In addition to controlling the execution mode using the `{Execution}` annotation, JUnit Jupiter provides another annotation-based declarative synchronization mechanism. The `{ResourceLock}` annotation allows you to declare that a test class or method uses a specific shared resource that requires synchronized access to ensure reliable test execution. The shared resource is identified by a unique name which is a `String`. The name can be user-defined or one of the predefined constants in `{Resources}`: `SYSTEM_PROPERTIES`, `SYSTEM_OUT`, `SYSTEM_ERR`, `LOCALE`, or `TIME_ZONE`. If the tests in the following example were run in parallel _without_ the use of {ResourceLock}, they would be _flaky_. Sometimes they would pass, and at other times they would fail due to the inherent race condition of writing and then reading the same JVM System Property. When access to shared resources is declared using the `{ResourceLock}` annotation, the JUnit Jupiter engine uses this information to ensure that no conflicting tests are run in parallel. [NOTE] .Running tests in isolation ==== If most of your test classes can be run in parallel without any synchronization but you have some test classes that need to run in isolation, you can mark the latter with the `{Isolated}` annotation. Tests in such classes are executed sequentially without any other tests running at the same time. ==== In addition to the `String` that uniquely identifies the shared resource, you may specify an access mode. Two tests that require `READ` access to a shared resource may run in parallel with each other but not while any other test that requires `READ_WRITE` access to the same shared resource is running. [source,java] ---- include::{testDir}/example/SharedResourcesDemo.java[tags=user_guide] ---- [[writing-tests-built-in-extensions]] === Built-in Extensions While the JUnit team encourages reusable extensions to be packaged and maintained in separate libraries, the JUnit Jupiter API artifact includes a few user-facing extension implementations that are considered so generally useful that users shouldn't have to add another dependency. [[writing-tests-built-in-extensions-TempDirectory]] ==== The TempDirectory Extension The built-in `{TempDirectory}` extension is used to create and clean up a temporary directory for an individual test or all tests in a test class. It is registered by default. To use it, annotate a non-final, unassigned field of type `java.nio.file.Path` or `java.io.File` with `{TempDir}` or add a parameter of type `java.nio.file.Path` or `java.io.File` annotated with `@TempDir` to a lifecycle method or test method. For example, the following test declares a parameter annotated with `@TempDir` for a single test method, creates and writes to a file in the temporary directory, and checks its content. [source,java,indent=0] .A test method that requires a temporary directory ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_parameter_injection] ---- You can inject multiple temporary directories by specifying multiple annotated parameters. [source,java,indent=0] .A test method that requires multiple temporary directories ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_multiple_directories] ---- WARNING: To revert to the old behavior of using a single temporary directory for the entire test class or method (depending on which level the annotation is used), you can set the `junit.jupiter.tempdir.scope` configuration parameter to `per_context`. However, please note that this option is deprecated and will be removed in a future release. `@TempDir` is not supported on constructor parameters. If you wish to retain a single reference to a temp directory across lifecycle methods and the current test method, please use field injection by annotating an instance field with `@TempDir`. The following example stores a _shared_ temporary directory in a `static` field. This allows the same `sharedTempDir` to be used in all lifecycle methods and test methods of the test class. For better isolation, you should use an instance field so that each test method uses a separate directory. [source,java,indent=0] .A test class that shares a temporary directory across test methods ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_field_injection] ---- The `@TempDir` annotation has an optional `cleanup` attribute that can be set to either `NEVER`, `ON_SUCCESS`, or `ALWAYS`. If the cleanup mode is set to `NEVER`, temporary directories are not deleted after a test completes. If it is set to `ON_SUCCESS`, temporary directories are deleted only after a test completed successfully. The default cleanup mode is `ALWAYS`. You can use the `junit.jupiter.tempdir.cleanup.mode.default` <> to override this default. [source,java,indent=0] .A test class with a temporary directory that doesn't get cleaned up ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_cleanup_mode] ---- `@TempDir` supports the programmatic creation of temporary directories via the optional `factory` attribute. This is typically used to gain control over the temporary directory creation, like defining the parent directory or the file system that should be used. Factories can be created by implementing `TempDirFactory`. Implementations must provide a no-args constructor and should not make any assumptions regarding when and how many times they are instantiated, but they can assume that their `createTempDirectory(...)` and `close()` methods will both be called once per instance, in this order, and from the same thread. The default implementation available in Jupiter delegates the directory creation to `java.nio.file.Files::createTempDirectory`, passing `junit` as the prefix string to be used in generating the directory's name. The following example defines a factory that uses the test name as the directory name prefix instead of the `junit` constant value. [source,java,indent=0] .A test class with a temporary directory having the test name as the directory name prefix ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_factory_name_prefix] ---- It's also possible to use an in-memory file system like `{Jimfs}` for the creation of the temporary directory. The following example demonstrates how to achieve that. [source,java,indent=0] .A test class with a temporary directory created with the Jimfs in-memory file system ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_factory_jimfs] ---- `@TempDir` can also be used as a <> to reduce repetition. The following code listing shows how to create a custom `@JimfsTempDir` annotation that can be used as a drop-in replacement for `@TempDir(factory = JimfsTempDirFactory.class)`. [source,java,indent=0] .A custom annotation meta-annotated with `@TempDir` ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_composed_annotation] ---- The following example demonstrates how to use the custom `@JimfsTempDir` annotation. [source,java,indent=0] .A test class using the custom annotation ---- include::{testDir}/example/TempDirectoryDemo.java[tags=user_guide_composed_annotation_usage] ---- Meta-annotations or additional annotations on the field or parameter the `TempDir` annotation is declared on might expose additional attributes to configure the factory. Such annotations and related attributes can be accessed via the `AnnotatedElementContext` parameter of `createTempDirectory`. You can use the `junit.jupiter.tempdir.factory.default` <> to specify the fully qualified class name of the `TempDirFactory` you would like to use by default. Just like for factories configured via the `factory` attribute of the `@TempDir` annotation, the supplied class has to implement the `TempDirFactory` interface. The default factory will be used for all `@TempDir` annotations unless the `factory` attribute of the annotation specifies a different factory. In summary, the factory for a temporary directory is determined according to the following precedence rules: 1. The `factory` attribute of the `@TempDir` annotation, if present 2. The default `TempDirFactory` configured via the configuration parameter, if present 3. Otherwise, `org.junit.jupiter.api.io.TempDirFactory$Standard` will be used.