[[testkit]] === JUnit Platform Test Kit The `junit-platform-testkit` artifact provides support for executing a test plan on the JUnit Platform and then verifying the expected results. As of JUnit Platform 1.4, this support is limited to the execution of a single `TestEngine` (see <>). [[testkit-engine]] ==== Engine Test Kit The `{testkit-engine-package}` package provides support for executing a `{TestPlan}` for a given `{TestEngine}` running on the JUnit Platform and then accessing the results via a fluent API to verify the expected results. The key entry point into this API is the `{EngineTestKit}` which provides static factory methods named `engine()` and `execute()`. It is recommended that you select one of the `engine()` variants to benefit from the fluent API for building a `LauncherDiscoveryRequest`. NOTE: If you prefer to use the `LauncherDiscoveryRequestBuilder` from the `Launcher` API to build your `LauncherDiscoveryRequest`, you must use one of the `execute()` variants in `EngineTestKit`. The following test class written using JUnit Jupiter will be used in subsequent examples. [[testkit-engine-ExampleTestCase]] [source,java,indent=0] ---- include::{testDir}/example/ExampleTestCase.java[tags=user_guide] ---- For the sake of brevity, the following sections demonstrate how to test JUnit's own `JupiterTestEngine` whose unique engine ID is `"junit-jupiter"`. If you want to test your own `TestEngine` implementation, you need to use its unique engine ID. Alternatively, you may test your own `TestEngine` by supplying an instance of it to the `EngineTestKit.engine(TestEngine)` static factory method. [[testkit-engine-statistics]] ==== Asserting Statistics One of the most common features of the Test Kit is the ability to assert statistics against events fired during the execution of a `TestPlan`. The following tests demonstrate how to assert statistics for _containers_ and _tests_ in the JUnit Jupiter `TestEngine`. For details on what statistics are available, consult the Javadoc for `{EventStatistics}`. [source,java,indent=0] ---- include::{testDir}/example/testkit/EngineTestKitStatisticsDemo.java[tags=user_guide] ---- <1> Select the JUnit Jupiter `TestEngine`. <2> Select the <> test class. <3> Execute the `TestPlan`. <4> Filter by _container_ events. <5> Assert statistics for _container_ events. <6> Filter by _test_ events. <7> Assert statistics for _test_ events. NOTE: In the `verifyJupiterContainerStats()` test method, the counts for the `started` and `succeeded` statistics are `2` since the `JupiterTestEngine` and the <> class are both considered containers. [[testkit-engine-events]] ==== Asserting Events If you find that <> alone is insufficient for verifying the expected behavior of test execution, you can work directly with the recorded `{Event}` elements and perform assertions against them. For example, if you want to verify the reason that the `skippedTest()` method in <> was skipped, you can do that as follows. [TIP] ==== The `assertThatEvents()` method in the following example is a shortcut for `org.assertj.core.api.Assertions.assertThat(events.list())` from the {AssertJ} assertion library. For details on what _conditions_ are available for use with AssertJ assertions against events, consult the Javadoc for `{EventConditions}`. ==== [source,java,indent=0] ---- include::{testDir}/example/testkit/EngineTestKitSkippedMethodDemo.java[tags=user_guide] ---- <1> Select the JUnit Jupiter `TestEngine`. <2> Select the `skippedTest()` method in the <> test class. <3> Execute the `TestPlan`. <4> Filter by _test_ events. <5> Save the _test_ `Events` to a local variable. <6> Optionally assert the expected statistics. <7> Assert that the recorded _test_ events contain exactly one skipped test named `skippedTest` with `"for demonstration purposes"` as the _reason_. If you want to verify the type of exception thrown from the `failingTest()` method in <>, you can do that as follows. [TIP] ==== For details on what _conditions_ are available for use with AssertJ assertions against events and execution results, consult the Javadoc for `{EventConditions}` and `{TestExecutionResultConditions}`, respectively. ==== [source,java,indent=0] ---- include::{testDir}/example/testkit/EngineTestKitFailedMethodDemo.java[tags=user_guide] ---- <1> Select the JUnit Jupiter `TestEngine`. <2> Select the <> test class. <3> Execute the `TestPlan`. <4> Filter by _test_ events. <5> Assert that the recorded _test_ events contain exactly one failing test named `failingTest` with an exception of type `ArithmeticException` and an error message equal to `"/ by zero"`. Although typically unnecessary, there are times when you need to verify **all** of the events fired during the execution of a `TestPlan`. The following test demonstrates how to achieve this via the `assertEventsMatchExactly()` method in the `EngineTestKit` API. [TIP] ==== Since `assertEventsMatchExactly()` matches conditions exactly in the order in which the events were fired, <> has been annotated with `@TestMethodOrder(OrderAnnotation.class)` and each test method has been annotated with `@Order(...)`. This allows us to enforce the order in which the test methods are executed, which in turn allows our `verifyAllJupiterEvents()` test to be reliable. ==== If you want to do a _partial_ match _with_ or _without_ ordering requirements, you can use the methods `assertEventsMatchLooselyInOrder()` and `assertEventsMatchLoosely()`, respectively. [source,java,indent=0] ---- include::{testDir}/example/testkit/EngineTestKitAllEventsDemo.java[tags=user_guide] ---- <1> Select the JUnit Jupiter `TestEngine`. <2> Select the <> test class. <3> Execute the `TestPlan`. <4> Filter by _all_ events. <5> Print all events to the supplied `writer` for debugging purposes. Debug information can also be written to an `OutputStream` such as `System.out` or `System.err`. <6> Assert _all_ events in exactly the order in which they were fired by the test engine. The `debug()` invocation from the preceding example results in output similar to the following. [source,options="nowrap"] ---- All Events: Event [type = STARTED, testDescriptor = JupiterEngineDescriptor: [engine:junit-jupiter], timestamp = 2018-12-14T12:45:14.082280Z, payload = null] Event [type = STARTED, testDescriptor = ClassTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase], timestamp = 2018-12-14T12:45:14.089339Z, payload = null] Event [type = SKIPPED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:skippedTest()], timestamp = 2018-12-14T12:45:14.094314Z, payload = 'for demonstration purposes'] Event [type = STARTED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:succeedingTest()], timestamp = 2018-12-14T12:45:14.095182Z, payload = null] Event [type = FINISHED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:succeedingTest()], timestamp = 2018-12-14T12:45:14.104922Z, payload = TestExecutionResult [status = SUCCESSFUL, throwable = null]] Event [type = STARTED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:abortedTest()], timestamp = 2018-12-14T12:45:14.106121Z, payload = null] Event [type = FINISHED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:abortedTest()], timestamp = 2018-12-14T12:45:14.109956Z, payload = TestExecutionResult [status = ABORTED, throwable = org.opentest4j.TestAbortedException: Assumption failed: abc does not contain Z]] Event [type = STARTED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:failingTest()], timestamp = 2018-12-14T12:45:14.110680Z, payload = null] Event [type = FINISHED, testDescriptor = TestMethodTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase]/[method:failingTest()], timestamp = 2018-12-14T12:45:14.111217Z, payload = TestExecutionResult [status = FAILED, throwable = java.lang.ArithmeticException: / by zero]] Event [type = FINISHED, testDescriptor = ClassTestDescriptor: [engine:junit-jupiter]/[class:example.ExampleTestCase], timestamp = 2018-12-14T12:45:14.113731Z, payload = TestExecutionResult [status = SUCCESSFUL, throwable = null]] Event [type = FINISHED, testDescriptor = JupiterEngineDescriptor: [engine:junit-jupiter], timestamp = 2018-12-14T12:45:14.113806Z, payload = TestExecutionResult [status = SUCCESSFUL, throwable = null]] ----