JUnit support
cargo-nextest can produce output in the JUnit/XUnit XML format. This format is widely understood by test analysis tools and libraries.
To enable JUnit support, add this to your nextest configuration:
[profile.ci.junit] # this can be some other profile, too
path = "junit.xml"
If --profile ci is selected on the command line, a JUnit report will be written out to target/nextest/ci/junit.xml within the workspace root.
Some notes about the JUnit support:
- There are several slightly different formats all called "JUnit" or "XUnit". cargo-nextest adheres to the Jenkins XML format.
- Every test binary forms a single
<testsuite>. Every test forms a single<testcase>. - Standard output and standard error are included for failed and retried tests. (However, invalid XML characters are stripped out.)
Configuration
Configuration options supported for JUnit reports, within the junit section:
report-name— The name of the report. Defaults to"nextest-run".store-success-output— Whether to store output for successful tests in the<system-out>and<system-err>elements. Defaults to false.store-failure-output— Whether to store output for failing tests in the<system-out>and<system-err>elements. Defaults to true.
store-success-output and store-failure-output can also be configured as per-test overrides.
Example configuration
[profile.default.junit]
path = "junit.xml"
# These are the default values, specified for clarity.
store-success-output = false
store-failure-output = true
[[profile.default.overrides]]
filter = 'test(important-test)'
junit.store-success-output = true
In this example, the JUnit report will contain the output for all failing tests, and for successful tests that contain "important-test" in the name.
Post-processing
Some tools that read JUnit files don't follow the Jenkins standard. You can post-process the JUnit file in such cases. Here's some recommendations for post-processing tools written by community members:
- CircleCI:
circleci-junit-fix
Example
Here's an example JUnit file generated by cargo-nextest.
<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="nextest-run" tests="3" failures="1" errors="0" uuid="9ea8d2ee-e485-4e5b-8dba-4985c12081f7" timestamp="2022-07-28T01:09:27.584+00:00" time="0.007">
<testsuite name="nextest-tests::basic" tests="3" disabled="0" errors="0" failures="1">
<testcase name="test_cwd" classname="nextest-tests::basic" timestamp="2022-07-28T01:09:27.584+00:00" time="0.002">
</testcase>
<testcase name="test_failure_assert" classname="nextest-tests::basic" timestamp="2022-07-28T01:09:27.585+00:00" time="0.002">
<failure type="test failure">thread 'test_failure_assert' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`: this is an assertion', tests/basic.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace</failure>
<rerunFailure timestamp="2022-07-28T01:09:27.586+00:00" time="0.002" type="test failure">thread 'test_failure_assert' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`: this is an assertion', tests/basic.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_failure_assert ... FAILED
failures:
failures:
test_failure_assert
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_failure_assert' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`: this is an assertion', tests/basic.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</rerunFailure>
<rerunFailure timestamp="2022-07-28T01:09:27.588+00:00" time="0.002" type="test failure">thread 'test_failure_assert' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`: this is an assertion', tests/basic.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_failure_assert ... FAILED
failures:
failures:
test_failure_assert
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_failure_assert' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`: this is an assertion', tests/basic.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</rerunFailure>
<system-out>
running 1 test
test test_failure_assert ... FAILED
failures:
failures:
test_failure_assert
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_failure_assert' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`: this is an assertion', tests/basic.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</testcase>
<testcase name="test_flaky_mod_4" classname="nextest-tests::basic" timestamp="2022-07-28T01:09:27.589+00:00" time="0.001">
<flakyFailure timestamp="2022-07-28T01:09:27.586+00:00" time="0.002" type="test failure">thread 'test_flaky_mod_4' panicked at 'Failed because attempt 1 % 4 != 0', tests/basic.rs:33:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_flaky_mod_4 ... FAILED
failures:
failures:
test_flaky_mod_4
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_flaky_mod_4' panicked at 'Failed because attempt 1 % 4 != 0', tests/basic.rs:33:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</flakyFailure>
<flakyFailure timestamp="2022-07-28T01:09:27.587+00:00" time="0.002" type="test failure">thread 'test_flaky_mod_4' panicked at 'Failed because attempt 2 % 4 != 0', tests/basic.rs:33:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
<system-out>
running 1 test
test test_flaky_mod_4 ... FAILED
failures:
failures:
test_flaky_mod_4
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 17 filtered out; finished in 0.00s
</system-out>
<system-err>thread 'test_flaky_mod_4' panicked at 'Failed because attempt 2 % 4 != 0', tests/basic.rs:33:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
</system-err>
</flakyFailure>
</testcase>
</testsuite>
</testsuites>