Configuring nextest¶

cargo-nextest supports repository-specific configuration at the location .config/nextest.toml from the Cargo workspace root. The location of the configuration file can be overridden with the --config-file option.

The default configuration shipped with cargo-nextest is:

# This is the default config used by nextest. It is embedded in the binary at
# build time. It may be used as a template for .config/nextest.toml.

[store]
# The directory under the workspace root at which nextest-related files are
# written. Profile-specific storage is currently written to dir/<profile-name>.
dir = "target/nextest"

# This section defines the default nextest profile. Custom profiles are layered
# on top of the default profile.
[profile.default]
# The set of tests run by `cargo nextest run` by default.
default-filter = "all()"

# "retries" defines the number of times a test should be retried. If set to a
# non-zero value, tests that succeed on a subsequent attempt will be marked as
# flaky. Can be overridden through the `--retries` option.
# Examples
# * retries = 3
# * retries = { backoff = "fixed", count = 2, delay = "1s" }
# * retries = { backoff = "exponential", count = 10, delay = "1s", jitter = true, max-delay = "10s" }
retries = 0

# The number of threads to run tests with. Supported values are either an integer or
# the string "num-cpus". Can be overridden through the `--test-threads` option.
test-threads = "num-cpus"

# The number of threads required for each test. This is generally used in overrides to
# mark certain tests as heavier than others. However, it can also be set as a global parameter.
threads-required = 1

# Extra arguments to pass in to the test binary at runtime. Intended primarily for
# communication with custom test harnesses -- use with caution!
run-extra-args = []

# Show these test statuses in the output.
#
# The possible values this can take are:
# * none: no output
# * fail: show failed (including exec-failed) tests
# * retry: show flaky and retried tests
# * slow: show slow tests
# * pass: show passed tests
# * skip: show skipped tests (most useful for CI)
# * all: all of the above
#
# Each value includes all the values above it; for example, "slow" includes
# failed and retried tests.
#
# Can be overridden through the `--status-level` flag.
status-level = "pass"

# Similar to status-level, show these test statuses at the end of the run.
final-status-level = "flaky"

# "failure-output" defines when standard output and standard error for failing tests are produced.
# Accepted values are
# * "immediate": output failures as soon as they happen
# * "final": output failures at the end of the test run
# * "immediate-final": output failures as soon as they happen and at the end of
#   the test run; combination of "immediate" and "final"
# * "never": don't output failures at all
#
# For large test suites and CI it is generally useful to use "immediate-final".
#
# Can be overridden through the `--failure-output` option.
failure-output = "immediate"

# "success-output" controls production of standard output and standard error on success. This should
# generally be set to "never".
success-output = "never"

# Cancel the test run on the first failure. For CI runs, consider setting this
# to false.
fail-fast = true

# Treat a test that takes longer than the configured 'period' as slow, and print a message.
# See <https://nexte.st/docs/features/slow-tests> for more information.
#
# Optional: specify the parameter 'terminate-after' with a non-zero integer,
# which will cause slow tests to be terminated after the specified number of
# periods have passed.
# Example: slow-timeout = { period = "60s", terminate-after = 2 }
slow-timeout = { period = "60s" }

# Treat a test as leaky if after the process is shut down, standard output and standard error
# aren't closed within this duration.
#
# This usually happens in case of a test that creates a child process and lets it inherit those
# handles, but doesn't clean the child process up (especially when it fails).
#
# See <https://nexte.st/docs/features/leaky-tests> for more information.
leak-timeout = "100ms"

# `nextest archive` automatically includes any build output required by a standard build.
# However sometimes extra non-standard files are required.
# To address this, "archive.include" specifies additional paths that will be included in the archive.
archive.include = [
    # Examples:
    #
    # { path = "application-data", relative-to = "target" },
    # { path = "data-from-some-dependency/file.txt", relative-to = "target" },
    #
    # In the above example:
    # * the directory and its contents at "target/application-data" will be included recursively in the archive.
    # * the file "target/data-from-some-dependency/file.txt" will be included in the archive.
]

[profile.default.junit]
# Output a JUnit report into the given file inside 'store.dir/<profile-name>'.
# If unspecified, JUnit is not written out.

# path = "junit.xml"

# The name of the top-level "report" element in JUnit report. If aggregating
# reports across different test runs, it may be useful to provide separate names
# for each report.
report-name = "nextest-run"

# Whether standard output and standard error for passing tests should be stored in the JUnit report.
# Output is stored in the <system-out> and <system-err> elements of the <testcase> element.
store-success-output = false

# Whether standard output and standard error for failing tests should be stored in the JUnit report.
# Output is stored in the <system-out> and <system-err> elements of the <testcase> element.
#
# Note that if a description can be extracted from the output, it is always stored in the
# <description> element.
store-failure-output = true

# This profile is activated if MIRI_SYSROOT is set.
[profile.default-miri]

Profiles¶

With cargo-nextest, local and CI runs often need to use different settings. For example, CI test runs should not be cancelled as soon as the first test failure is seen.

cargo-nextest supports multiple profiles, where each profile is a set of options for cargo-nextest. Profiles are selected on the command line with the -P or --profile option. Most individual configuration settings can also be overridden at the command line.

Here is a recommended profile for CI runs:

Configuring a CI profile in .config/nextest.toml

[profile.ci]
# Do not cancel the test run on the first failure.
fail-fast = false

After checking the profile into .config/nextest.toml, use cargo nextest --profile ci in your CI runs.

Default profiles

Nextest's embedded configuration may define new profiles whose names start with default- in the future. To avoid backwards compatibility issues, do not name custom profiles starting with default-.

Tool-specific configuration¶

Some tools that integrate with nextest may wish to customize nextest's defaults. However, in most cases, command-line arguments and repository-specific configuration should still override those defaults.

To support these tools, nextest supports the --tool-config-file argument. Values to this argument are specified in the form tool:/path/to/config.toml. For example, if your tool my-tool needs to call nextest with customized defaults, it should run:

cargo nextest run --tool-config-file my-tool:/path/to/my/config.toml

The --tool-config-file argument may be specified multiple times. Config files specified earlier are higher priority than those that come later.

Hierarchical configuration¶

For this example:

Configuration is resolved in the following order:

Command-line arguments. For example, if --retries=3 is specified on the command line, failing tests are retried up to 3 times.
Environment variables. For example, if NEXTEST_RETRIES=4 is specified on the command line, failing tests are retried up to 4 times.
Per-test overrides, if they're supported for this configuration variable.
If a profile is specified, profile-specific configuration in .config/nextest.toml. For example, if the repository-specific configuration looks like:
```
[profile.ci]
retries = 2
```
then, if --profile ci is selected, failing tests are retried up to 2 times.
If a profile is specified, tool-specific configuration for the given profile.
Repository-specific configuration for the default profile. For example, if the repository-specific configuration looks like:
```
[profile.default]
retries = 5
```
then failing tests are retried up to 5 times.
Tool-specific configuration for the default profile.
The default configuration listed above, which is that tests are never retried.