Filter expressions
Nextest supports a domain-specific language (DSL) for filtering tests. The DSL is inspired by, and is similar to, Bazel query and Mercurial revsets.
Example: Running all tests in a crate and its dependencies
To run all tests in my-crate
and its dependencies, run:
cargo nextest run -E 'deps(my-crate)'
The argument passed into the -E
command-line option is called a filter expression. The rest of this page describes the full syntax for the expression DSL.
The filter expression DSL
A filter expression defines a set of tests. A test will be run if it matches a filter expression.
On the command line, multiple filter expressions can be passed in. A test will be run if it matches any of these expressions. For example, to run tests whose names contain the string my_test
as well as all tests in package my-crate
, run:
cargo nextest run -E 'test(my_test)' -E 'package(my-crate)'
This is equivalent to:
cargo nextest run -E 'test(my_test) + package(my-crate)'
Examples
package(serde) and test(deserialize)
: every test containing the stringdeserialize
in the packageserde
deps(nextest*)
: all tests in packages whose names start withnextest
, and all of their (possibly transitive) dependenciesnot (test(/parse[0-9]*/) | test(run))
: every test name not matching the regexparse[0-9]*
or the substringrun
Note: If you pass in both a filter expression and a standard, substring-based filter, tests must match both filter expressions and substring-based filters.
For example, the command:
cargo nextest run -E 'package(foo)' -- test_bar test_baz
will run all tests that are both in package
foo
and matchtest_bar
ortest_baz
.
DSL reference
This section contains the full set of operators supported by the DSL.
Basic predicates
all()
: include all tests.test(name-matcher)
: include all tests matchingname-matcher
.package(name-matcher)
: include all tests in packages (crates) matchingname-matcher
.deps(name-matcher)
: include all tests in crates matchingname-matcher
, and all of their (possibly transitive) dependencies.rdeps(name-matcher)
: include all tests in crates matchingname-matcher
, and all the crates that (possibly transitively) depend onname-matcher
.kind(name-matcher)
: include all tests in binary kinds matchingname-matcher
. Binary kinds include:lib
for unit tests, typically in thesrc/
directorytest
for integration tests, typically in thetests/
directorybench
for benchmark testsbin
for tests within[[bin]]
targetsproc-macro
for tests in thesrc/
directory of a procedural macro
binary(name-matcher)
: include all tests in binary names matchingname-matcher
.- For tests of kind
lib
andproc-macro
, the binary name is the same as the name of the crate. - Otherwise, it's the name of the integration test, benchmark, or binary target.
- For tests of kind
binary_id(name-matcher)
: include all tests in binary IDs matchingname-matcher
.platform(host)
orplatform(target)
: include all tests that are built for the host or target platform, respectively.none()
: include no tests.
Note: If a filter expression always excludes a particular binary, it will not be run, even to get the list of tests within it. This means that a command like:
cargo nextest list -E 'platform(host)'
will not execute any test binaries built for the target platform. This is generally what you want, but if you would like to list tests anyway, include a
test()
predicate. For example, to list test binaries for the target platform (using, for example, a target runner), but skip running them:cargo nextest list -E 'platform(host) + not test(/.*/)' --verbose
Name matchers
=string
: equality matcher—match a package or test name that's equal tostring
.~string
: contains matcher—match a package or test name containingstring
./regex/
: regex matcher—match a package or test name if any part of it matches the regular expressionregex
. To match the entire string against a regular expression, use/^regex$/
. The implementation uses the regex crate.#glob
: glob matcher—match a package or test name if the full name matches the glob expressionglob
. The implementation uses the globset crate.string
: Default matching strategy.- For
test()
predicates, this is the contains matcher, equivalent to~string
. - For package-related predicates (
package()
,deps()
, andrdeps()
), this is the glob matcher, equivalent to#string
. - For binary-related predicates (
binary()
andbinary_id()
), this is also the glob matcher. - For
kind()
andplatform()
, this is the equality matcher, equivalent to=string
.
- For
If you're constructing an expression string programmatically, always use a prefix to avoid ambiguity.
Escape sequences
The equality, contains, and glob matchers can contain escape sequences, preceded by a
backslash (\
).
\n
: line feed\r
: carriage return\t
: tab\\
: backslash\/
: forward slash\)
: closing parenthesis\,
: comma\u{7FFF}
: 24-bit Unicode character code (up to 6 hex digits)
For the glob matcher, to match against a literal glob metacharacter such as *
or ?
, enclose it in square brackets: [*]
or [?]
.
All other escape sequences are invalid.
The regular expression matcher supports the same escape sequences that the regex crate does. This includes character classes like \d
. Additionally, \/
is interpreted as an escaped /
.
Operators
set_1 & set_2
,set_1 and set_2
: the intersection ofset_1
andset_2
set_1 | set_2
,set_1 + set_2
,set_1 or set_2
: the union ofset_1
orset_2
not set
,!set
: include everything not included inset
set_1 - set_2
: equivalent toset_1 and not set_2
(set)
: include everything inset
Operator precedence
In order from highest to lowest, or in other words from tightest to loosest binding:
()
not
,!
and
,&
,-
or
,|
,+
Within a precedence group, operators bind from left to right.
Examples
test(a) & test(b) | test(c)
is equivalent to(test(a) & test(b)) | test(c)
.test(a) | test(b) & test(c)
is equivalent totest(a) | (test(b) & test(c))
.test(a) & test(b) - test(c)
is equivalent to(test(a) & test(b)) - test(c)
.not test(a) | test(b)
is equivalent to(not test(a)) | test(b)
.
More information about filter expressions
This section covers additional information that is of interest to nextest's developers and curious readers.
Click to expand
Motivation
Developer tools often work with some notion of sets, and many of them have grown some kind of domain-specific query language to be able to efficiently specify those sets.
The biggest advantage of a query language is orthogonality: rather than every command having to grow a number of options such as --include
and --exclude
, developers can learn the query language once and use it everywhere.
Design decisions
Nextest's filter expressions are meant to be specified at the command line as well as in configuration. This led to the following design decisions:
- No quotes: Filter expressions do not have embedded quotes. This lets users use either single (
''
) or double quotes (""
) to specify filter expressions, without having to worry about escaping them. - Minimize nesting of parens: If an expression language uses parentheses or other brackets heavily (e.g. Rust's
cfg()
expressions), getting them wrong can be annoying when trying to write an expression. Text editors typically highlight matching and missing parens, but there's so such immediate feedback on the command line. - Infix operators: Nextest's filter expressions use infix operators, which are more natural to read and write for most people. (As an alternative, Rust's
cfg()
expressions use the prefix operatorsall()
andany()
). - Operator aliases: Operators are supported as both words (
and
,or
,not
) and symbols (&
,|
,+
,-
,!
), letting users write expressions in the style most natural to them. Filter expressions are a small language, so there's no need to be particularly opinionated.