Stability policy

This section contains information on how cargo-nextest will evolve in a backwards-compatible way over time.

The cargo-nextest binary

The cargo-nextest binary follows semantic versioning, where the public API consists of exactly the following:

  1. command-line arguments, options and flags
  2. machine-readable output
  3. the configuration file format

Experimental features are not part of the public API. They may change or be removed in a patch release.

Within a version series, the public API will be append-only. New options or keys may be added, but existing keys will continue to be as they were. Existing options may be deprecated but will not be removed within a version series, except via the policy listed in Making behavior changes.

The public API does not include human-readable output generated by nextest, or in general anything printed to stderr.

Making behavior changes

In general, the meaning of options will not change and functionality will not be removed from cargo-nextest within a version series.

However, in some cases, we may need to make behavior changes to fix a bug or security issue. These sorts of changes will be made based on standard semantic versioning rules: bugs will be fixed in patch releases, with due consideration given to existing use cases.

We may also need to make behavior changes to address what we consider to be major user interface deficiencies. This will not be done lightly: the maintainers of nextest understand the importance of stable, dependable CLI interfaces.

In those cases, we will follow the following procedure:

  1. On the nextest issue tracker, an issue will be filed under the behavior-change label. This issue will outline the motivation, and be used for coordination and feedback.
  2. The behavior will be marked as deprecated and a warning will be provided within the CLI. If applicable, an alternative will be provided in the warning message. If there is no alternative, a configuration option and/or environment variable will be provided to silence the warning.
  3. A timeline to change the behavior will be provided. The timeline will be at least 3 months long to provide ample time for users to adapt to the new behavior.
  4. After that time has elapsed, the behavior will be changed in the following version of nextest.

Libraries

The libraries used by cargo-nextest, nextest-metadata and nextest-runner, follow the standard Rust library versioning policy.

nextest-metadata

nextest-metadata contains data structures for deserializing cargo-nextest's machine-readable output.

nextest-metadata is forward-compatible with cargo-nextest. A given version of nextest-metadata will continue to work with all future versions of cargo-nextest (within the same version series).

However, currently, nextest-metadata is not backwards-compatible: a new version of nextest-metadata may not be able to parse metadata generated by older versions of cargo-nextest. In other words, each version of nextest-metadata has a minimum supported cargo-nextest version, analogous to a crate's minimum supported Rust version (MSRV).

A bump to the minimum supported cargo-nextest version is considered a breaking change, and will be paired with a major version bump.

(The policy around backwards compatibility may be relaxed in the future based on user needs.)

nextest-runner

nextest-runner is built to serve the needs of cargo-nextest. Every cargo-nextest release is likely to correspond to a breaking change to nextest-runner.

Minimum supported Rust version (MSRV)

The MSRV of cargo-nextest or dependent crates may be changed in a patch release. At least the last 3 versions of Rust will be supported at any time.