pub fn compute_dir_hash(dir: impl AsRef<Utf8Path>) -> Result<Sha256Hash>Expand description
Computes the hash of a directory and its contents, in a way that hopefully represents what Cargo does somewhat.
With any cache, invalidation is an issue – specifically, Cargo has its own notion of cache invalidation. Ideally, we could ask Cargo to give us a hash for a particular command that deterministically says “a rebuild will happen if and only if this hash changes”. But that doesn’t exist with stable Rust as of this writing (Rust 1.83), so we must guess at what Cargo does.
We take some basic precautions:
- preserving mtimes while copying the source directory
- using both mtimes and hashes while computing the overall hash below.
Beyond that, it’s possible for this implementation to have issues in three different ways:
§1. Cargo invalidates cache, but we don’t
In this case, the cache becomes useless – Cargo will rebuild the project
anyway. This can cause flaky tests (see __NEXTEST_ALT_TARGET_DIR for a fix
to a flake that was caught because of this divergence).
To be clear, any divergence merely due to the cached seed not being used is
a bug. That was the case with the issue which __NEXTEST_ALT_TARGET_DIR
works around.
§2. We invalidate our cache, but Cargo doesn’t
In this case, we’ll regenerate a new seed but Cargo will reuse it. This isn’t too bad since generating the seed is a one-time cost.
§3. Something about the way nextest generates archives changes
This is the most difficult case to handle, because a brute hash (just hash all of the files in the nextest repo) would invalidate far too often. So if you’re altering this code, you have to be careful to remove the cache as well. Hopefully CI (which doesn’t cache the seed archive) will catch issues.
In general, this implementation appears to be pretty reliable, though occasionally the cache has not worked (case 1 above) in Windows CI.