12. Testing, Documentation, and Quality

The compiler rules out whole classes of bugs, but it cannot tell whether your function computes the right answer, whether a refactor kept behaviour intact, or whether a change made the hot path twice as slow. This chapter covers the tools Rust provides for those questions: the built-in test harness, advanced testing techniques (doc tests, property testing, fuzzing, mocking, snapshots, coverage), benchmarking and profiling, and writing documentation that is itself tested. Together they turn "it compiles" into "it is correct, fast enough, and usable by others".

<a id="121-testing-fundamentals"></a>

12.1 Testing Fundamentals

Rust ships a test framework in the toolchain itself: no runner to install and no configuration to write. This section covers how unit and integration tests are written, where they live, how to express expected failures, and how to select which tests run.

Unit Tests and #[test]

Imagine changing a parsing function to support a new syntax. Without tests, the only way to know whether the old inputs still work is to try them all by hand, every time. A unit test automates that check: a small function that calls one piece of code with known input and asserts on the output. Run hundreds of them in a second and you can refactor without fear.

In Rust, a test is any function annotated with #[test]. When you run cargo test, Cargo compiles your crate a second time with the test cfg enabled and links it against libtest, the built-in harness. libtest collects every #[test] function, runs them on a pool of threads, and reports which passed. A test passes if it returns normally and fails if it panics; the assertion macros are just convenient ways to panic with a useful message.

By convention, unit tests live in the same file as the code they test, inside a tests module guarded by #[cfg(test)] so that it is compiled only for cargo test, never into your real binary:

// src/lib.rs

/// Splits `"key=value"` into its two parts, trimming whitespace.
pub fn parse_kv(line: &str) -> Option<(&str, &str)> {
    let (k, v) = line.split_once('=')?;
    let (k, v) = (k.trim(), v.trim());
    if k.is_empty() { None } else { Some((k, v)) }
}

fn normalize_key(k: &str) -> String {      // private helper
    k.to_ascii_lowercase().replace('-', "_")
}

#[cfg(test)]                                // compiled only by `cargo test`
mod tests {
    use super::*;                           // bring the parent's items, private ones too

    #[test]
    fn splits_and_trims() {
        assert_eq!(parse_kv(" port = 8080 "), Some(("port", "8080")));
    }

    #[test]
    fn rejects_missing_key() {
        assert_eq!(parse_kv("=8080"), None);
        assert!(parse_kv("no equals sign").is_none());
    }

    #[test]
    fn private_functions_are_testable() {
        // Child modules can see their parent's private items.
        assert_eq!(normalize_key("Max-Conn"), "max_conn");
    }
}

The assertion macros:

Macro	Panics when	On failure prints
assert!(cond)	cond is false	The condition's source text
assert_eq!(a, b)	a != b	Both values with {:?} (left, right)
assert_ne!(a, b)	a == b	Both values with {:?}
debug_assert! and friends	Same, but only when debug assertions are on	Same
assert!(matches!(v, Pattern))	v does not match the pattern	The condition's source text

All of them accept an optional format string, which is the cheapest way to make failures easy to diagnose:

#[test]
fn every_line_parses() {
    let config = "host = example.com\nport = 8080\nmode = fast";
    for (n, line) in config.lines().enumerate() {
        // Without the message you learn only that *some* line failed.
        assert!(parse_kv(line).is_some(), "line {} did not parse: {line:?}", n + 1);
    }
}

Running the suite:

$ cargo test
   Compiling kv v0.1.0 (/home/user/kv)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.61s
     Running unittests src/lib.rs (target/debug/deps/kv-3f2a9c1d)

running 4 tests
test tests::private_functions_are_testable ... ok
test tests::rejects_missing_key ... ok
test tests::splits_and_trims ... ok
test tests::every_line_parses ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests kv
...

Two harness behaviours surprise newcomers:

• Output is captured. println! inside a passing test is swallowed so the report stays readable; it is shown only for failing tests. Use cargo test -- --nocapture to see it live, or -- --show-output to print the captured output of passing tests after they finish.
• Tests run in parallel, one thread per test, up to the number of CPUs. Tests that share global state (the current directory, environment variables, a fixed file path, a port) can interfere with each other. Prefer giving each test its own resources (a temporary directory, port 0); as a last resort, run serially with cargo test -- --test-threads=1. Note that since the 2024 edition, std::env::set_var is unsafe precisely because changing the environment while other threads read it is a data race.

Arguments before -- go to Cargo; arguments after -- go to the test binary.

Key Takeaways

• A test is a #[test] function; it passes if it returns and fails if it panics.
• Unit tests go in a #[cfg(test)] mod tests in the same file and can test private items through use super::*.
• assert_eq! prints both values on failure; add a custom message whenever the values alone will not explain the failure.
• The harness captures output and runs tests in parallel: avoid shared global state, or use --nocapture and --test-threads=1 when debugging.

🧪 Practice

1. Write a function fn word_count(s: &str) -> usize and four unit tests covering empty input, a single word, repeated whitespace, and Unicode words.
2. Write a test that prints with println!, make it fail, and compare the output of cargo test, cargo test -- --nocapture, and cargo test -- --show-output.
3. Interview: Two tests pass individually but fail intermittently when run together. What is the most likely cause, and how would you fix it properly rather than with --test-threads=1? (Hint: think about what the harness runs concurrently and what state those tests might share.)

Integration Tests

Unit tests check pieces from the inside. They can call private functions, so they can pass even when the public API is awkward or broken. Integration tests check the crate from the outside, exactly as a user would: they can only call pub items, through the crate's name.

Cargo treats every .rs file directly inside a top-level tests/ directory as a separate test crate that depends on your library:

kv/
├── Cargo.toml
├── src/
│   └── lib.rs            library under test
└── tests/
    ├── parsing.rs        test crate #1 -> its own binary
    ├── config_file.rs    test crate #2 -> its own binary
    └── common/
        └── mod.rs        shared helpers; NOT a test crate (it is in a subdirectory)

// tests/common/mod.rs: helpers shared by several test files
use std::path::PathBuf;

pub fn fixture(name: &str) -> PathBuf {
    // CARGO_MANIFEST_DIR is the package root, regardless of the working directory.
    PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures").join(name)
}

// tests/config_file.rs
mod common;                                  // pull in tests/common/mod.rs

use kv::parse_kv;                            // only the public API is reachable

#[test]
fn parses_the_sample_config() {
    let text = std::fs::read_to_string(common::fixture("sample.conf")).unwrap();
    let pairs: Vec<_> = text.lines().filter_map(parse_kv).collect();
    assert_eq!(pairs.first(), Some(&("host", "example.com")));
}

Why tests/common/mod.rs rather than tests/common.rs? Every top-level file in tests/ becomes a test binary, so tests/common.rs would show up in the output as a crate with "running 0 tests". Files in subdirectories are ignored by test discovery and can be included as modules.

Things to know about integration tests:

Aspect	Behaviour
Visibility	Only pub items; #[cfg(test)] items of the library are not visible, because the library is compiled normally for them
Dependencies	[dependencies] and [dev-dependencies]
Binary crates	A src/main.rs-only crate cannot be imported; move logic to src/lib.rs, or run the executable
Running one file	cargo test --test config_file
Build cost	Each file is linked into its own executable; many files mean many link steps

To test a binary end to end, run the compiled executable. Cargo builds the package's binaries before integration tests and exposes their paths through CARGO_BIN_EXE_<name>:

// tests/cli.rs: exercise the real `kv` executable
use std::process::Command;

#[test]
fn prints_value_for_key() {
    let out = Command::new(env!("CARGO_BIN_EXE_kv"))   // path set by Cargo at compile time
        .args(["get", "port", "tests/fixtures/sample.conf"])
        .output()
        .expect("failed to run binary");

    assert!(out.status.success());
    assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "8080");
}

The assert_cmd crate wraps this pattern with fluent assertions on exit codes and output.

For larger suites, link time adds up. A common pattern is a single integration test crate with modules, which links once:

tests/
└── it/
    ├── main.rs        mod parsing; mod config_file; mod cli;
    ├── parsing.rs
    ├── config_file.rs
    └── cli.rs

Cargo discovers tests/it/main.rs as one test target named it.

Key Takeaways

• Each top-level file in tests/ is a separate crate that sees only the library's public API.
• Shared helpers go in tests/common/mod.rs (a subdirectory), not tests/common.rs.
• Integration tests need a library target; test binaries end to end through env!("CARGO_BIN_EXE_<name>").
• Many test files mean many link steps; a single tests/it/main.rs crate with modules scales better.

🧪 Practice

1. Move the tests of a small library that only exercise public functions from src/lib.rs into tests/api.rs. Which tests could not be moved, and why?
2. Add a tests/common/mod.rs with a helper that creates a temporary directory containing a config file (use the tempfile crate), and use it from two different test files.
3. Interview: You add #[cfg(test)] pub fn test_only_constructor() to your library, but your integration test cannot find it. Why? (Hint: how is the library compiled when an integration test links against it?)

Test Organization and Modules

A test suite is code, and it rots like code. Good organization makes it obvious where a test for a given behaviour belongs, keeps test-only code out of production builds, and keeps the suite fast enough that people actually run it.

Rust has several kinds of test, each with its own place:

Kind	Location	Can access	Compiled as	Run by
Unit	#[cfg(test)] mod tests beside the code	Everything in the module tree, private included	Part of the library/binary test build	cargo test --lib / --bins
Integration	tests/*.rs	Public API	One crate per file	cargo test --test NAME / --tests
Doc	Code blocks in /// and //!	Public API	One crate per example (or merged)	cargo test --doc
Examples	examples/*.rs	Public API	Binaries	Compiled (not run) by cargo test
Benchmarks	benches/*.rs	Public API	Release-mode binaries	cargo bench

When a module's tests grow long, move them into their own file but keep them a child module, so they still see private items:

src/
├── lib.rs
├── parser.rs          ends with:  #[cfg(test)] mod tests;
└── parser/
    └── tests.rs       the body of parser::tests (no `mod tests { }` wrapper)

// src/parser/tests.rs
use super::*;          // `super` is still `parser`, so private items are visible

#[test]
fn tokenizes_numbers() {
    assert_eq!(tokenize("12 + 3").len(), 3);
}

Test-only code in the library belongs behind #[cfg(test)], and test-only dependencies belong in [dev-dependencies], which are never compiled into users' builds:

pub struct Config {
    pub port: u16,
    pub host: String,
}

#[cfg(test)]
impl Config {
    // A builder that only tests need; absent from release builds.
    pub(crate) fn for_test() -> Self {
        Config { port: 0, host: "127.0.0.1".into() }
    }
}

Table-driven tests keep many similar cases readable. A loop with a descriptive message works with no dependencies; the rstest crate turns each case into a separately reported test:

#[test]
fn parse_kv_cases() {
    let cases = [
        ("a=1", Some(("a", "1"))),
        (" a = 1 ", Some(("a", "1"))),
        ("=1", None),
        ("a", None),
    ];
    for (input, expected) in cases {
        assert_eq!(parse_kv(input), expected, "input: {input:?}");
    }
}

// With rstest (dev-dependency): each #[case] is its own test, e.g. kv_cases::case_3
#[rstest::rstest]
#[case("a=1", Some(("a", "1")))]
#[case(" a = 1 ", Some(("a", "1")))]
#[case("=1", None)]
fn kv_cases(#[case] input: &str, #[case] expected: Option<(&str, &str)>) {
    assert_eq!(parse_kv(input), expected);
}

Naming matters because test names are what you read in failure reports and what you filter on. Prefer names that state the behaviour (rejects_missing_key, retries_after_timeout) over names that repeat the function (test_parse_1). Grouping with nested modules gives filterable paths such as parser::tests::numbers::rejects_leading_zero.

Key Takeaways

• Unit tests sit beside the code, integration tests in tests/, examples in examples/, benchmarks in benches/; doc tests live in doc comments.
• Long unit test modules can move to src/<module>/tests.rs and still see private items.
• Keep test helpers behind #[cfg(test)] and test libraries in [dev-dependencies].
• Use table-driven tests (or rstest) for many similar cases, and give tests behaviour-describing names.

🧪 Practice

1. Take a module with an inline mod tests and move its tests to src/<module>/tests.rs without changing any test.
2. Convert five near-identical tests into a single table-driven test, then into an rstest test, and compare the failure output of each when one case breaks.
3. Interview: Where would you put a test that checks a private invariant of a data structure, and where would you put a test that checks a user-facing workflow? Justify each choice. (Hint: consider what each location can see and what breaks when internals are refactored.)

should_panic and Result Tests

Some behaviour is supposed to fail: indexing past the end, constructing an invalid value, parsing garbage. Testing only the happy path leaves those contracts unchecked. Rust offers two ways to write tests about failure, matching its two failure mechanisms (Chapter 7): panics and Result.

#[should_panic] inverts the pass condition: the test passes only if the body panics. Always add expected = "...", a substring the panic message must contain. Otherwise the test also passes when the code panics for an unrelated reason, such as an unwrap() on the wrong line:

pub struct Stack<T> {
    items: Vec<T>,
}

impl<T> Stack<T> {
    pub fn new() -> Self { Stack { items: Vec::new() } }
    pub fn push(&mut self, x: T) { self.items.push(x) }
    pub fn pop(&mut self) -> Option<T> { self.items.pop() }

    /// # Panics
    /// Panics if the stack is empty.
    pub fn peek(&self) -> &T {
        self.items.last().expect("peek on empty stack")
    }
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    #[should_panic(expected = "empty stack")]   // substring of the panic message
    fn peek_on_empty_panics() {
        Stack::<i32>::new().peek();
    }
}

Result-returning tests let you use ? instead of unwrap(). The test fails if it returns Err, and the harness prints the error's Debug representation:

use std::error::Error;

#[test]
fn reads_port_from_config() -> Result<(), Box<dyn Error>> {
    let text = "port = 8080";
    let (_, value) = parse_kv(text).ok_or("no key/value pair")?;
    let port: u16 = value.parse()?;           // ParseIntError converts into Box<dyn Error>
    assert_eq!(port, 8080);
    Ok(())
}

To assert that a function returns an error, which is what you usually want for Result-based APIs, check the value directly. Matching on the error variant is more robust than comparing message strings:

#[derive(Debug, PartialEq)]
pub enum PortError { Empty, NotANumber, Reserved(u16) }

pub fn parse_port(s: &str) -> Result<u16, PortError> {
    let s = s.trim();
    if s.is_empty() { return Err(PortError::Empty); }
    let n: u16 = s.parse().map_err(|_| PortError::NotANumber)?;
    if n < 1024 { Err(PortError::Reserved(n)) } else { Ok(n) }
}

#[test]
fn rejects_reserved_ports() {
    assert_eq!(parse_port("80"), Err(PortError::Reserved(80)));
    assert!(matches!(parse_port("http"), Err(PortError::NotANumber)));
}

Approach	Use when	Caveat
#[should_panic(expected = ...)]	The documented contract is a panic	Must return (); cannot be combined with a Result return type
-> Result<(), E> with ?	Setup steps are fallible and you want concise code	Error location is lost; the report shows only the Debug of the error
assert_eq!(f(x), Err(...)) / matches!	The API returns Result and the error is part of the contract	Requires PartialEq for assert_eq!
std::panic::catch_unwind	You need to check a panic and then continue the test	Requires UnwindSafe closures; rarely needed

Tests always unwind on panic: Cargo ignores panic = "abort" for test and bench builds, because the harness relies on catching each test's panic.

Key Takeaways

• #[should_panic(expected = "...")] passes only if the body panics with a message containing the substring; always include expected.
• Tests may return Result<(), E> (with E: Debug) so that ? works; returning Err fails the test.
• For Result-based APIs, assert on the returned error variant rather than using should_panic.
• should_panic tests must return ().

🧪 Practice

1. Write fn divide(a: i32, b: i32) -> i32 that panics with "division by zero" and test it with should_panic. Then change the panic message and observe what happens to the test.
2. Rewrite three tests that use several .unwrap() calls into Result-returning tests with ?. Compare the failure output of both styles.
3. Interview: Why is #[should_panic] without expected considered a weak test? Describe a bug it would fail to catch. (Hint: what else in the test body could panic?)

Ignoring and Filtering Tests

As a suite grows, running everything on every save becomes slow, and some tests need resources (a database, the network, ten minutes of CPU) that are not always available. libtest lets you mark tests as ignored by default and select tests by name, so the fast feedback loop and the exhaustive CI run can use the same suite.

#[ignore] excludes a test from normal runs while keeping it compiled, so it cannot silently stop building. A reason string documents why:

#[test]
#[ignore = "needs a running PostgreSQL; run with `cargo test -- --ignored`"]
fn migrates_real_database() {
    // ...
}

#[test]
#[cfg_attr(miri, ignore)]               // ignored only when running under Miri (Chapter 14)
fn uses_the_filesystem() {
    // ...
}

Filtering by name uses a substring match on the full test path (module::submodule::test_name):

cargo test parse                       # every test whose path contains "parse"
cargo test -- parse config             # several filters: matches either one
cargo test -- --exact tests::parse_kv  # exactly this path, no substring matching
cargo test -- --skip slow              # everything except paths containing "slow"
cargo test -- --ignored                # ONLY the ignored tests
cargo test -- --include-ignored        # everything, ignored or not
cargo test -- --list                   # list test names without running them

Cargo-level flags select which targets are built and run, before libtest filters by name:

Command	Runs
cargo test --lib	Unit tests of the library only
cargo test --test cli	Only the integration test crate tests/cli.rs
cargo test --tests	All unit and integration tests, no doc tests
cargo test --doc	Only doc tests
cargo test -p kv-core	Only one workspace member
cargo test --no-fail-fast	Keep running other test binaries after one fails

The output tells you what the filter excluded:

running 2 tests
test tests::parse_kv_cases ... ok
test tests::splits_and_trims ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 8 filtered out; finished in 0.00s

Many projects use cargo-nextest (cargo nextest run) as a faster test runner. It runs each test in its own process, which isolates global state and makes per-test timeouts and retries possible, and it produces clearer output for large suites. It does not run doc tests, so CI usually runs cargo nextest run followed by cargo test --doc.

A typical split:

local edit loop:   cargo test parser            (seconds, one area)
pre-push:          cargo test                   (whole fast suite)
CI:                cargo test -- --include-ignored  (or nextest + doc tests)
nightly CI job:    slow, ignored, and external-service tests

Key Takeaways

• #[ignore = "reason"] keeps slow or resource-dependent tests compiled but out of the default run; --ignored or --include-ignored runs them.
• Name filters are substring matches on the full test path; --exact and --skip refine them.
• Arguments before -- select Cargo targets (--lib, --test, --doc); arguments after -- go to libtest.
• cargo-nextest offers process-per-test isolation and better reporting but does not run doc tests.

🧪 Practice

1. Mark one test #[ignore] with a reason, then run the suite three ways: default, --ignored, and --include-ignored. Explain the counts.
2. Using only filters, run every test in the parser module except those whose names contain unicode.
3. Interview: Your CI runs cargo test and takes 25 minutes, mostly because of a few end-to-end tests. How would you restructure the suite and the CI pipeline so developers get fast feedback without losing coverage? (Hint: think about #[ignore], separate jobs, and test targets.)

<a id="122-advanced-testing"></a>

12.2 Advanced Testing

Hand-written example-based tests check the cases you thought of. This section covers techniques that reach further: documentation that is tested, inputs generated by the thousand, fuzzers that hunt for crashes, test doubles that isolate code from the outside world, snapshots for large outputs, and coverage reports that show what the suite never touched.

Doc Tests

Chapters 1 and 8 introduced the headline feature: code blocks in doc comments are compiled and run by cargo test. This topic covers how that works and the controls you need to write doc examples that are honest, readable, and fast.

How rustdoc runs an example. Each fenced code block is treated as a small program outside your crate. Roughly, rustdoc:

1. Adds extern crate your_crate; (so use your_crate::... works).
2. Wraps the code in fn main() { ... } unless it already contains one.
3. Compiles it, runs it, and treats a panic as a failure.

/// ```                               What rustdoc compiles (simplified):
/// use kv::parse_kv;                 extern crate kv;
/// assert_eq!(                       fn main() {
///     parse_kv("a=1"),                  use kv::parse_kv;
///     Some(("a", "1")));                assert_eq!(parse_kv("a=1"), Some(("a", "1")));
/// ```                               }

Because examples are external code, they see only the public API, which makes them a good check that the public API is actually usable. Examples on private items still run, but they cannot call those items.

Hidden lines. Lines starting with # (hash, space) are compiled but not shown in the rendered documentation. Use them for setup that would distract the reader:

/// Reads the configured port.
///
/// ```
/// # use std::error::Error;
/// # fn main() -> Result<(), Box<dyn Error>> {   // hidden: enables `?` below
/// let port: u16 = "8080".parse()?;
/// assert_eq!(port, 8080);
/// # Ok(())
/// # }
/// ```
pub fn port() {}

There is a shorter idiom for ?: if the block ends with an Ok(()) whose error type is spelled out, rustdoc wraps it in a function returning Result:

/// ```
/// let port: u16 = "8080".parse()?;
/// assert_eq!(port, 8080);
/// Ok::<(), std::num::ParseIntError>(())      // turbofish names the error type
/// ```
pub fn port_short() {}

Code block attributes control what rustdoc does with a block:

Fence	Compiled	Run	Use for
```rust or ```	Yes	Yes	Normal examples (the default)
```no_run	Yes	No	Examples that need the network, a server, or run forever
```should_panic	Yes	Must panic	Demonstrating a documented panic
```compile_fail	Must fail	No	Showing what the type system forbids
```compile_fail,E0382	Must fail with that error code	No	A more precise compile_fail
```ignore	No	No	Pseudo-code; avoid, since it is never checked
```text (or any non-Rust language)	No	No	Shell sessions, output, config files
```edition2021	Yes, with that edition	Yes	Edition-specific behaviour
```standalone_crate	As its own crate	Yes	Examples that must not be merged (see below)

/// A `Token` cannot be used after it has been consumed:
///
/// ```compile_fail,E0382
/// let t = auth::Token::new("secret");
/// auth::consume(t);
/// auth::consume(t);                  // error[E0382]: use of moved value
/// ```
///
/// Connecting needs a live server, so this example is only compiled:
///
/// ```no_run
/// let session = auth::connect("db.internal:5432")?;
/// # Ok::<(), auth::Error>(())
/// ```
pub mod docs_only {}

compile_fail tests are useful but fragile: the block passes if it fails to compile for any reason, including a typo. Adding the error code makes the check meaningful.

Merged doc tests (2024 edition). Historically each example was compiled as its own executable, which made doc tests the slowest part of many suites. In the 2024 edition, rustdoc merges compatible examples into a single binary and runs them together; you can see this in the output:

   Doc-tests kv

running 7 tests
.......
test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

running 3 tests                    <- examples that must stay standalone
i..
test result: ok. 2 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.08s

all doctests ran in 0.29s; merged doctests compilation took 0.20s

Examples that use crate-level attributes (#![...]), compile_fail, or that are marked standalone_crate are still compiled separately.

Key Takeaways

• Each doc example is compiled as an external crate wrapped in fn main, so it exercises only the public API.
• Hide setup with # lines; enable ? with a hidden main returning Result or a trailing Ok::<(), E>(()).
• Choose the right fence: no_run for side effects, compile_fail,EXXXX for type-system guarantees, text for non-Rust content; avoid ignore.
• The 2024 edition merges doc tests into one binary, which makes them much faster to run.

🧪 Practice

1. Write a doc example that uses ? twice, first with a hidden main, then with the trailing Ok::<(), E>(()) idiom. Check the rendered output with cargo doc --open.
2. Write a compile_fail example demonstrating that your type is not Send, then add the matching error code and confirm the test still passes.
3. Interview: A colleague marks every failing doc example ignore "to get CI green". What are the risks, and which attributes would you use instead? (Hint: compare what ignore, no_run, and text each verify.)

Property-Based Testing

Example-based tests check the inputs you thought of, and bugs tend to hide in the inputs you did not: empty strings, i32::MIN, a single repeated byte, Unicode combining characters. Property-based testing turns the problem around. Instead of writing inputs and expected outputs, you state a property that must hold for all valid inputs, and the framework generates hundreds of random inputs trying to break it.

When it finds a failing input, the framework shrinks it: it repeatedly tries simpler versions (shorter vectors, smaller numbers) that still fail, and reports the minimal one. You get [0, 0] instead of a 300-element vector of noise.

An analogy: example tests are a checklist inspector, while property tests are a crash-test lab that throws thousands of randomized collisions at the car and then reports the simplest collision that broke it.

The most widely used crate is proptest (quickcheck is a smaller alternative):

[dev-dependencies]
proptest = "1"

// A run-length encoder: "aaab" -> [(b'a', 3), (b'b', 1)]
pub fn encode(input: &[u8]) -> Vec<(u8, u8)> {
    let mut out: Vec<(u8, u8)> = Vec::new();
    for &b in input {
        match out.last_mut() {
            Some((last, n)) if *last == b && *n < u8::MAX => *n += 1,
            _ => out.push((b, 1)),
        }
    }
    out
}

pub fn decode(runs: &[(u8, u8)]) -> Vec<u8> {
    runs.iter()
        .flat_map(|&(b, n)| std::iter::repeat_n(b, n as usize))
        .collect()
}

#[cfg(test)]
mod tests {
    use super::*;
    use proptest::prelude::*;

    proptest! {
        // Each test runs 256 random cases by default; raise it for this block.
        #![proptest_config(ProptestConfig::with_cases(1000))]

        #[test]
        fn roundtrip(data in prop::collection::vec(any::<u8>(), 0..1000)) {
            // Property: decoding an encoding gives back the original.
            prop_assert_eq!(decode(&encode(&data)), data);
        }

        #[test]
        fn runs_are_never_empty_or_adjacent_duplicates(data in prop::collection::vec(0u8..4, 0..500)) {
            // Small alphabet (0..4) makes long runs and repeats likely.
            let runs = encode(&data);
            for w in runs.windows(2) {
                // Equal neighbours are only allowed when the first run is full.
                prop_assert!(w[0].0 != w[1].0 || w[0].1 == u8::MAX);
            }
            prop_assert!(runs.iter().all(|&(_, n)| n > 0));
        }
    }
}

Note the *n < u8::MAX guard. Without it, a run of 256 identical bytes overflows the counter. That is exactly the kind of bug an example-based suite misses and a property test finds within seconds.

Strategies describe how to generate values. They compose:

use proptest::prelude::*;

#[derive(Debug, Clone)]
enum Cmd { Push(i32), Pop }

// A strategy for one command, then for a whole program of commands.
fn cmd() -> impl Strategy<Value = Cmd> {
    prop_oneof![
        any::<i32>().prop_map(Cmd::Push),   // transform generated values
        Just(Cmd::Pop),                     // a constant
    ]
}

proptest! {
    #[test]
    fn stack_matches_vec_model(cmds in prop::collection::vec(cmd(), 0..100)) {
        // Model-based testing: compare the real type against a trusted "oracle".
        let mut real = Stack::new();
        let mut model: Vec<i32> = Vec::new();
        for c in cmds {
            match c {
                Cmd::Push(x) => { real.push(x); model.push(x); }
                Cmd::Pop => prop_assert_eq!(real.pop(), model.pop()),
            }
        }
    }
}

Common shapes of properties:

Property	Example
Round trip	decode(encode(x)) == x; parse(format(x)) == x
Oracle / model	Optimized implementation agrees with a slow, obviously correct one
Invariant	Output is sorted; a tree stays balanced; length is preserved
Idempotence	normalize(normalize(x)) == normalize(x)
Metamorphic	search(q) results are a superset of search(q + " extra_term")
No crash	Arbitrary input returns Ok or Err, never panics

When a case fails, proptest records the seed in a proptest-regressions/<module>.txt file. Commit it: every later run replays those cases first, so a bug once found stays covered.

Key Takeaways

• Property tests state rules that hold for all inputs; the framework generates inputs and shrinks failures to a minimal counterexample.
• Round trips, oracles, and invariants are the most productive properties.
• Strategies compose with prop_map, prop_oneof!, and collection strategies; constrain them to valid inputs rather than filtering heavily.
• Commit proptest-regressions/ so past failures are always re-tested.

🧪 Practice

1. Write a property test for a function that reverses the words in a sentence: reversing twice should give the original (after whitespace normalization).
2. Implement a sorted-insert function for Vec<i32> and check it against the oracle "push, then sort".
3. Interview: How does shrinking work, and why does it matter more than the random generation itself when debugging? (Hint: think about what you would do with a failing 800-element input.)

Fuzzing with cargo-fuzz

Property testing generates inputs from a strategy you design. Fuzzing goes further for code that consumes untrusted bytes, such as parsers, decoders, and protocol handlers. A coverage-guided fuzzer watches which branches each input reaches. Inputs that reach new code are kept and mutated further, so over millions of executions it digs into corners no one thought to test. In Rust, safe code cannot corrupt memory, but a fuzzer still finds panics, arithmetic overflow, infinite loops, excessive allocation, and memory bugs in unsafe code.

            +-----------+     mutate      +-------------+
            |  corpus   | --------------> |  new input  |
            | (inputs)  |                 +------+------+
            +-----^-----+                        |
                  | keep if new                  v
                  | coverage            +-----------------+
                  +-------------------- | fuzz target     | -- panic/crash --> artifacts/
                                        | (instrumented)  |
                                        +-----------------+

cargo-fuzz drives LLVM's libFuzzer. It needs a nightly toolchain (it uses sanitizer flags) and works on Linux and macOS.

cargo install cargo-fuzz
cargo fuzz init                  # creates fuzz/ (a separate crate) with a first target
cargo fuzz add parse_config      # adds fuzz/fuzz_targets/parse_config.rs

A fuzz target is a function that receives arbitrary bytes and calls your code:

// fuzz/fuzz_targets/parse_config.rs
#![no_main]                                   // libFuzzer provides main()
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    // Only valid UTF-8 is interesting for a text parser.
    if let Ok(text) = std::str::from_utf8(data) {
        // Must never panic, whatever the input. Errors are fine.
        let _ = kv::Config::parse(text);
    }
});

cargo +nightly fuzz run parse_config                       # runs until a crash or Ctrl-C
cargo +nightly fuzz run parse_config -- -max_total_time=300  # bound it (for CI)

When it finds a crash, libFuzzer saves the input and prints how to reproduce it:

thread '<unnamed>' panicked at src/config.rs:41:27:
attempt to subtract with overflow
==12345== ERROR: libFuzzer: deadly signal
artifact_prefix='fuzz/artifacts/parse_config/'; Test unit written to
    fuzz/artifacts/parse_config/crash-5f1c0a...

Reproduce with:
    cargo fuzz run parse_config fuzz/artifacts/parse_config/crash-5f1c0a...

Minimize test case with:
    cargo fuzz tmin parse_config fuzz/artifacts/parse_config/crash-5f1c0a...

By default cargo-fuzz builds with optimizations and debug assertions, so integer overflow panics instead of wrapping silently. Once fixed, turn the minimized input into an ordinary regression test.

Structured fuzzing. For APIs that take structured values rather than bytes, derive arbitrary::Arbitrary and let the fuzzer build typed inputs from its raw bytes:

#![no_main]
use arbitrary::Arbitrary;
use libfuzzer_sys::fuzz_target;

#[derive(Debug, Arbitrary)]
enum Op { Insert(u16, u32), Remove(u16), Get(u16) }

fuzz_target!(|ops: Vec<Op>| {
    // Differential fuzzing: our map must behave exactly like std's.
    let mut ours = kv::SmallMap::new();
    let mut reference = std::collections::BTreeMap::new();
    for op in ops {
        match op {
            Op::Insert(k, v) => assert_eq!(ours.insert(k, v), reference.insert(k, v)),
            Op::Remove(k) => assert_eq!(ours.remove(&k), reference.remove(&k)),
            Op::Get(k) => assert_eq!(ours.get(&k), reference.get(&k)),
        }
    }
});

	Property testing (proptest)	Fuzzing (cargo-fuzz)
Input generation	Random, from a strategy you define	Mutation guided by code coverage
Runs	Hundreds of cases, inside cargo test	Millions, for minutes to days
Toolchain	Stable	Nightly (sanitizers)
Best for	Logic properties, every commit	Parsers and decoders of untrusted input
Failure output	Shrunk counterexample	Crash file, minimized with tmin

Key Takeaways

• Coverage-guided fuzzers mutate inputs that reach new code, finding panics, overflows, hangs, and unsafe memory bugs.
• cargo fuzz init / add / run manage targets in a separate fuzz/ crate; it requires nightly.
• Crashes are saved under fuzz/artifacts/; minimize with cargo fuzz tmin and keep them as regression tests.
• Derive Arbitrary for structured inputs and compare against a reference implementation (differential fuzzing).

🧪 Practice

1. Write a fuzz target for a function that parses "HH:MM" times. Run it for five minutes and fix anything it finds.
2. Write a structured fuzz target that checks a custom ring buffer against VecDeque.
3. Interview: Safe Rust prevents memory corruption, so why do Rust projects still fuzz their parsers? (Hint: list the failure modes that remain in safe code, and think about what an attacker controls.)

Mocking and Test Doubles

Code that sends email, calls an HTTP API, reads the clock, or writes to a database is hard to test: real dependencies are slow, flaky, costly, or have side effects. A test double stands in for the real dependency during a test, the way a stunt double stands in for an actor.

Languages with runtime reflection can swap dependencies behind your back ("monkeypatching"). Rust cannot, and that is a design advantage: the substitution point, the seam, has to be explicit. The idiomatic seam is a trait. Code depends on the trait, production passes the real implementation, and tests pass a double.

Double	What it does	Example
Dummy	Fills a parameter, never used	NoopLogger
Stub	Returns canned answers	FixedClock(1_700_000_000)
Fake	Working but simplified implementation	In-memory repository backed by a HashMap
Spy	Records calls for later assertions	Mailer that pushes messages into a Vec
Mock	Pre-programmed with expectations, verifies them	mockall mock expecting send exactly once

pub trait Clock {
    fn now_secs(&self) -> u64;
}

pub trait Mailer {
    fn send(&self, to: &str, body: &str) -> Result<(), String>;
}

// The code under test depends on traits, not concrete services.
pub struct Reminder<'a, C: Clock, M: Mailer> {
    pub clock: &'a C,
    pub mailer: &'a M,
}

impl<C: Clock, M: Mailer> Reminder<'_, C, M> {
    pub fn remind_if_due(&self, due: u64, user: &str) -> Result<bool, String> {
        if self.clock.now_secs() < due {
            return Ok(false);
        }
        self.mailer.send(user, "Your task is due")?;
        Ok(true)
    }
}

Hand-written doubles are often all you need:

#[cfg(test)]
mod tests {
    use super::*;
    use std::cell::RefCell;

    struct FixedClock(u64);                              // stub
    impl Clock for FixedClock {
        fn now_secs(&self) -> u64 { self.0 }
    }

    #[derive(Default)]
    struct SpyMailer { sent: RefCell<Vec<(String, String)>> }   // spy
    impl Mailer for SpyMailer {
        fn send(&self, to: &str, body: &str) -> Result<(), String> {
            // RefCell: `send` takes &self, but the spy must record (Chapter 9).
            self.sent.borrow_mut().push((to.into(), body.into()));
            Ok(())
        }
    }

    #[test]
    fn sends_when_due() {
        let mailer = SpyMailer::default();
        let r = Reminder { clock: &FixedClock(100), mailer: &mailer };
        assert_eq!(r.remind_if_due(100, "ana"), Ok(true));
        assert_eq!(mailer.sent.borrow().len(), 1);
    }

    #[test]
    fn stays_quiet_before_due() {
        let mailer = SpyMailer::default();
        let r = Reminder { clock: &FixedClock(99), mailer: &mailer };
        assert_eq!(r.remind_if_due(100, "ana"), Ok(false));
        assert!(mailer.sent.borrow().is_empty());
    }
}

For traits with many methods, or when you need call-count and argument expectations, mockall generates mocks from the trait definition:

// The mock type MockMailer is generated only in test builds.
#[cfg_attr(test, mockall::automock)]
pub trait Mailer {
    fn send(&self, to: &str, body: &str) -> Result<(), String>;
}

#[test]
fn propagates_mailer_errors() {
    let mut mailer = MockMailer::new();
    mailer.expect_send()
        .withf(|to, _body| to == "bo")              // argument matcher
        .times(1)                                   // verified when the mock is dropped
        .returning(|_, _| Err("smtp down".into())); // canned response

    let r = Reminder { clock: &FixedClock(500), mailer: &mailer };
    assert_eq!(r.remind_if_due(100, "bo"), Err("smtp down".to_string()));
}

Other seams worth knowing:

• HTTP: point the client at a local mock server (wiremock, httpmock) instead of mocking the client type. This also tests serialization and status handling.
• Time in async code: #[tokio::test(start_paused = true)] freezes Tokio's clock and auto-advances it, so a 30-second timeout test finishes instantly.
• Filesystem: use a real temporary directory (tempfile::tempdir()); it is fast and avoids mocking std::fs.

Over-mocking has a cost. Tests that assert every call couple themselves to the implementation and break on harmless refactors. Prefer fakes and real lightweight dependencies, and use strict mocks only where the interaction is the behaviour, such as "the payment API is called exactly once".

Key Takeaways

• Rust has no monkeypatching; make dependencies explicit through traits (generic parameters or dyn Trait) and substitute doubles in tests.
• Hand-written stubs, fakes, and spies cover most needs; mockall generates mocks with argument matchers and call-count verification.
• Prefer real lightweight substitutes (temp dirs, local HTTP servers, paused clocks) over mocking library types.
• Mock interactions only when the interaction itself is the contract.

🧪 Practice

1. Refactor a function that calls std::time::SystemTime::now() directly so that it receives a Clock, and test both branches of its time-dependent logic.
2. Write an in-memory fake for a UserRepository trait (get, insert, delete) and use it to test a service that registers users.
3. Interview: What is the difference between a stub and a mock, and why can heavy use of mocks make a codebase harder to refactor? (Hint: think about whether a test checks outcomes or interactions.)

Snapshot Testing

Some outputs are large and structured: a rendered error message, a pretty-printed AST, a JSON API response, a CLI's help text. Writing assert_eq! against a hand-typed 40-line string is tedious, and updating it after an intentional change is worse. Snapshot testing records the output once, stores it alongside the tests, and on later runs compares new output against the stored snapshot. When output changes, you review the diff and either accept it or fix the code.

The standard crate is insta, paired with the cargo-insta CLI:

[dev-dependencies]
insta = { version = "1", features = ["json", "redactions"] }

cargo install cargo-insta

#[derive(Debug, serde::Serialize)]
struct Report { id: u64, created: String, warnings: Vec<String> }

#[test]
fn renders_error_message() {
    let err = kv::Config::parse("port = abc").unwrap_err();
    // Stored in src/snapshots/<crate>__tests__renders_error_message.snap
    insta::assert_snapshot!(err.to_string());
}

#[test]
fn ast_shape() {
    // Snapshot of the Debug representation (pretty-printed).
    insta::assert_debug_snapshot!(kv::parse_kv("a = b"));
}

#[test]
fn inline_snapshot() {
    // Inline: the expected value lives in the source; cargo-insta rewrites it.
    insta::assert_snapshot!(format!("{:?}", kv::encode(b"aaab")), @"[(97, 3), (98, 1)]");
}

#[test]
fn report_json() {
    let r = Report { id: 7, created: "2026-10-13T09:00:00Z".into(), warnings: vec![] };
    // Redactions replace volatile fields so the snapshot is stable.
    insta::assert_json_snapshot!(r, { ".created" => "[timestamp]" });
}

The workflow:

1. Write the test with no snapshot yet  -> test fails, writes a .snap.new file
2. cargo insta review                   -> shows each new/changed snapshot as a diff
                                           a = accept, r = reject, s = skip
3. Commit the accepted .snap files      -> they are the expected output from now on
4. Later change alters output           -> test fails with a colored diff
5. Intentional? cargo insta review      -> accept. Bug? fix the code.

cargo insta test --review runs the tests and opens the review in one step. In CI (detected through the CI environment variable), insta does not write pending snapshot files; it simply fails on mismatch.

Use snapshots for	Avoid snapshots for
Error and diagnostic messages	Small values where assert_eq! is clearer
Serialized formats (JSON, YAML, TOML)	Output containing timestamps or random IDs (unless redacted)
Pretty-printed ASTs and IRs	Checking a single important field among many
CLI help and output	Anything where reviewers will accept diffs without reading them

The main danger is "snapshot fatigue": accepting changed snapshots without reading them turns the test into a record of whatever the code does, bugs included. Keep snapshots small and focused so diffs stay reviewable. Related tools include expect-test (inline snapshots with UPDATE_EXPECT=1) and trybuild, which snapshots compiler errors and is the standard way to test procedural macros (Chapter 13).

Key Takeaways

• Snapshot tests store expected output in files (or inline) and fail with a diff when output changes.
• cargo insta review turns intentional changes into a quick, reviewable accept/reject step.
• Use redactions for timestamps and IDs so snapshots are deterministic.
• Snapshots only protect you if humans actually read the diffs; keep them focused.

🧪 Practice

1. Add a snapshot test for the Debug output of a nested struct, change one field name, and walk through cargo insta review.
2. Snapshot a JSON response that includes a UUID and a timestamp, using redactions to keep it stable.
3. Interview: When would you choose a snapshot test over a set of targeted assert_eq! checks, and what process prevents snapshots from silently blessing regressions? (Hint: think about output size and code review.)

Code Coverage

How do you know which code your tests never exercise? Code coverage instruments the program to record which lines, regions, and branches execute during the test run, then reports the gaps. It is a flashlight for finding untested code, not a score that proves correctness.

Rust has source-based coverage built into the compiler (-C instrument-coverage), which uses LLVM to map counters precisely back to source regions, including individual sub-expressions. The convenient front end is cargo-llvm-cov:

rustup component add llvm-tools-preview
cargo install cargo-llvm-cov

cargo llvm-cov                              # run tests, print a per-file summary
cargo llvm-cov --open                       # HTML report with highlighted source
cargo llvm-cov --lcov --output-path lcov.info   # for Codecov, Coveralls, IDE plugins
cargo llvm-cov --fail-under-lines 80        # fail CI if line coverage drops below 80%
cargo llvm-cov nextest                      # same, using cargo-nextest as the runner

(abridged: the real report also has function and branch columns)
Filename          Regions  Missed Regions  Cover   Lines  Missed Lines  Cover
-----------------------------------------------------------------------------
src/config.rs         48               6  87.50%     102            9  91.18%
src/parser.rs         95              21  77.89%     188           33  82.45%
-----------------------------------------------------------------------------
TOTAL                143              27  81.12%     290           42  85.52%

What the metrics mean:

Metric	Counts as covered when	Blind spot
Line	Any code on the line ran	if a && b { .. } on one line looks fully covered even if b was never evaluated
Region	Each source region (sub-expression, block) ran	Does not say both outcomes of a condition were seen
Branch	Both the true and false side of each condition ran	Requires nightly at the time of writing (-Z coverage-options=branch)

The alternative tool, cargo-tarpaulin, is popular on Linux CI and supports both its own ptrace-based engine and LLVM coverage.

Reading a report well:

• Uncovered error paths are the most valuable finding: they show which failures have never been exercised, which is where production incidents come from.
• 100% coverage does not mean correct. A test that calls a function and asserts nothing still "covers" it. Mutation testing (cargo-mutants) checks the assertions instead: it changes your code (replaces < with <=, makes a function return Default::default()) and reports mutations that no test caught.
• Ratchet, do not mandate. A rule like "coverage must not decrease" works better than a fixed high target, which encourages assertion-free tests.

Coverage answers:     "Did the tests RUN this code?"
Mutation testing:     "Would the tests NOTICE if this code were wrong?"

Key Takeaways

• -C instrument-coverage provides precise source-based coverage; cargo llvm-cov wraps it with HTML, LCOV, and threshold options.
• Line coverage can hide unevaluated conditions; region and branch coverage are finer grained.
• Use coverage to find untested code, especially error paths, not as proof of correctness.
• Complement coverage with mutation testing (cargo-mutants) to measure whether assertions are meaningful.

🧪 Practice

1. Run cargo llvm-cov --open on a project, find one uncovered error branch, and write a test that reaches it.
2. Write a test that reaches 100% coverage of a small function while asserting nothing useful, then run cargo mutants on it and explain the surviving mutants.
3. Interview: A manager proposes requiring 95% line coverage on every pull request. What would you suggest instead, and why? (Hint: think about what coverage measures, what it incentivizes, and the cost of the last 10%.)

<a id="123-benchmarking-and-profiling"></a>

12.3 Benchmarking and Profiling

Performance work without measurement is guesswork, and naive measurement is often worse than guesswork because it looks authoritative. This section covers how to benchmark Rust code with statistical rigor, how to choose the right scale of benchmark, how to find where time and memory actually go, and how to tell a real improvement from noise.

Criterion Benchmarks

The obvious approach, wrapping code in Instant::now() and printing the elapsed time, is unreliable. One run says nothing about variance. The optimizer may delete code whose result is unused. The first iterations run with cold caches. And when the number changes after an edit, you cannot tell whether the change is real.

Rust's built-in #[bench] attribute is still nightly-only, so the ecosystem standard is Criterion.rs. It warms up the code, runs it many times, applies statistics to the samples, and compares each run with the previous one to tell you whether performance changed significantly.

Setup: benchmarks live in benches/, and each needs harness = false because Criterion supplies its own main:

[dev-dependencies]
criterion = { version = "0.8", features = ["html_reports"] }

[[bench]]
name = "rle"              # benches/rle.rs
harness = false           # use Criterion's main, not libtest's

// benches/rle.rs
use criterion::{criterion_group, criterion_main, Criterion};
use std::hint::black_box;

fn bench_encode(c: &mut Criterion) {
    let input = vec![7u8; 4096];
    c.bench_function("encode 4 KiB", |b| {
        // b.iter runs the closure many times and measures each batch.
        // black_box hides the value from the optimizer so the work is not
        // constant-folded or deleted.
        b.iter(|| kv::encode(black_box(&input)))
    });
}

criterion_group!(benches, bench_encode);   // a named group of benchmark functions
criterion_main!(benches);                  // generates main()

cargo bench                          # all benchmarks (built with the `bench` profile: optimized)
cargo bench --bench rle              # one bench target
cargo bench -- encode                # filter benchmarks by name (a regex)

encode 4 KiB            time:   [9.6130 µs 9.7075 µs 9.8105 µs]
                        change: [−2.1270% −0.5640% +0.9820%] (p = 0.48 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe

The time line shows a confidence interval: lower bound, best estimate, and upper bound for the time per iteration. The change line compares against the previous run, with a p-value saying whether the difference is statistically significant. An HTML report with plots is written to target/criterion/report/index.html.

Parameterized benchmarks and setup. Groups compare implementations across input sizes. iter_batched keeps per-iteration setup, such as cloning input that the benchmark consumes, out of the measurement:

use criterion::{BatchSize, BenchmarkId, Criterion, Throughput};

fn bench_sort(c: &mut Criterion) {
    let mut group = c.benchmark_group("sort");
    for n in [1_000usize, 10_000, 100_000] {
        // Pseudo-random data, generated once per size (outside the timing).
        let data: Vec<u64> = (0..n as u64)
            .map(|x| x.wrapping_mul(0x9E37_79B9_7F4A_7C15))
            .collect();
        group.throughput(Throughput::Elements(n as u64));   // report elements/second too

        group.bench_with_input(BenchmarkId::new("stable", n), &data, |b, d| {
            // setup closure (not timed), then routine (timed) consuming its output
            b.iter_batched(|| d.clone(), |mut v| v.sort(), BatchSize::LargeInput)
        });
        group.bench_with_input(BenchmarkId::new("unstable", n), &data, |b, d| {
            b.iter_batched(|| d.clone(), |mut v| v.sort_unstable(), BatchSize::LargeInput)
        });
    }
    group.finish();
}

This produces IDs like sort/unstable/10000, a throughput line (thrpt: [41.694 Melem/s ...]), and comparison plots across sizes in the HTML report.

Baselines make before/after comparisons across branches explicit:

git switch main     && cargo bench -- --save-baseline main
git switch feature  && cargo bench -- --baseline main     # compare against main's numbers

Other benchmark harnesses:

Tool	Measures	Strength
Criterion.rs	Wall-clock time with statistics	Mature, rich reports, baselines
divan	Wall-clock time	Attribute-based (#[divan::bench]), concise, also tracks allocations
gungraun (formerly iai-callgrind)	Instruction counts and cache simulation under Valgrind	Deterministic numbers, ideal for CI
#[bench] (nightly)	Wall-clock time	Built in, but unstable and minimal

Key Takeaways

• Use Criterion in benches/ with harness = false; it handles warm-up, sampling, statistics, and change detection.
• Wrap inputs (and if needed outputs) in std::hint::black_box so the optimizer cannot delete or precompute the work.
• Use groups, BenchmarkId, and Throughput to compare implementations across sizes; use iter_batched to keep setup out of the measurement.
• Save and compare named baselines for branch-to-branch comparisons.

🧪 Practice

1. Benchmark String::push_str in a loop versus format! versus [..].concat() for joining 100 short strings.
2. Write a benchmark group that compares a linear search and a binary search over sorted vectors of 10, 1,000, and 100,000 elements. Find the crossover point.
3. Interview: Why does a naive benchmark of a pure function sometimes report 0 ns per iteration, and what does black_box do about it? (Hint: think about what the optimizer may assume about unused results and constant inputs.)

Micro vs Macro Benchmarks

A benchmark answers a question, and different questions need different scales. A micro benchmark measures a tiny unit, such as one function or one data structure operation, in isolation and in a tight loop. A macro benchmark measures a whole workload: the CLI processing a real file, the server handling realistic traffic. Confusing the two is one of the most common ways to "optimize" something that does not matter.

	Micro benchmark	Macro benchmark
Scope	One function or operation	End-to-end program or request
Tools	Criterion, divan, gungraun	hyperfine, load generators (oha, wrk, k6), production metrics
Precision	Nanosecond-level, low variance	Millisecond-level, higher variance
Realism	Low: hot caches, same input every time, no I/O contention	High: real data, real I/O, real interactions
Answers	"Is implementation A faster than B?"	"Is the program faster for users?"
Typical trap	Optimizing something that is 1% of the runtime	Too noisy to detect small changes

The traps of micro benchmarks, which all make results look better than reality:

• Warm caches. The same small input is reused, so it sits in L1 cache. In production the data might come from main memory, 50 to 100 times slower.
• Trained branch predictor. Repeating one input lets the CPU predict every branch perfectly.
• Dead code and constant folding. Without black_box, the compiler may compute the answer once, or not at all.
• Unrepresentative inputs. Sorting already-sorted data, or hashing 8-byte keys when production keys are 200 bytes.

Amdahl's law explains why the macro view matters. If a function accounts for a fraction p of total runtime and you make it s times faster, the overall speedup is 1 / ((1 - p) + p / s):

Function is 5% of runtime, made 10x faster:   1 / (0.95 + 0.005) = 1.047  -> 4.7% faster overall
Function is 60% of runtime, made 2x faster:   1 / (0.40 + 0.30)  = 1.43   -> 43% faster overall

A practical workflow combines both scales:

1. Macro: measure the real workload        -> "the import takes 4.2 s"
2. Profile it (next topic)                 -> "62% is in parse_record"
3. Micro: benchmark parse_record alone     -> iterate quickly on alternatives
4. Macro again: confirm the user-visible   -> "the import now takes 2.6 s"
   improvement

For command-line programs, hyperfine is the standard macro tool. It runs commands repeatedly, handles warm-up, and reports mean, deviation, and relative speed:

cargo build --release
hyperfine --warmup 3 \
    './old-kv big.conf' \
    './target/release/kv big.conf'

Benchmark 1: ./old-kv big.conf
  Time (mean ± σ):     412.3 ms ±   6.1 ms    [User: 380.2 ms, System: 30.9 ms]
  Range (min … max):   404.8 ms … 423.0 ms    10 runs

Benchmark 2: ./target/release/kv big.conf
  Time (mean ± σ):     314.9 ms ±   4.4 ms    [User: 284.0 ms, System: 30.1 ms]
  Range (min … max):   309.2 ms … 322.7 ms    10 runs

Summary
  ./target/release/kv big.conf ran
    1.31 ± 0.03 times faster than ./old-kv big.conf

Key Takeaways

• Micro benchmarks compare implementations precisely; macro benchmarks tell you whether users benefit.
• Micro results flatter code through warm caches, trained branch predictors, and unrealistic inputs.
• Amdahl's law: overall speedup is bounded by the fraction of runtime you improved.
• Measure the whole workload, profile, micro-optimize the hotspot, then confirm with the whole workload again.

🧪 Practice

1. Write a micro benchmark for a JSON-parsing function using one small input, then a macro benchmark that parses a 50 MB file with hyperfine. Compare how a change to the function affects each.
2. Benchmark a HashMap lookup with 10 keys and with 10 million keys. Explain the difference in time per lookup.
3. Interview: A colleague's PR makes a hashing function 3x faster in a Criterion benchmark, but the service's p99 latency does not move. Give three possible explanations. (Hint: Amdahl, caches, and what dominates a request's latency.)

Flamegraphs and perf

Benchmarks tell you how long something takes; a profiler tells you where the time goes. Intuition about hotspots is wrong surprisingly often: the slow part is frequently an innocent-looking clone, a hash function, or an allocation inside a loop.

A sampling profiler interrupts the program many times per second (for example 1,000 Hz) and records the call stack each time. Functions that appear in many samples are where the program spends its time. Overhead is low and no code changes are needed, but the binary needs debug symbols to map addresses back to function names.

# Cargo.toml: a dedicated profile (Chapter 8) - release speed plus symbols
[profile.profiling]
inherits = "release"
debug = true

On Linux, the standard tool is perf:

cargo build --profile profiling
perf record -g --call-graph dwarf ./target/profiling/kv big.conf   # sample with call stacks
perf report                                                         # interactive breakdown

  Children      Self  Symbol
-   71.20%     0.40%  kv::run
   - 70.80% kv::run
      - 52.10% kv::parse_record
           31.70% alloc::string::String::push_str
           12.20% core::str::<impl str>::trim
      - 18.30% kv::index::build
           15.10% hashbrown::map::HashMap::insert

"Self" is time spent in the function's own code; "Children" includes everything it calls.

Flame graphs turn thousands of stack samples into one picture. Install the Cargo wrapper with cargo install flamegraph; it uses perf on Linux and DTrace on macOS:

cargo flamegraph --profile profiling --bin kv -- big.conf     # writes flamegraph.svg
cargo flamegraph --bench rle -- --bench --profile-time 10     # profile a Criterion benchmark

              [push_str ]   [ trim ]  [ HashMap::insert ]
         [        parse_record       ][    index::build     ][fmt]
         [                       kv::run                           ][ read_file ]
         [                             main                                     ]
         <------------------------- 100% of samples --------------------------->

How to read it:

Feature	Meaning
Y axis	Stack depth: each box's caller is directly below it
Box width	Fraction of samples containing that function, i.e. its share of time
X axis	Not time; boxes are sorted alphabetically to merge identical stacks
Wide box with nothing on top	Time spent in that function's own code: a hotspot
Wide "towers"	Expensive call paths worth investigating first
Color	Usually random warm tones, with no meaning

Practical notes:

• samply (samply record ./target/profiling/kv big.conf) works on Linux, macOS, and Windows and opens the result in the Firefox Profiler UI, with a timeline, call tree, and flame graph.
• Inlining hides functions. Inlined callees are attributed to their caller. If a stack looks implausibly flat, that is usually why.
• Broken stacks often mean the unwinder could not walk the frames. --call-graph dwarf or building with RUSTFLAGS="-C force-frame-pointers=yes" helps.
• Permissions. On Linux, perf may require lowering kernel.perf_event_paranoid (sudo sysctl kernel.perf_event_paranoid=1).
• Profile release builds. Debug builds have entirely different hotspots, such as un-inlined iterator adapters.

Key Takeaways

• Sampling profilers record call stacks periodically; they need debug symbols but not code changes.
• Use a profiling profile (inherits = "release", debug = true) so profiles reflect optimized code with readable names.
• In a flame graph, width is time and height is call depth; the x axis is not chronological.
• perf plus cargo flamegraph on Linux, or samply everywhere, covers most needs.

🧪 Practice

1. Profile a program that counts word frequencies in a large text file with cargo flamegraph. Identify the widest leaf box and optimize it.
2. Profile the same program in a debug build and compare the flame graphs. Which functions appear only in the debug profile, and why?
3. Interview: A flame graph shows memcpy as 25% of samples, called from many places. How would you find out which Rust code is responsible, and what are common causes in Rust? (Hint: look below memcpy in the stacks; think about large values moved by value and clone.)

Allocation Profiling

A CPU profile may show time spread thinly across malloc, free, memcpy, and page faults, with no single hotspot. That is often the signature of excessive allocation. Each heap allocation costs a call into the allocator, possible synchronization, cache misses on fresh memory, and later a free. In a hot loop, a stray to_string() can cost more than the actual work.

Allocation profilers answer: how many allocations, how many bytes, peak memory, and which call stacks allocate.

Tool	How it works	Platforms
dhat crate	Replaces the global allocator in your binary; writes a JSON profile	All
heaptrack	Intercepts malloc/free of an unmodified binary; GUI analysis	Linux
Valgrind DHAT (valgrind --tool=dhat)	Runs the binary on a simulated CPU; slow but detailed	Linux
Instruments (Allocations)	System profiler	macOS
Counting allocator (hand-written)	Wraps System and counts calls	All; good for tests

dhat-rs is the most portable option. It is enabled through a feature so it never ships in normal builds:

[dependencies]
dhat = "0.3"

[features]
dhat-heap = []          # cargo run --release --features dhat-heap

[profile.release]
debug = 1               # line tables, so stacks have file:line information

#[cfg(feature = "dhat-heap")]
#[global_allocator]
static ALLOC: dhat::Alloc = dhat::Alloc;      // route every allocation through dhat

fn main() {
    #[cfg(feature = "dhat-heap")]
    let _profiler = dhat::Profiler::new_heap();  // writes dhat-heap.json when dropped

    let mut words = Vec::new();
    for i in 0..1000 {
        words.push(format!("word{i}"));          // one heap allocation per String
    }
}

dhat: Total:     57,056 bytes in 1,009 blocks
dhat: At t-gmax: 32,576 bytes in 1,001 blocks
dhat: At t-end:  0 bytes in 0 blocks
dhat: The data has been saved to dhat-heap.json, and is viewable with dhat/dh_view.html

Total is every allocation made; t-gmax is the moment of peak heap usage; t-end is what was still allocated at exit. Open the JSON file in the DHAT viewer (dh_view.html, shipped with Valgrind and hosted online) to see the allocating call stacks sorted by bytes or block count. Here the 1,000 Strings plus nine buffers for the growing Vec (capacity 4, 8, 16, ..., 1024) explain the 1,009 blocks. At the peak, 1,001 blocks are live: every string plus the final Vec buffer. Vec::with_capacity(1000) would remove eight of the allocations.

Allocation regression tests. A counting global allocator can assert that a hot function does not allocate at all. Put it in its own integration test file, since it replaces the allocator for that whole test binary:

// tests/allocations.rs
use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::atomic::{AtomicUsize, Ordering};

struct Counting;
static ALLOCATIONS: AtomicUsize = AtomicUsize::new(0);

unsafe impl GlobalAlloc for Counting {
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        ALLOCATIONS.fetch_add(1, Ordering::Relaxed);
        unsafe { System.alloc(layout) }          // delegate to the system allocator
    }
    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
        unsafe { System.dealloc(ptr, layout) }
    }
}

#[global_allocator]
static GLOBAL: Counting = Counting;

fn allocations_during<R>(f: impl FnOnce() -> R) -> (R, usize) {
    let before = ALLOCATIONS.load(Ordering::Relaxed);
    let r = f();
    (r, ALLOCATIONS.load(Ordering::Relaxed) - before)
}

// Keep this the only test in the file: parallel tests would allocate
// concurrently and pollute the global counter.
#[test]
fn parse_kv_is_allocation_free() {
    let (result, n) = allocations_during(|| kv::parse_kv("port = 8080"));
    assert_eq!(result, Some(("port", "8080")));
    assert_eq!(n, 0, "parse_kv should borrow from its input, not allocate");
}

Common sources of avoidable allocation, and their fixes:

Pattern	Fix
s.to_string() / format! only to compare or look up	Borrow: &str, Cow<str>, HashMap::get with &str keys
Vec::new() then many push calls with a known size	Vec::with_capacity(n)
New Vec/String per loop iteration	Hoist it out of the loop and clear() it each time
.collect::<Vec<_>>() just to iterate again	Keep it as an iterator chain
clone() to satisfy the borrow checker	Restructure borrows, or share with Rc/Arc
Many tiny short-lived objects	Small-buffer types (smallvec) or an arena (bumpalo)

Key Takeaways

• Allocation-heavy code shows up as time spread across allocator functions rather than one hotspot.
• dhat reports total, peak (t-gmax), and leaked (t-end) memory with allocating stacks; gate it behind a feature.
• A counting #[global_allocator] in a dedicated test file turns "this does not allocate" into an enforced regression test.
• Most fixes are about borrowing instead of owning, pre-sizing, and reusing buffers.

🧪 Practice

1. Profile a log-processing program with dhat, find the call site with the most blocks, and reduce its allocations by at least 90%.
2. Write an allocation regression test asserting that a tokenizer returning &str slices performs zero allocations for a 1 KB input.
3. Interview: Why can reducing allocations speed up a multi-threaded program more than a single-threaded one? (Hint: think about what a general-purpose allocator must coordinate between threads, and about cache lines.)

Interpreting Benchmark Noise

Run the same benchmark twice without changing a line of code, and you will get two different numbers. Sometimes the difference is dramatic. Here is real output from running the encode 4 KiB benchmark above twice in a row on a laptop, with no code change in between:

first run:   encode 4 KiB   time:   [19.435 µs 20.099 µs 20.808 µs]

second run:  encode 4 KiB   time:   [9.6130 µs 9.7075 µs 9.8105 µs]
                            change: [−52.127% −50.564% −48.982%] (p = 0.00 < 0.05)
                            Performance has improved.

Criterion reported a statistically significant 50% improvement for identical code. The statistics were not wrong: the machine changed between runs (CPU frequency, background load, thermal state). Statistics can only tell you that two sets of samples differ, not why. Interpreting benchmarks is mostly about controlling and recognizing those other causes.

Sources of noise:

Source	Effect	Mitigation
CPU frequency scaling and turbo boost	Speed varies with temperature and load, often by 2x	Disable turbo, fix the governor (cpupower frequency-set -g performance), let the machine cool
Other processes	Steal CPU time and cache	Close browsers and IDE indexers; benchmark on an idle machine
Virtual machines and cloud CI	Noisy neighbours, shared caches	Use instruction counts (gungraun) in CI, wall clock only on dedicated hardware
Code layout and alignment	Unrelated changes shift hot loops across cache lines, ±5% or more	Distrust small changes; confirm with several builds or a macro benchmark
Memory layout (ASLR, environment size)	Different stack/heap addresses change cache behaviour	Repeat runs; compare distributions, not single numbers
Thread scheduling and migration	Cold caches after moving cores	Pin the process: taskset -c 2 cargo bench

What Criterion's output tells you, and what it does not:

time:   [9.6130 µs 9.7075 µs 9.8105 µs]
         ^ lower      ^ estimate  ^ upper    95% confidence interval of the mean

change: [−2.12% −0.56% +0.98%] (p = 0.48 > 0.05)
         interval includes 0 and p > 0.05     -> "No change in performance detected."

Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild                         -> some samples were slow:
  2 (2.00%) high severe                          interrupts, other processes

• Confidence interval width is your precision. If the interval is ±10%, you cannot detect a 3% change, however many times you rerun.
• p-value below the significance level (default 0.05) means the difference is unlikely to be random sampling variation. It does not mean the difference is large or that your code caused it.
• Noise threshold (default 1%): changes smaller than this are reported as "within noise threshold" even when statistically significant.
• Outliers, especially many "high severe" ones, indicate interference during measurement. A few percent is normal; 20% suggests a noisy machine.

A real improvement:          Noise:

  before   |  ####              before   |   ######
  after    |####                after    |  ######
           +-----------> time            +-----------> time
  distributions separated      distributions overlap heavily

A disciplined procedure:

1. Stabilize the machine before trusting any number: plugged in, idle, turbo disabled if possible.
2. Measure the noise floor: run the unchanged benchmark several times and see how much it varies. Changes smaller than that range are not evidence.
3. Interleave A/B comparisons (baseline, change, baseline, change) instead of measuring each once, so drift affects both equally.
4. Prefer deterministic metrics in CI: instruction counts from gungraun vary by fractions of a percent where wall-clock time varies by 10% or more.
5. Confirm with a macro benchmark. A 3% micro-improvement that disappears end to end was probably layout noise or irrelevant (Amdahl).
6. Report distributions, not single numbers: median and spread, or Criterion's interval.

Criterion's statistical knobs can be changed when a benchmark is noisy by nature:

use criterion::Criterion;
use std::time::Duration;

fn config() -> Criterion {
    Criterion::default()
        .sample_size(200)                         // default 100
        .measurement_time(Duration::from_secs(10)) // default 5 s
        .warm_up_time(Duration::from_secs(3))      // default 3 s
        .noise_threshold(0.03)                     // ignore changes under 3% (default 1%)
        .significance_level(0.01)                  // stricter than the default 0.05
}

criterion::criterion_group! {
    name = benches;
    config = config();
    targets = bench_encode
}

Key Takeaways

• Identical code can produce significantly different timings; machine state (frequency, load, layout) is often the cause, not your change.
• Read the confidence interval as your precision, the p-value as "probably not random", and the noise threshold as "too small to care".
• Establish the noise floor, interleave A/B runs, and pin or quiet the machine before trusting small differences.
• In shared CI, prefer instruction-count benchmarks; confirm micro-level wins with macro benchmarks.

🧪 Practice

1. Run one Criterion benchmark ten times without changing code and record the reported changes. What is the noise floor on your machine?
2. Repeat the experiment while compiling a large project in the background, and then with the process pinned to one core with taskset. Compare the outlier counts.
3. Interview: A benchmark in CI reports a 4% regression on a pull request that only changed documentation. How do you explain this to the team, and how would you redesign the CI benchmarking setup? (Hint: noise floor, shared runners, and deterministic metrics.)

<a id="124-documentation"></a>

12.4 Documentation

In Rust, documentation is part of the product: it ships on docs.rs for every published crate, and its examples are tested. Chapter 1 introduced doc comment syntax and Chapter 8 the cargo doc tool. This section covers how to write documentation well: the conventions readers expect, linking between items, runnable examples, the crate's front page, and configuring how docs.rs builds it.

Doc Comment Conventions

Readers come to API documentation with specific questions: What does this do? How do I call it? What can go wrong? Rust's ecosystem has settled on a shared structure that answers them in a predictable order, so a reader who knows the standard library's docs can navigate any crate's docs.

The anatomy of a well-documented item:

/// Parses a duration such as `"1h30m"` or `"45s"` into a [`Duration`].
///
/// Units are `h`, `m`, and `s`, and may appear at most once each, in that
/// order. Whitespace is not allowed. Use [`format_duration`] for the inverse.
///
/// # Examples
///
/// ```
/// use std::time::Duration;
/// use timefmt::parse_duration;
///
/// assert_eq!(parse_duration("1h30m")?, Duration::from_secs(5400));
/// assert_eq!(parse_duration("45s")?, Duration::from_secs(45));
/// # Ok::<(), timefmt::ParseError>(())
/// ```
///
/// # Errors
///
/// Returns [`ParseError::Empty`] if `s` is empty, and
/// [`ParseError::BadUnit`] if it contains a unit other than `h`, `m`, or `s`.
///
/// # Panics
///
/// Panics if the total number of seconds overflows `u64`.
pub fn parse_duration(s: &str) -> Result<Duration, ParseError> {
    // ...
}

1. Summary line. One sentence, ending with a period, saying what the item does. rustdoc shows this first paragraph alone in module listings and search results, so it must make sense on its own.
2. Details. Behaviour, edge cases, relationships to other items.
3. Standard sections, as # headings in this conventional order:

Section	When to include	Enforced by
# Examples	Every public item where usage is not utterly obvious	Code review (rustdoc's missing_doc_code_examples lint is nightly-only)
# Errors	Functions returning Result: which errors, and when	clippy::missing_errors_doc (pedantic)
# Panics	Functions that can panic on some input	clippy::missing_panics_doc (pedantic)
# Safety	Every unsafe fn and unsafe trait: the caller's obligations	clippy::missing_safety_doc (warn by default)
Optional: # Performance, # Platform-specific behavior	Complexity guarantees, OS differences	Convention

Style conventions, from the Rust API Guidelines and the standard library:

• Use the third person, present tense: "Returns the number of elements", not "Return..." or "This function will return...".
• Put identifiers in backticks and link them (next topic).
• Document what and why, not how. Implementation details belong in // comments inside the body.
• Document every public item: types, fields, enum variants, trait methods, modules (//! at the top of the module file), and constants.
• In examples, prefer ? over unwrap(). Readers copy examples, so they should model good error handling.

A weak doc versus a strong one:

// Weak: restates the signature and hides what a reader needs to know.
/// This function gets the user.
pub fn get_user(id: u64) -> Option<User> { todo!() }

// Strong: behaviour, edge cases, and cost.
/// Returns the user with the given `id`, or `None` if no such user exists.
///
/// Results are served from an in-memory cache; a cache miss reads from the
/// database and may take several milliseconds.
pub fn get_user_cached(id: u64) -> Option<User> { todo!() }

/// is sugar for the #[doc = "..."] attribute. That matters when documentation must be computed, for example in macro-generated code:

macro_rules! int_parser {
    ($name:ident, $t:ty) => {
        #[doc = concat!("Parses a decimal `", stringify!($t), "`.")]   // per-type docs
        pub fn $name(s: &str) -> Option<$t> { s.parse().ok() }
    };
}
int_parser!(parse_u8, u8);       // documented as: Parses a decimal `u8`.
int_parser!(parse_i64, i64);

To make missing documentation a build error in a library, combine the compiler lint with Clippy's section lints:

// lib.rs
#![warn(missing_docs)]                 // every public item needs a doc comment
#![warn(clippy::missing_errors_doc, clippy::missing_panics_doc)]

Key Takeaways

• Start with a one-sentence summary that stands alone in search results, then details, then the standard sections.
• Use # Examples, # Errors, # Panics, and # Safety (mandatory for unsafe items) consistently; Clippy can enforce them.
• Write in the third person present tense, document fields and variants, and use ? in examples.
• /// is the #[doc] attribute, which lets macros generate documentation.

🧪 Practice

1. Document a function that returns Result and can panic, with all four standard sections. Run cargo clippy -- -W clippy::pedantic before and after.
2. Find a public item in one of your projects whose docs merely restate its name, and rewrite them to explain behaviour and edge cases.
3. Interview: Why does every unsafe fn need a # Safety section, and what exactly should it contain? (Hint: think about who must uphold the invariants and what happens if they do not; see Chapter 14.)

Intra-Doc Links

Documentation that mentions Duration or ParseError::BadUnit should let the reader click through to them. Writing raw HTML links (../struct.Foo.html) is brittle: the paths change when items move, and they cannot point into other crates reliably. Intra-doc links let you link to items by their Rust path. rustdoc resolves the path like the compiler would and generates the correct URL, even into dependencies' docs on docs.rs.

use std::collections::HashMap;
use std::time::Duration;

/// A cache of parsed values, keyed by name.
///
/// - Built on [`HashMap`] (resolved through the `use` above)
/// - Create one with [`Cache::new`], add entries with [`insert`](Cache::insert)
/// - Entries expire after a [`Duration`]; see [`Self::ttl`]
/// - Errors are reported as [`crate::error::CacheError`]
/// - Uses the [`parse_kv()`] function and the [`log!`] macro
/// - Free text pointing anywhere: see [the module docs][mod-docs]
///
/// [mod-docs]: crate::cache
pub struct Cache {
    ttl: Duration,
    entries: HashMap<String, String>,
}

Syntax	Links to
[`Name`]	The item Name, resolved in the current scope
[`path::to::Item`]	A full or relative path (crate::, super::, self:: work)
[`Type::method`]	A method or associated item
[`Self::method`]	Associated item of the documented type (inside impl and type docs)
[text](path::Item)	Custom link text with a Rust path as target
[text][label] + [label]: path::Item	Reference-style link, keeps long paths out of prose
[`f()`] / [`m!`]	Disambiguates a function / macro
[`struct@Foo`], [`fn@foo`], [`mod@foo`], [`macro@foo`], [`prim@str`]	Explicit disambiguators when names clash

Disambiguation is needed when one name refers to several things, such as a struct Point and a function point, or a module and a macro with the same name. rustdoc reports an ambiguity warning and suggests the fix.

Links are checked at doc-build time. A link that does not resolve is a warning by default, which CI should turn into an error:

$ RUSTDOCFLAGS="-D warnings" cargo doc --no-deps
error: unresolved link to `Missing`
   --> src/lib.rs:195:7
    |
195 | /// [`Missing`]
    |       ^^^^^^^ no item named `Missing` in scope
    |
    = help: to escape `[` and `]` characters, add '\' before them like `\[` or `\]`
    = note: `-D rustdoc::broken-intra-doc-links` implied by `-D warnings`

This turns renames into doc-build failures, so docs cannot silently point at items that no longer exist. Two subtleties:

• Links resolve in the scope of the documented item, so a use statement that exists only for the docs may trigger an "unused import" warning. Writing the full path in the link ([`std::time::Duration`]) avoids it.
• Linking from public docs to a private item triggers rustdoc::private_intra_doc_links, because readers of the public docs cannot follow the link.

Key Takeaways

• Intra-doc links ([`Path`]) link items by Rust path; rustdoc computes the URL, including into dependencies.
• Use Self::, crate::, custom text [text](Path), and reference-style links for readable prose.
• Disambiguate with fn@, struct@, macro@, () or ! when names clash.
• Build docs with -D warnings in CI so broken links fail the build.

🧪 Practice

1. Convert every backticked type name in a module's docs into an intra-doc link, then rename one of the types and observe the doc build fail.
2. Create a struct Config and a function config() in the same module and write a doc comment linking to each unambiguously.
3. Interview: Why are intra-doc links preferable to hard-coded HTML links in a library that will be published on docs.rs? (Hint: think about renames, re-exports, and links into other crates' versions.)

Examples Directory

Doc examples show how to use one item. Real users also need to see how the pieces fit together: build a client, configure it, handle errors, shut down cleanly. That is the job of the examples/ directory (Chapter 8 introduced it as a Cargo target kind): complete, runnable programs that demonstrate realistic workflows.

kv/
├── Cargo.toml
├── src/lib.rs
└── examples/
    ├── basic.rs               cargo run --example basic
    ├── watch_file.rs          cargo run --example watch_file --features watch
    └── http_server/           multi-file example
        ├── main.rs            cargo run --example http_server
        └── routes.rs

// examples/basic.rs
//! Loads a config file and prints every key.
//!
//! Run with:
//!     cargo run --example basic -- path/to/app.conf

use std::error::Error;

fn main() -> Result<(), Box<dyn Error>> {
    let path = std::env::args().nth(1).ok_or("usage: basic <file>")?;
    let config = kv::Config::load(&path)?;       // public API only, like a user
    for (key, value) in config.iter() {
        println!("{key} = {value}");
    }
    Ok(())
}

# Examples can use [dev-dependencies] (e.g. tokio, clap) without
# forcing them on users of the library.
[dev-dependencies]
tokio = { version = "1", features = ["full"] }

[[example]]
name = "watch_file"
required-features = ["watch"]     # skipped unless the feature is enabled

How the three kinds of runnable documentation compare:

	Doc examples	examples/	Integration tests
Audience	API readers	New users, evaluators	Maintainers
Scope	One item	A complete workflow	A behaviour, with assertions
Verified by cargo test	Compiled and run	Compiled only	Compiled and run
Can use dev-dependencies	Yes	Yes	Yes
Typical length	3 to 15 lines	30 to 200 lines	Varies

Guidelines for good examples:

• Start each file with a //! comment saying what it demonstrates and the exact command to run it.
• Order them by difficulty and mention them from the crate docs ("see examples/basic.rs"). Readers do not find examples on their own.
• Keep them compiling in CI. cargo test builds them, but examples with required-features are skipped unless the features are enabled, so CI should also run cargo build --examples --all-features.
• Use realistic error handling (?, main returning Result) because examples are copied into production code.

docs.rs can also show scraped examples: rustdoc finds calls to each function in your examples/ and displays them on that function's page. The feature is unstable and enabled through docs.rs metadata (next topics): cargo-args = ["-Zunstable-options", "-Zrustdoc-scrape-examples"].

Key Takeaways

• examples/ holds complete programs that show workflows; doc examples show individual items.
• Examples use only the public API and may use [dev-dependencies].
• cargo test compiles examples but does not run them; gate optional ones with required-features and build them all in CI.
• Document each example's purpose and run command at the top, and link to examples from the crate docs.

🧪 Practice

1. Write an examples/quickstart.rs for a library you maintain that goes from configuration to a successful result in under 40 lines.
2. Add an example that requires an optional feature, declare it with required-features, and verify the behaviour of cargo run --example with and without the feature.
3. Interview: What belongs in a doc example, what belongs in examples/, and what belongs in an integration test? (Hint: consider audience, scope, and whether results are asserted.)

README and Crate-Level Docs

A crate has two front doors. The README is what people see on GitHub and crates.io when deciding whether to use the crate. The crate-level documentation (//! at the top of src/lib.rs) is the first page of the API docs on docs.rs, read by people who have already decided and want to start. Both should answer, within seconds: what is this, why would I use it, and what does the simplest use look like?

A proven structure for crate-level docs:

//! Fast, zero-copy parsing of `key = value` configuration files.
//!
//! `kv` reads configuration without allocating for keys or values, validates
//! it against an optional schema, and reports errors with line numbers.
//!
//! # Quick start
//!
//! ```
//! let config = kv::Config::parse("host = example.com\nport = 8080")?;
//! assert_eq!(config.get("port"), Some("8080"));
//! # Ok::<(), kv::Error>(())
//! ```
//!
//! # Overview
//!
//! - [`Config`]: a parsed file; borrow values with [`Config::get`]
//! - [`Schema`]: optional validation rules
//! - [`Error`]: everything that can go wrong, with source locations
//!
//! # Feature flags
//!
//! | Feature | Default | Enables |
//! |---------|---------|---------|
//! | `serde` | no      | `Deserialize` into your own structs |
//! | `watch` | no      | Reloading when the file changes |
//!
//! # Minimum supported Rust version
//!
//! Rust 1.85 or newer.
//!
//! See the `examples/` directory for complete programs.

Sharing one source. Maintaining two near-identical texts invites drift. You can make the README be the crate docs:

// src/lib.rs
#![doc = include_str!("../README.md")]     // README becomes the crate docs, and its
                                           // ```rust blocks become doc tests

Things to watch with this approach:

• README code blocks are now doc tests. Mark shell snippets as ```sh or ```text, or they will be compiled as Rust and fail.
• Relative links in the README ([guide](docs/guide.md)) work on GitHub but not on docs.rs. Use absolute URLs.
• Badges and HTML that look right on GitHub may render oddly in rustdoc.

If you prefer separate texts but still want the README's examples tested, a #[cfg(doctest)] item includes it only when collecting doc tests:

// src/lib.rs: tests README examples without making the README the crate docs
#[cfg(doctest)]
#[doc = include_str!("../README.md")]
struct ReadmeDoctests;

The reverse direction, generating the README from the crate docs, is handled by tools such as cargo-rdme, which can also check in CI that the two are in sync.

Concern	README	Crate-level docs
Shown on	GitHub, crates.io	docs.rs (crate root page)
Audience	Evaluators: "should I use this?"	Users: "how do I start?"
Typical content	Pitch, installation, quick example, badges, license, contributing	Quick start, concepts, links to key types, feature flags, MSRV
Links	URLs	Intra-doc links
Tested	Only if included in doc tests	Always (it is a doc comment)

In Cargo.toml, readme = "README.md" sets the file crates.io displays (Cargo picks up README.md automatically if present), and documentation can point to a custom docs URL. Without it, crates.io links to docs.rs.

Key Takeaways

• The README sells and orients; crate-level //! docs onboard users of the API. Both need a one-line pitch and a minimal working example.
• Structure crate docs as summary, quick start, overview of key types, feature flags, and MSRV.
• #![doc = include_str!("../README.md")] shares one source and tests the README's examples; mark non-Rust blocks and use absolute links.
• #[cfg(doctest)] with include_str! tests README examples without making it the crate docs.

🧪 Practice

1. Write crate-level docs for a library following the structure above, including a feature-flag table, and view them with cargo doc --open.
2. Include your README with #![doc = include_str!("../README.md")], run cargo test --doc, and fix every block that fails.
3. Interview: A crate's README example stopped compiling two releases ago and nobody noticed. What process change would have caught it? (Hint: consider which documentation cargo test actually executes.)

docs.rs Configuration

Every version of every crate published to crates.io gets documentation built automatically on docs.rs. That is convenient, but the build happens on docs.rs's machines with its defaults. Unless told otherwise, it documents only the default features, for one target (x86_64-unknown-linux-gnu), with a nightly rustdoc, in a sandbox without network access. If your crate's most interesting API sits behind optional features, or only exists on Windows, the hosted docs will simply omit it.

Configuration lives in Cargo.toml under [package.metadata.docs.rs], which Cargo itself ignores:

[package.metadata.docs.rs]
all-features = true                       # document everything, including optional APIs
# features = ["serde", "watch"]           # ...or a specific list
# no-default-features = true
rustdoc-args = ["--cfg", "docsrs"]        # lets the code detect a docs.rs build
targets = ["x86_64-unknown-linux-gnu", "x86_64-pc-windows-msvc"]  # extra platforms
# default-target = "x86_64-unknown-linux-gnu"

docs.rs passes --cfg docsrs itself; many crates still list it explicitly, which is harmless and also documents the intent.

Feature badges with doc(cfg). When everything is documented, readers need to know which items need a feature or platform. rustdoc can render a note such as "Available on crate feature serde only." next to the item. The attribute behind this, doc_cfg, is still nightly-only, so it is enabled only when docsrs is set: docs.rs uses nightly, while normal users build on stable and never see the attribute.

// lib.rs
#![cfg_attr(docsrs, feature(doc_cfg))]   // only on docs.rs (nightly)

#[cfg(feature = "serde")]
#[cfg_attr(docsrs, doc(cfg(feature = "serde")))]   // explicit badge
pub mod serde_support {
    //! `Deserialize` integration.
}

#[cfg(windows)]
#[cfg_attr(docsrs, doc(cfg(windows)))]   // "Available on Windows only."
pub fn registry_config() {}

Rendered on docs.rs:

  pub mod serde_support
  ┌──────────────────────────────────────────────────┐
  │ Available on crate feature serde only.           │
  └──────────────────────────────────────────────────┘
  `Deserialize` integration.

Recent nightlies merged the former doc_auto_cfg feature into doc_cfg, so with feature(doc_cfg) enabled rustdoc also infers badges from plain #[cfg] attributes. The explicit doc(cfg(...)) form still works and is useful when the inferred condition is not what you want to show.

Preview locally with the same settings before publishing:

RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc --all-features --no-deps --open

Build-environment pitfalls:

Problem on docs.rs	Cause	Fix
Build fails in build.rs	No network; native library or tool not installed	Check the DOCS_RS environment variable in build.rs and skip the native build
Feature-gated items missing	Only default features are built	all-features = true or a features list
Platform-specific API missing	Only the default target is built	Add it to targets
Build times out	Heavy dependencies or build scripts	Disable expensive features for docs; skip work when DOCS_RS is set
#![feature(...)] error on stable	Unconditional nightly feature	Gate with cfg_attr(docsrs, ...)

// build.rs: skip compiling the bundled C library on docs.rs
fn main() {
    if std::env::var_os("DOCS_RS").is_some() {
        return;                         // docs only need the Rust API, not a linkable library
    }
    // ... compile and link the native library ...
}

Once published, a version's docs.rs page cannot be edited. Fixing the configuration requires publishing a new version, so preview locally first.

Key Takeaways

• docs.rs builds docs for every published version with default features, one target, a nightly rustdoc, and no network access.
• Configure it with [package.metadata.docs.rs]: all-features, features, targets, rustdoc-args.
• Gate nightly-only doc features with #![cfg_attr(docsrs, feature(doc_cfg))] and mark feature- or platform-specific items with doc(cfg(...)).
• Make build.rs skip native work when DOCS_RS is set, and preview with RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc --all-features.

🧪 Practice

1. Add [package.metadata.docs.rs] to a crate with optional features, add doc(cfg) badges, and preview the result locally with nightly.
2. Write a build.rs that compiles a small C file with the cc crate but skips it when DOCS_RS is set, and verify the skip by setting the variable locally.
3. Interview: A user reports that a function from your crate's README "does not exist" on docs.rs, although it compiles fine for them. What are the likely causes, and how do you fix them for the next release? (Hint: think about features, targets, and what docs.rs builds by default.)

---