Skip to content

P0018 fix-without-test

Rule: A commit that fixes code adds a test.

Code P0018 (stable, never reused)
Id fix-without-test
Category commits
Source spirit_level
Fix by hand
Needs git installed

Level per tier

Tier Level Reason
small off
small-workspace off
medium off
big warn a bug is fixed when a test fails without the fix; a warning, since a fix is judged from its subject
ai warn an agent writes the failing test first; a warning, since a fix is judged from its subject
embedded off
ui off
egui off
units-strict off
units-paranoid off

Parameters

Name Type Default Meaning
fix-words list of strings ["fix", "fixes", "fixed"] words that, opening a subject, make a commit a fix (case-insensitive); a Fixes: trailer always does
code list of strings ["**/*.rs"] globs of the files whose change makes a fix need a test
tests list of strings ["tests/**", "**/tests/**", "**/benches/**", "fuzz/**"] globs of test files: changing one counts as a test
markers list of strings ["#[test]", "#[tokio::test", "#[test_case", "#[rstest", "proptest!"] text of an added line that counts as a test
base string "main" the branch work merges into; commits in base..HEAD are checked

Why

A bug is fixed only when a test that fails without the fix is in the suite: the suite then remembers the mistake, and the fix stays fixed. Writing the failing test first also shows that the fix repairs the bug that was seen.

A commit counts as a fix when it has a Fixes: trailer (fixes-tag) or its subject opens with a fix word (Fix, Fixes, Fixed). It needs a test when it changes code (**/*.rs outside test paths). It has one when it changes a file under a test path (tests/, benches/, fuzz/) or adds a line with a test marker (#[test], #[tokio::test], proptest!). The judgement from the subject is an estimate, so the finding says which rule made the commit a fix.

How to fix

Add the test to the same commit: write it first, see it fail, then fix. Where no test can be written (the bug needs hardware, a race no test reaches), say so in a trailer: Spirit-Level: allow(fix-without-test) needs the board on the bench.

Configure

[lints.fix-without-test]
level = "deny"
fix-words = ["fix", "fixes", "fixed"]
code = ["**/*.rs"]
tests = ["tests/**", "**/tests/**", "**/benches/**", "fuzz/**"]
markers = ["#[test]", "#[tokio::test", "#[test_case", "#[rstest", "proptest!"]
base = "main"
reason = "why this project wants it at this level"

Allow one finding in the code with // spirit-level: allow(fix-without-test) <reason> (waivers); the code works there too, as in [lints.P0018].