P0018 fix-without-test¶
Rule: A commit that fixes code adds a test.
| Code | P0018 (stable, never reused) |
| Id | fix-without-test |
| Category | commits |
| Source | spirit_level |
| Fix | by hand |
| Needs | git installed |
Level per tier¶
| Tier | Level | Reason |
|---|---|---|
| small | off | |
| small-workspace | off | |
| medium | off | |
| big | warn |
a bug is fixed when a test fails without the fix; a warning, since a fix is judged from its subject |
| ai | warn |
an agent writes the failing test first; a warning, since a fix is judged from its subject |
| embedded | off | |
| ui | off | |
| egui | off | |
| units-strict | off | |
| units-paranoid | off |
Parameters¶
| Name | Type | Default | Meaning |
|---|---|---|---|
fix-words |
list of strings | ["fix", "fixes", "fixed"] |
words that, opening a subject, make a commit a fix (case-insensitive); a Fixes: trailer always does |
code |
list of strings | ["**/*.rs"] |
globs of the files whose change makes a fix need a test |
tests |
list of strings | ["tests/**", "**/tests/**", "**/benches/**", "fuzz/**"] |
globs of test files: changing one counts as a test |
markers |
list of strings | ["#[test]", "#[tokio::test", "#[test_case", "#[rstest", "proptest!"] |
text of an added line that counts as a test |
base |
string | "main" |
the branch work merges into; commits in base..HEAD are checked |
Why¶
A bug is fixed only when a test that fails without the fix is in the suite: the suite then remembers the mistake, and the fix stays fixed. Writing the failing test first also shows that the fix repairs the bug that was seen.
A commit counts as a fix when it has a Fixes: trailer (fixes-tag) or its subject opens with a fix word (Fix, Fixes, Fixed). It needs a test when it changes code (**/*.rs outside test paths). It has one when it changes a file under a test path (tests/, benches/, fuzz/) or adds a line with a test marker (#[test], #[tokio::test], proptest!). The judgement from the subject is an estimate, so the finding says which rule made the commit a fix.
How to fix¶
Add the test to the same commit: write it first, see it fail, then fix. Where no test can be written (the bug needs hardware, a race no test reaches), say so in a trailer: Spirit-Level: allow(fix-without-test) needs the board on the bench.
Configure¶
[lints.fix-without-test]
level = "deny"
fix-words = ["fix", "fixes", "fixed"]
code = ["**/*.rs"]
tests = ["tests/**", "**/tests/**", "**/benches/**", "fuzz/**"]
markers = ["#[test]", "#[tokio::test", "#[test_case", "#[rstest", "proptest!"]
base = "main"
reason = "why this project wants it at this level"
Allow one finding in the code with // spirit-level: allow(fix-without-test) <reason> (waivers); the code works there too, as in [lints.P0018].