Motivation
Test coverage across compwa_policy is currently low — most check modules sit between 10–30%. Coverage here is not just a number: while pushing cli/_settings.py to 100% (in the #610 PR), the missing line turned out to be a real bug — the settings_customise_sources hook was misspelled (...customize...), so it never ran and environment variables silently leaked into the resolved policy config. Exercising the logic is what found it.
This issue tracks a focused effort to raise coverage where it has genuine correctness value, in the self-contained style established in #610 (inline dedent fixtures, no separate -bad/-good fixture files, tmp_path / dumps() / changelog assertions, doctests for small pure helpers).
The shape that makes a module unit-testable
Most check modules follow the same pattern: a main() plus _update_* / _remove_* helpers that take a ModifiablePyproject, ModifiablePrecommit, or text stream, mutate it in place, and raise PrecommitError with a changelog. These are pure-ish transforms — input config in, output config out — and are exactly what unit tests cover cleanly. The test suite is already kept offline (conftest.py stubs git ls-remote), so rev pinning is deterministic.
Genuinely unit-testable (high value)
Config-transform modules where inline-fixture unit tests fit well:
format/ — toml.py, prettier.py, precommit.py, cspell.py
python/ — black.py, mypy.py, pyproject.py, pytest.py, pyupgrade.py, ruff.py
repo/ — citation.py, commitlint.py, deprecated.py, gitpod.py, poe.py, vscode.py
github/ — dependabot.py, labels.py, release_drafter.py, upgrade_lock.py, workflows.py (YAML in/out)
nb/ — binder.py, jupyter.py, nbstripout.py
cli/migrate.py — has real branching logic (dry-run, args: → pyproject.toml relocation, nb-hook relocation); a 604 deliverable worth locking down.
Small pure helpers in utilities/ are good doctest candidates rather than tests/ files.
Lower value / integration-only (skip or use CliRunner)
cli/_checks.py — the orchestrator that runs every side-effecting check; only meaningfully tested as an integration test against a full fake repo.
cli/ subcommand wrappers (python.py, github.py, nb.py, format.py, repo.py, env.py) — thin forwarders; a unit test just asserts argument forwarding (low signal). Better covered by a couple of typer.testing.CliRunner smoke tests.
self_check.py and anything that shells out to git/subprocess or the network — keep behind mocks or out of scope.
Suggested approach
- Inline
dedent fixtures + tmp_path; assert on .dumps() output and/or the raised changelog message.
- Doctests for small pure functions (counts toward coverage, documents in place).
- One module per PR to keep reviews small; start with
cli/migrate.py and the format/ group.
Motivation
Test coverage across
compwa_policyis currently low — most check modules sit between 10–30%. Coverage here is not just a number: while pushingcli/_settings.pyto 100% (in the #610 PR), the missing line turned out to be a real bug — thesettings_customise_sourceshook was misspelled (...customize...), so it never ran and environment variables silently leaked into the resolved policy config. Exercising the logic is what found it.This issue tracks a focused effort to raise coverage where it has genuine correctness value, in the self-contained style established in #610 (inline
dedentfixtures, no separate-bad/-goodfixture files,tmp_path/dumps()/ changelog assertions, doctests for small pure helpers).The shape that makes a module unit-testable
Most check modules follow the same pattern: a
main()plus_update_*/_remove_*helpers that take aModifiablePyproject,ModifiablePrecommit, or text stream, mutate it in place, and raisePrecommitErrorwith a changelog. These are pure-ish transforms — input config in, output config out — and are exactly what unit tests cover cleanly. The test suite is already kept offline (conftest.pystubsgit ls-remote), sorevpinning is deterministic.Genuinely unit-testable (high value)
Config-transform modules where inline-fixture unit tests fit well:
format/—toml.py,prettier.py,precommit.py,cspell.pypython/—black.py,mypy.py,pyproject.py,pytest.py,pyupgrade.py,ruff.pyrepo/—citation.py,commitlint.py,deprecated.py,gitpod.py,poe.py,vscode.pygithub/—dependabot.py,labels.py,release_drafter.py,upgrade_lock.py,workflows.py(YAML in/out)nb/—binder.py,jupyter.py,nbstripout.pycli/migrate.py— has real branching logic (dry-run,args:→pyproject.tomlrelocation, nb-hook relocation); a 604 deliverable worth locking down.Small pure helpers in
utilities/are good doctest candidates rather thantests/files.Lower value / integration-only (skip or use
CliRunner)cli/_checks.py— the orchestrator that runs every side-effecting check; only meaningfully tested as an integration test against a full fake repo.cli/subcommand wrappers (python.py,github.py,nb.py,format.py,repo.py,env.py) — thin forwarders; a unit test just asserts argument forwarding (low signal). Better covered by a couple oftyper.testing.CliRunnersmoke tests.self_check.pyand anything that shells out togit/subprocessor the network — keep behind mocks or out of scope.Suggested approach
dedentfixtures +tmp_path; assert on.dumps()output and/or the raised changelog message.cli/migrate.pyand theformat/group.