Skip to content

Increase unit-test coverage of check modules #615

Description

@redeboer

Motivation

Test coverage across compwa_policy is currently low — most check modules sit between 10–30%. Coverage here is not just a number: while pushing cli/_settings.py to 100% (in the #610 PR), the missing line turned out to be a real bug — the settings_customise_sources hook was misspelled (...customize...), so it never ran and environment variables silently leaked into the resolved policy config. Exercising the logic is what found it.

This issue tracks a focused effort to raise coverage where it has genuine correctness value, in the self-contained style established in #610 (inline dedent fixtures, no separate -bad/-good fixture files, tmp_path / dumps() / changelog assertions, doctests for small pure helpers).

The shape that makes a module unit-testable

Most check modules follow the same pattern: a main() plus _update_* / _remove_* helpers that take a ModifiablePyproject, ModifiablePrecommit, or text stream, mutate it in place, and raise PrecommitError with a changelog. These are pure-ish transforms — input config in, output config out — and are exactly what unit tests cover cleanly. The test suite is already kept offline (conftest.py stubs git ls-remote), so rev pinning is deterministic.

Genuinely unit-testable (high value)

Config-transform modules where inline-fixture unit tests fit well:

  • format/toml.py, prettier.py, precommit.py, cspell.py
  • python/black.py, mypy.py, pyproject.py, pytest.py, pyupgrade.py, ruff.py
  • repo/citation.py, commitlint.py, deprecated.py, gitpod.py, poe.py, vscode.py
  • github/dependabot.py, labels.py, release_drafter.py, upgrade_lock.py, workflows.py (YAML in/out)
  • nb/binder.py, jupyter.py, nbstripout.py
  • cli/migrate.py — has real branching logic (dry-run, args:pyproject.toml relocation, nb-hook relocation); a 604 deliverable worth locking down.

Small pure helpers in utilities/ are good doctest candidates rather than tests/ files.

Lower value / integration-only (skip or use CliRunner)

  • cli/_checks.py — the orchestrator that runs every side-effecting check; only meaningfully tested as an integration test against a full fake repo.
  • cli/ subcommand wrappers (python.py, github.py, nb.py, format.py, repo.py, env.py) — thin forwarders; a unit test just asserts argument forwarding (low signal). Better covered by a couple of typer.testing.CliRunner smoke tests.
  • self_check.py and anything that shells out to git/subprocess or the network — keep behind mocks or out of scope.

Suggested approach

  • Inline dedent fixtures + tmp_path; assert on .dumps() output and/or the raised changelog message.
  • Doctests for small pure functions (counts toward coverage, documents in place).
  • One module per PR to keep reviews small; start with cli/migrate.py and the format/ group.

Metadata

Metadata

Assignees

Labels

🔨 MaintenanceMaintenance and upkeep improvements🖱️ DXImprovements to the Developer Experience

Type

Fields

No fields configured for Task.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions