test(vault-intercept): soft-dep regression tests for PR #1764#1852
test(vault-intercept): soft-dep regression tests for PR #1764#1852john-the-dev wants to merge 1 commit into
Conversation
17 structural + functional tests covering the detect-secrets soft-dep behavior added in sonichi#1764: lazy import inside try/except, unquoted value refused (not stored) when scanner missing, self-documenting REFUSED placeholder, quoted values bypass the guard and store normally, and key lands in `failed[]` not `stored[]` on refusal. Closes sonichi#1764 (test gap). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
Good regression coverage for the soft-dep refusal path added in #1764. Key invariants tested: lazy import (module-level import would crash bridges on hosts without detect-secrets), REFUSED-not-stored on unquoted values, self-documenting placeholder, quoted-value bypass, and Keychain-never-called on failure. All assertions are source-code structural checks — no live process or Keychain access needed, so the suite is safe to run in CI. Base is main.
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
The soft-dep structural tests (import inside try:, except ImportError present, etc.) are the correct technique for verifying a lazy-import design without actually uninstalling detect-secrets. The mock-based behavioral tests (refusal on missing dep, REFUSED placeholder, quoted-value bypass, Keychain never called on refusal) directly cover the four PR #1764 behavioral invariants. ✓
bassilkhilo-ag2
left a comment
There was a problem hiding this comment.
Structural source-code invariants for the soft-dep design are a clean way to guard this without needing to mock the import at runtime. 8 properties tested: lazy import position, try/except guard, REFUSED placeholder text, install instruction, never-echo instruction, is_quoted bypass. All meaningful for PR #1764's contract. LGTM.
Problem
PR #1764 (fix(vault): make detect-secrets a soft/dev-only dep) changed the behavior of
intercept_vault_commandswhendetect-secretsis absent: unquoted values are now refused (not stored) with a self-documenting placeholder rather than silently dropped or stored. No tests were added for this new path at the time of merge.Tests added (
tests/vault-intercept-soft-dep.test.py, 17 tests)Structural (9 tests) — source-code pattern assertions:
secret_scannerimport is NOT at module top level (lazy import, soft dep)try/except ImportErrorpython3 -m pip install detect-secretsinstructionis_quotedflag is present to bypass the FP scanif not is_quoted:wraps the scanner callfailed.append(key)is inside the ImportError handlerFunctional (8 tests) — simulate absent
detect-secretsviasys.modulesmock:failed[]; not instored[]failed[]Verification
Closes #1764 (test gap).