Skip to content

test(vault-intercept): soft-dep regression tests for PR #1764#1852

Open
john-the-dev wants to merge 1 commit into
sonichi:mainfrom
john-the-dev:test/vault-intercept-soft-dep-1764
Open

test(vault-intercept): soft-dep regression tests for PR #1764#1852
john-the-dev wants to merge 1 commit into
sonichi:mainfrom
john-the-dev:test/vault-intercept-soft-dep-1764

Conversation

@john-the-dev

Copy link
Copy Markdown
Contributor

Problem

PR #1764 (fix(vault): make detect-secrets a soft/dev-only dep) changed the behavior of intercept_vault_commands when detect-secrets is absent: unquoted values are now refused (not stored) with a self-documenting placeholder rather than silently dropped or stored. No tests were added for this new path at the time of merge.

Tests added (tests/vault-intercept-soft-dep.test.py, 17 tests)

Structural (9 tests) — source-code pattern assertions:

  • secret_scanner import is NOT at module top level (lazy import, soft dep)
  • Import is guarded by try/except ImportError
  • REFUSED placeholder text is present in source
  • Placeholder contains python3 -m pip install detect-secrets instruction
  • Placeholder includes "Never echo" guard
  • is_quoted flag is present to bypass the FP scan
  • if not is_quoted: wraps the scanner call
  • failed.append(key) is inside the ImportError handler

Functional (8 tests) — simulate absent detect-secrets via sys.modules mock:

  • Unquoted value → REFUSED in result text
  • REFUSED placeholder explains "detect-secrets not installed"
  • Refused key in failed[]; not in stored[]
  • Keychain subprocess never called on refusal
  • Quoted value stores normally even without scanner (bypasses FP guard)
  • Raw value never in sanitized text on refusal
  • Multiple unquoted refusals accumulate in failed[]

Verification

python3 tests/vault-intercept-soft-dep.test.py
Ran 17 tests in 0.003s — OK

Closes #1764 (test gap).

17 structural + functional tests covering the detect-secrets soft-dep
behavior added in sonichi#1764: lazy import inside try/except, unquoted value
refused (not stored) when scanner missing, self-documenting REFUSED
placeholder, quoted values bypass the guard and store normally, and
key lands in `failed[]` not `stored[]` on refusal.

Closes sonichi#1764 (test gap).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good regression coverage for the soft-dep refusal path added in #1764. Key invariants tested: lazy import (module-level import would crash bridges on hosts without detect-secrets), REFUSED-not-stored on unquoted values, self-documenting placeholder, quoted-value bypass, and Keychain-never-called on failure. All assertions are source-code structural checks — no live process or Keychain access needed, so the suite is safe to run in CI. Base is main.

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The soft-dep structural tests (import inside try:, except ImportError present, etc.) are the correct technique for verifying a lazy-import design without actually uninstalling detect-secrets. The mock-based behavioral tests (refusal on missing dep, REFUSED placeholder, quoted-value bypass, Keychain never called on refusal) directly cover the four PR #1764 behavioral invariants. ✓

@bassilkhilo-ag2 bassilkhilo-ag2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structural source-code invariants for the soft-dep design are a clean way to guard this without needing to mock the import at runtime. 8 properties tested: lazy import position, try/except guard, REFUSED placeholder text, install instruction, never-echo instruction, is_quoted bypass. All meaningful for PR #1764's contract. LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants