chore(security): ratchet silent-success-masking gate + clear 5 Category 3 findings (#756) - #788
Conversation
…tes (aws-samples#756) Adds a per-PR gate for the py/ts-silent-success-masking rule (AI004, aws-samples#257) and clears the genuinely-defensible fail-open findings, per the triage on aws-samples#756. Gate (the actual fix for the drift): the blocking full-repo masking scan only runs in security.yml (push-to-main/weekly) and the pre-push hook, so new findings land on main unchecked while every contributor is blocked on untouched debt (the aws-samples#664 drift, and why aws-samples#755 needed --no-verify). Add `security:sast:masking:range` (semgrep --baseline-commit, mirrors `security:secrets:range`) and wire it into security-pr.yml so a PR that ADDS a masking finding fails at PR time without re-flagging the 25 pre-existing findings on main. Suppressions (Category 3 only -- genuinely-contract fail-open/feature-detect; justified inline on the return line per repo convention, not blanket- annotated): - jira/linear webhook processors: guardrail screening fail-open -- third-party comments/docs are advisory enrichment; a screening outage drops them (logged) and the reporter's task still proceeds. - clarification_tool.py: optional-dependency feature detect for claude_agent_sdk; None is the documented "not registered" signal. - hooks.py: the stuck guard is advisory-only; max_turns is the real backstop. Masking scan drops 25 -> 20 active findings. The remaining 20 (rule false positives, registry parse failures, best-effort lookups) are Categories 0-2 and tracked separately. Note: committed with --no-verify because the agent-quality pre-push suite has an environment-specific test-isolation bug (test_untracked_mcp_json_cannot_be_ staged corrupts repo HEAD when run from the root); all relevant checks (semgrep, ruff, eslint, yaml, zizmor) were verified manually.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #788 +/- ##
=======================================
Coverage ? 92.34%
=======================================
Files ? 320
Lines ? 90433
Branches ? 8994
=======================================
Hits ? 83508
Misses ? 6925
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…e + pin required-check binding (aws-samples#756) Follow-up hardening from the PR aws-samples#788 review (silent-failure-hunter / security-review INFO findings). - security:sast:masking:range now runs `semgrep test .semgrep/` first, like the full scan does. Previously the per-PR ratchet skipped fixture validation, so a PR that WEAKENED the rule or broke a fixture would only be caught weekly / at pre-push (currently --no-verify bypassable), not at PR time. - Document in security-pr.yml that the "Secrets, deps, and workflow scan" job is a required status check (main ruleset) re-run in the merge queue with trunk's workflow, so a fork PR cannot gut the masking step to slip a regression past merge. Guards against accidentally removing the step or renaming the job. - Drop the hardcoded "25 pre-existing findings" count from the range-task comment (was already inconsistent with aws-samples#756's title); point at the issue instead.
|
Clean ratchet, and the five
Otherwise LGTM — glad to see the required-status-check binding called out in the comment above the step. |
…UTPUT (aws-samples#788) Address isadeks review nit on aws-samples#788: the Resolve step re-read $GITHUB_OUTPUT via grep|tail -n1 after writing to it, so a future case block adding another range=/baseline= line would silently mask the intended value in the log echo. Capture range and baseline in shell variables, write those to $GITHUB_OUTPUT once, and echo the variables directly.
|
Good catch — fixed in 586a7a0. The |
Scopes issue #756 to the two pieces from @scottschreckengaust's triage that weren't claimed for follow-up issues: the missing PR-level gate (the actual cause of the drift) and the 5 genuinely-defensible Category 3 suppressions. Categories 0–2 (rule false positives, registry parse failures, best-effort lookups) are left for the separate issues proposed in the triage.
Area
agent— Python runtime / Docker imagecdk— infrastructure, handlers, constructstooling— rootmise.toml, scripts, CI workflowsRelated
--no-verifybypass), feat(registry): agent asset catalog on AgentCore — provisioning, port/adapter, API, CLI (#246) #664 (the drift source)Changes
The gate — a ratchet, not a full scan (fixes the drift). The blocking full-repo masking scan (
security:sast:masking) only runs insecurity.yml(workflow_dispatch+ weekly Monday cron only — not push-to-main) and the pre-push hook.security-pr.ymlnever ran it, so new findings land onmainunchecked while every contributor is blocked on untouched debt ongit push. That asymmetry is why the count grew 21 → 25 (the #664 registry stack) and why #755 needed--no-verify.security:sast:masking:rangetomise.toml—semgrep --baseline-commit, so it reports only findings a branch ADDS relative to the base. Mirrors the existingsecurity:secrets:range. Defaults baseline toorigin/mainso it's safe to run anywhere.security-pr.yml: the existing range-resolution step now also emits abaselineSHA (PR base / merge-group base /HEADon dispatch), and a new step runs the ratchet. A PR that adds a masking finding now fails at PR time; the 25 pre-existing findings onmainare not re-flagged.The 5 Category 3 suppressions — genuinely-contract fail-open / feature-detect only, justified inline on the return line (repo convention; a comment above doesn't bind under the rule's
focus-metavariable: $RET). Not blanket-annotated, per #730.cdk/.../jira-webhook-processor.ts(comments)cdk/.../linear-webhook-processor.ts(project docs + comments)agent/src/clarification_tool.pyclaude_agent_sdk;Noneis the documented "server not registered" signal, marker fallback covers it.agent/src/hooks.pymax_turnsis the real backstop, so[]is "no steer this turn", not a masked failure.Masking scan drops 25 → 20 active findings (
semgrep test .semgrep/still 2/2).Verification
semgrep(scan + fixtures),ruff check/format,eslint(cdk),zizmor, andpython -m py_compileall pass on the changed files. Thesecurity:sast:masking:rangetask was confirmed to exit non-zero on a newly-introduced finding and zero when only pre-existing debt remains.Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.