feat(workflows): add weekly GitHub code scanning automation#1495
feat(workflows): add weekly GitHub code scanning automation#1495rezatnoMsirhC wants to merge 17 commits into
Conversation
- Create weekly-security-scanning.yml on Monday 03:00 UTC schedule - Reuse gh-code-scanning skill script; no duplication to scripts/security/ - Deduplicate backlog issues via automation marker per unique rule - Detect scan analysis failures and file labeled ci-scanning-failure issue 🔒 - Generated by Copilot
- Delete weekly-security-scanning.yml (replaced by two new files) - Create gh-security-scanning.yml as workflow_call-only reusable - Create weekly-gh-security-scanning.yml as thin schedule orchestrator ♻️ - Generated by Copilot
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1495 +/- ##
==========================================
+ Coverage 85.46% 85.69% +0.22%
==========================================
Files 82 81 -1
Lines 11802 11780 -22
==========================================
+ Hits 10087 10095 +8
+ Misses 1715 1685 -30
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…le workflow - Add create-gh-security-scanning-issues.yml as workflow_call reusable - Strip issue-creation steps from gh-security-scanning.yml; add artifact upload - Update weekly orchestrator to call both workflows via needs: 🔒 - Generated by Copilot
- Remove owner and repo inputs from gh-security-scanning.yml - Remove owner and repo inputs from create-gh-security-scanning-issues.yml - Drop with: blocks from weekly orchestrator jobs 🔧 - Generated by Copilot
🧹 - Generated by Copilot
…apply Q1/Q2/Q3 fixes - Remove deferred Check for scan analysis errors step (Q1) - Rename all three workflow files to gh-code-scanning naming (Q2) - Add else branch to issue loop: gh issue edit + gh issue comment (Q3) 🔧 - Generated by Copilot
rezatnoMsirhC
changed the title
- Add on: pull_request targeting main with opened/synchronize/reopened/ready_for_review types - Update concurrency to cancel stale PR runs, preserve scheduled run behavior 🧪 - Generated by Copilot
- Extend PS grouper: sentinel filter, AffectedPaths, HasFilePaths, AlertUrl, FindingDescription - Rewrite bash step: bulleted linked paths, conditional repo-level section - Fix labels, title severity, dedup to body marker, grammar, enriched body 🔒 - Generated by Copilot
- rename SamplePaths to AffectedPaths in JSON example and prose - add HasFilePaths, AlertUrl, FindingDescription to example and field list - fix BranchProtectionID example to show empty AffectedPaths array - add GroupedJson alias and Key fields API path disambiguation note 📝 - Generated by Copilot
…e finding - add --title and --add-label flags to gh issue edit on existing issues - remove duplicate FindingDescription from repo-level PATHS_SECTION 🔒 - Generated by Copilot
…back field - add Severity (rule.severity) field to PS script grouped output - fall back to Severity when SecuritySeverity is null - omit [SEVERITY] title bracket and severity body line when both are null 🔧 - Generated by Copilot
- rename SamplePaths references to AffectedPaths in three tests - update no-file test to assert empty array and HasFilePaths false 🧪 - Generated by Copilot
…scan 🔧 - Generated by Copilot
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
PR Review — feat(workflows): add weekly GitHub code scanning automation
The overall architecture is well-conceived — splitting the scan, issue creation, and scheduling into three composable reusable workflows is the right pattern. The improvements to Get-CodeScanningAlerts.ps1 (richer output fields, HasFilePaths guard, AlertUrl, FindingDescription) are a clear improvement, and the updated tests correctly track the renamed fields. The request-changes verdict is driven by three concrete issues that must be addressed before merging.
🔗 Issue Alignment
Linked issue #1329 could not be read (integrity policy), but the PR description is comprehensive and the implementation matches the declared scope. No scope creep detected. ✅
📋 PR Template Compliance
| Check | Status | Note |
|---|---|---|
| Description filled in | ✅ | Detailed and well-structured |
| Related Issue(s) | ✅ | Closes #1329 |
| Type of Change checked | ✅ | New feature + GitHub Actions workflow |
| Testing section | ✅ | All automated checks listed with results |
| "Documentation is updated" | SKILL.md was substantively updated; this checkbox should be checked |
|
| Security Considerations section | The PR template includes a ## Security Considerations block with three checkboxes that is absent from this PR description. Please add it and check the applicable items. |
|
npm run plugin:generate and npm run docs:test |
These two required automated checks appear in the template but are absent from the PR checklist. Mark them N/A if they don't apply, or run and record results. |
🔍 Coding Standards
Bash script conventions: The run: block in create-gh-code-scanning-issues.yml is missing set -euo pipefail (see inline comment, line 39). The bash conventions file explicitly requires this at the top of every bash script block.
Workflow permissions: security-events: read appears in create-gh-code-scanning-issues.yml at both the workflow and job level but is never used (see inline comment, line 14). All other permission declarations follow least-privilege correctly.
SHA-pinned actions: All three new workflows correctly pin SHA-referenced actions. ✅
🛡️ Code Quality and Security
❌ Orphaned file with broken field reference — create-gh-security-scanning-issues.yml
This is the primary blocking issue. A fourth new workflow file, create-gh-security-scanning-issues.yml, was added but:
- It reads
.SamplePaths(the old field name), which was renamed to.AffectedPathsbyGet-CodeScanningAlerts.ps1in this same PR (see inline comment, line 44 of that file). Every issue body created by this workflow would have an empty paths section. - Its deduplication logic searches by title (
"[Security] ${RULE_DESC}" in:title) rather than by the HTML-comment marker in the body ("${MARKER}" in:body) — diverging from the strategy increate-gh-code-scanning-issues.yml. - It is not referenced by
weekly-gh-code-scanning.ymlor any other orchestrator. It is unreachable dead code.
Action required: Either remove create-gh-security-scanning-issues.yml entirely, or update it to use AffectedPaths, align its deduplication logic, and wire it into the scheduler.
ISSUE_BODY double-quoted string
ISSUE_BODY is assigned via a double-quoted string that embeds $(...) subshell expressions populated with API-sourced values such as FINDING_DESC. Bash performs word-splitting and command substitution inside double-quoted variable assignments, so a FINDING_DESC value containing $(cmd) would execute cmd. The risk is low given the source (GitHub API), but it violates the principle of defense-in-depth. See inline comment on line 81.
✅ Action Items
- Remove or fix
create-gh-security-scanning-issues.yml— the file is broken (stale field) and unreachable. This is the primary blocker. - Add
set -euo pipefailto therun:block increate-gh-code-scanning-issues.yml. - Mitigate shell expansion in the
ISSUE_BODYconstruction increate-gh-code-scanning-issues.yml. - Remove
security-events: readfromcreate-gh-code-scanning-issues.yml(both workflow and job level). - Check the "Documentation is updated" checkbox —
SKILL.mdwas updated. - Add the missing
## Security Considerationssection from the PR template. - Address
npm run plugin:generateandnpm run docs:testin the automated checks checklist.
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1329
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none
There was a problem hiding this comment.
PR Review: feat(workflows): add weekly GitHub code scanning automation
Overview
The core design — a composable three-workflow architecture for weekly code scanning automation — is well-structured and largely follows repository conventions. The SKILL.md update, PowerShell script enhancements, and test changes are all solid. However, one blocking issue must be addressed before this can merge.
Issue Alignment ✅
PR closes #1329. The implementation scope (three reusable workflows, skill improvements, enriched alert output) is coherent with the stated goal of automating weekly security backlog management. No significant scope gaps were identified.
PR Template Compliance ⚠️
The Copilot skill checkbox is checked under Type of Change (correct — SKILL.md was modified), but the AI Artifact Contributions checklist entries carry "(N/A — no AI artifacts changed)" annotations. These are contradictory. Additionally, the Sample Prompts section is empty, which should be filled when a SKILL.md change is declared.
Please either:
- Fill the Sample Prompts section with a brief invocation example for the updated skill, or
- Uncheck the
Copilot skillType of Change checkbox and note the SKILL.md changes are purely structural/documentation updates.
Code Quality ❌ — Blocking
.github/workflows/create-gh-security-scanning-issues.yml must be removed or corrected.
See the inline comment at line 42 for the full analysis. In summary:
- The workflow references
.SamplePaths(renamed to.AffectedPathsin this PR) — it will silently producenullin every issue body. - It expects the artifact
gh-security-scanning-alerts, butgh-code-scanning.ymlonly uploadsgh-code-scanning-alerts— the download step will fail immediately. - It is not described in the PR description, not called by any orchestrator in this PR, and appears to be an accidental inclusion of an early draft.
Recommended action: Remove this file from the PR.
Coding Standards ✅ (with minor notes)
- All three new workflows properly SHA-pin their actions using the repository-standard SHAs with semantic version comments. ✅
persist-credentials: falseis correctly applied on the checkout step. ✅concurrencyis set on the orchestrator withcancel-in-progress: false. ✅- Top-level and job-level permissions are declared on all workflows. ✅
- Minor:
create-gh-code-scanning-issues.ymlincludessecurity-events: readat both permission scopes, but this workflow never reads security events directly. See inline comment at line 14.
Skill and Script Quality ✅ (with minor note)
The Get-CodeScanningAlerts.ps1 enrichment (Severity, AffectedPaths, HasFilePaths, AlertUrl, FindingDescription) is well-implemented and the test suite was updated to match. The GroupedJson alias is a nice ergonomic improvement.
Minor: The sentinel path filter uses a broad substring regex (-notmatch '(?i)no file') that could incorrectly drop legitimate paths. See inline comment at line 91 of the PowerShell script.
Required Actions
| # | Severity | Action |
|---|---|---|
| 1 | 🔴 Blocking | Remove (or fully update) .github/workflows/create-gh-security-scanning-issues.yml |
| 2 | 🟡 Minor | Tighten the sentinel filter in Get-CodeScanningAlerts.ps1 to an exact string match |
| 3 | 🟡 Minor | Remove security-events: read from create-gh-code-scanning-issues.yml permissions |
| 4 | 🔵 Info | Resolve the PR template contradiction (Copilot skill checkbox vs AI Artifact annotations) |
Note
🔒 Integrity filter blocked 1 item
The following item were blocked because they don't meet the GitHub integrity level.
- #1329
issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
To allow these resources, lower min-integrity in your GitHub frontmatter:
tools:
github:
min-integrity: approved # merged | approved | unapproved | none
katriendg
left a comment
katriendg
left a comment
There was a problem hiding this comment.
Really nice addition - splitting this into a workflow_call scan job, a reusable issue-creation job, and a thin schedule orchestrator gives us the right composability, and the per-rule dedup marker (automation:security-scan:${RULE_ID}) plus the weekly count refresh is exactly the deterministic floor a security backlog needs. The skill changes (AffectedPaths, HasFilePaths, AlertUrl, FindingDescription, Severity fallback) are well-scoped and the rendered issues from the test run (#1499–#1502) read clearly.
A few items to address before merge - full list and validation results are in the review thread; the only blocker is the orphan create-gh-security-scanning-issues.yml (leftover from the rename, unreferenced, still uses the removed SamplePaths field). Beyond that: the automated label isn't in labels.yml, the (?i)no file filter is too loose, and the new skill output fields aren't unit-tested.
On the agentic-workflow angle (and follow-up #1510): I'd keep this PR exactly as a deterministic floor - predictable, zero LLM cost, easy to reason about, easy to roll back. The places where a gh-aw layer would genuinely add value are on top of this, not in place of it: narrative bodies for repo-level rules like BranchProtectionID where there is no source file to link, dedup-by-root-cause across CodeQL rules that share a fix, auto-closing issues when the next scan shows the alert dismissed or fixed, and linking related findings as sub-issues (the "Issue Arborist" / "Sub-Issue Closer" patterns from githubnext/agentics - https://github.com/githubnext/agentics). Crucially the title format and the automation:security-scan:${RULE_ID} marker should stay deterministic so the agentic layer can match against them safely. Guardrails on the gh-aw side should pin safe-outputs.create-issue to title-prefix: "[Security]" and a fixed label allowlist, cap add-comment per run, and keep the agent itself read-only with the write job gated - i.e., the agent requests, a scoped job decides. That keeps this PR's deterministic guarantees intact while letting the agent earn its keep on the parts that actually need contextual reasoning.
Left some inline comments and sharing a list of low items:
Low / Informational
💡 L1. gh issue edit ... --add-label runs as a second API call
.github/workflows/create-gh-code-scanning-issues.yml
gh issue edit accepts --add-label in the same call as --title/--body. Combining them halves API calls per existing-issue update.
💡 L2. MARKER is also a stable id; consider promoting to a typed field in skill output
The skill could add a MarkerId = "automation:security-scan:${RuleId}" field so consumers do not re-derive it inconsistently. Optional.
💡 L3. Body uses Markdown task list inside an automated message
The "Action Required" checkboxes will never get programmatically updated, and weekly comment refreshes do not reset them. Cosmetic, but consider plain bullet text or a separate "Reviewer checklist" section to set expectations.
- delete orphaned create-gh-security-scanning-issues.yml - replace security-events: read with actions: read - add set -euo pipefail and heredoc to bash step - fix sentinel filter to exact-string match in Get-CodeScanningAlerts.ps1 - add Severity, AlertUrl, FindingDescription test assertions 🔧 - Generated by Copilot
…t/1329-gh-security-weekly-scanning-workflow
3 participants
feat(workflows): add weekly GitHub code scanning automation
Description
Added a weekly GitHub code scanning backlog automation that runs every Monday at 03:00 UTC. The implementation is split into a reusable
workflow_callscan workflow (gh-code-scanning.yml), a reusable issue-creation workflow (create-gh-code-scanning-issues.yml), and a thin schedule orchestrator (weekly-gh-code-scanning.yml), keeping the logic composable and the scheduler minimal.The scan workflow fetches code scanning alerts using the existing
gh-code-scanningskill'sGet-CodeScanningAlerts.ps1script and uploads them as a workflow artifact. The issue-creation workflow downloads the artifact, then iterates over unique rules to create labeledsecurityissues — updating existing open issues with the current alert count when a matching title is found.Workflow Architecture
gh-code-scanning.yml: reusableworkflow_callworkflow. Runs onubuntu-latest, scopes permissions tocontents: readandsecurity-events: readat both workflow and job level. Uses SHA-pinnedactions/checkoutwithpersist-credentials: false. Uploads agh-code-scanning-alertsartifact.create-gh-code-scanning-issues.yml: reusableworkflow_callworkflow. Downloads the alerts artifact, iterates over unique rules, and creates or updatessecurity-labeled backlog issues.weekly-gh-code-scanning.yml: schedule orchestrator. Triggers oncron: '0 3 * * 1'andworkflow_dispatch. Delegates togh-code-scanning.ymlthencreate-gh-code-scanning-issues.ymlviauses:. Setscancel-in-progress: falseto prevent overlapping scan runs.Backlog Issue Management
"[Security] ${RULE_DESC}" in:title) against open issues before creation.gh issue editrefreshes the body with the current alert count andgh issue commentposts a dated weekly re-check note.automation:security-scan:${RULE_ID}) for traceability.Related Issue(s)
Closes #1329
Related issues created during research and implementation
Issues created from test run
py/empty-exceptactions/code-injection/mediumBranchProtectionIDpy/redundant-comparisonType of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md) — documentation-only update to SKILL.md (output field descriptions); no skill logic changesOther:
.ps1,.sh,.py)Sample Prompts (for AI Artifact Contributions)
User Request:
Execution Flow:
Output Artifacts:
Success Indicators:
For detailed contribution requirements, see:
Testing
Automated validation results:
npm run lint:md): passed — 0 errors across 196 filesnpm run spell-check): passed — 0 issues across 300 filesnpm run lint:frontmatter): passed — 536 files, 0 errorsnpm run validate:skills): passed — 19 skills, 0 errors (1 pre-existing warning unrelated to this PR)npm run lint:md-links): passednpm run lint:ps): passed — all files cleanSecurity analysis:
actions/checkoutis SHA-pinned tode0fac2e4500dabe0009e67214ff5f5447ce83dd.persist-credentials: falseset on checkout step.Diff-based assessments:
Note
Add manual testing descriptions when applicable.
Security Considerations
actions/checkoutis SHA-pinned withpersist-credentials: falseChecklist
Required Checks
gh-code-scanning/SKILL.mdupdated with new output fields: Severity, AlertUrl, FindingDescription)AI Artifact Contributions
/prompt-analyzeto review contribution (N/A — no AI artifacts changed)prompt-builderreview (N/A — no AI artifacts changed)Required Automated Checks
The following validation commands must pass before merging:
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-linksnpm run lint:psnpm run plugin:generate: N/A — no collections or skill structure changes affecting plugin outputnpm run docs:test: N/A — no documentation test targets changed