feat(scripts): add instruction applyTo scope Pester suite#1640
feat(scripts): add instruction applyTo scope Pester suite#1640WilliamBerryiii wants to merge 2 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1640 +/- ##
==========================================
- Coverage 85.50% 85.49% -0.01%
==========================================
Files 82 82
Lines 11805 11805
==========================================
- Hits 10094 10093 -1
- Misses 1711 1712 +1
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
Document that planner-related Pester tests are intentionally split between `scripts/tests/linting/` (rule validators) and `scripts/tests/security/` (artifact signing). Consolidation under a single `planner/` subdirectory is deferred to backlog item IV-011. Reference: .copilot-tracking/reviews/2026-05-22/stacked-prs-from-pr-1497-review.md (IV-011)
| } | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
The Pester instructions require -Tag 'Unit' on all Describe blocks. Without it, CI runs filtered by -Tag Unit will skip this suite silently.
Describe 'Phase-narrowed applyTo regression guard' -Tag 'Unit' {| $script:cases = foreach ($root in $script:scanRoots) { | ||
| if (-not (Test-Path $root)) { continue } | ||
| Get-ChildItem -Path $root -Filter '*.md' -Recurse -File | ForEach-Object { | ||
| @{ | ||
| Path = $_.FullName | ||
| Rel = ($_.FullName.Substring($script:repoRoot.Length + 1) -replace '\\','/') | ||
| } | ||
| } |
There was a problem hiding this comment.
$script:cases is populated at file scope before the Pester container runs. If either scan root is absent (renamed, shallow clone, etc.), the continue silently skips it. With zero cases, -ForEach produces zero It executions and Pester reports a green suite, so the guard gives no signal that it is covering nothing.
Consider moving the population into BeforeAll and asserting the collection is non-empty:
BeforeAll {
# ... existing setup ...
$script:cases = foreach ($root in $script:scanRoots) {
if (-not (Test-Path $root)) { throw "Scan root not found: $root" }
Get-ChildItem -Path $root -Filter '*.md' -Recurse -File | ForEach-Object {
@{ Path = $_.FullName; Rel = ($_.FullName.Substring($script:repoRoot.Length + 1) -replace '\\','/') }
}
}
$script:cases | Should -Not -BeNullOrEmpty -Because 'at least one instruction file must be discovered'
}| #> | ||
|
|
||
| $script:repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '../../..')).Path | ||
| Import-Module powershell-yaml -Force -ErrorAction SilentlyContinue |
There was a problem hiding this comment.
Import-Module powershell-yaml -Force -ErrorAction SilentlyContinue appears twice: once at file scope (line 13) and again inside BeforeAll (line 31). Both calls use -ErrorAction SilentlyContinue, so a missing module fails silently. The first visible error would be ConvertFrom-Yaml: The term 'ConvertFrom-Yaml' is not recognized, which obscures the actual cause.
The idiomatic fix is a single #Requires directive at the top of the file, replacing both calls:
#Requires -Modules Pester, powershell-yaml
# Copyright (c) Microsoft Corporation.
# SPDX-License-Identifier: MIT| Consolidation into a single `planner/` subdirectory is tracked as backlog | ||
| item IV-011 in | ||
| `.copilot-tracking/reviews/2026-05-22/stacked-prs-from-pr-1497-review.md`. |
There was a problem hiding this comment.
The sentence references .copilot-tracking/reviews/2026-05-22/stacked-prs-from-pr-1497-review.md, but .copilot-tracking/ is gitignored and never committed. The IV-011 ID is an Implementation Validator finding number that lives in that same local, unshared tree. Both references are inaccessible to any other contributor after merge.
Two options to consider:
Option A: Remove the sentence. The preceding lines already convey the rationale for the split, and the tracking reference adds no durable value.
Option B: Open a GitHub issue to track the consolidation work and replace the sentence with a link to it, giving any contributor a live pointer they can actually follow.
Which approach works for you?
Pull Request
Description
This PR adds a single Pester regression guard,
scripts/tests/linting/Test-InstructionApplyToScope.Tests.ps1, that enforces phase-narrowedapplyToglobs on every instruction file under.github/instructions/security/**and.github/instructions/rai-planning/**. Any file whose frontmatter declaresapplyTo: '**'is rejected unless its repo-relative path is on the in-script allowlist (currently empty, matching the Phase 6 audit-matrix migration map).The suite is data-driven (
It -ForEach) so each scanned file produces a discrete test case (currently 18 cases, all passing). It usespowershell-yaml(already required byscripts/linting/Validate-MarkdownFrontmatter.ps1and other existing suites — no new repo-level dependency) and explicitly throws on missing frontmatter so silent skips cannot mask regressions.This PR is a sibling of #1497, branched off
main. It is independent of PRs A/C/B in the post-#1497 stack and can land in any order relative to them.Scope reduction
The originating plan listed four Pester suites under "planner linter hardening." Three are deferred to PR B because they assert content not yet on
main:security/identity.instructions.mdandsssc-identity.instructions.md.## StartupH2 in six prompts under.github/prompts/security/and.github/prompts/security/sssc/.security-model.instructions.md.Shipping those tests now would produce a red base. They will land in PR B alongside the content edits they exercise.
Related Issue(s)
Sibling of #1497. No other issue references.
Type of Change
Code & Documentation:
Infrastructure & Configuration:
Other:
.ps1,.sh,.py)Testing
Automated validation performed by the agent:
npm run test:ps -- -TestPath scripts/tests/linting/Test-InstructionApplyToScope.Tests.ps1— 18/18 passed.npm run lint:ps— clean for repository code.Security analysis findings:
powershell-yamlis already required by existing linting and test suites.Diff-based assessments:
scripts/tests/linting/Test-*.Tests.ps1naming and structure.Note
Manual testing was not performed.
Checklist
Required Checks
Required Automated Checks
npm run lint:psSecurity Considerations
Additional Notes
Companion PRs in the post-#1497 work stream:
main, optional).