feat(scripts)(settings): add ADR consistency lint infrastructure#1552
feat(scripts)(settings): add ADR consistency lint infrastructure#1552WilliamBerryiii wants to merge 1 commit into
Conversation
- add ADR consistency module, parser, CLI, rules JSON, schemas, and Pester tests - register schemas, markdownlint excludes, and 'templates' skill subdirectory - wire lint:adr-consistency npm script (excluded from lint:all) - extend cspell and frontmatter validator excludes; regenerate plugin outputs 🔍 - Generated by Copilot
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1552 +/- ##
==========================================
- Coverage 85.46% 84.92% -0.55%
==========================================
Files 82 85 +3
Lines 11802 12248 +446
==========================================
+ Hits 10087 10402 +315
- Misses 1715 1846 +131
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
katriendg
left a comment
There was a problem hiding this comment.
Reviewed the full PR. Clean additive change — no regressions, no extension/plugin impact, conventions followed. Two non-blocking observations below.
|
|
||
| #region Helpers | ||
|
|
||
| function Remove-AdrFencedCodeBlocks { |
There was a problem hiding this comment.
Code Quality / Maintainability (Medium)
Remove-AdrFencedCodeBlocks is defined privately in both AdrConsistency.psm1 (here, regex-based) and AdrBodyParser.psm1 (line 15, stateful line-by-line parser) with divergent behavior:
| Module | Approach | Handles ~~~ fences |
Handles inline code spans |
|---|---|---|---|
| AdrConsistency.psm1 | Regex (?ms) |
No | Yes |
| AdrBodyParser.psm1 | Line-by-line state machine | Yes | No |
The AdrConsistency version misses ~~~-delimited fences (valid CommonMark). Consider consolidating into one shared function that handles both ~~~ fences and inline code spans, either exported from AdrBodyParser or placed in a shared helper module.
| - id: sc-1 | ||
| description: Validator runs in CI | ||
| source: README.md | ||
| decisionMetadata: |
There was a problem hiding this comment.
Test Quality (Low / Informational)
All 18 fixture files use id/description/source fields in success_criteria, but the ADR frontmatter schema (adr-frontmatter.schema.json) requires metric/target/measurement_window/source with additionalProperties: false.
The fixtures aren't schema-matched at runtime (their paths don't match docs/planning/adrs/[0-9][0-9][0-9][0-9]-*.md), so this doesn't break anything. Updating them to use the schema-defined shape would make them canonical examples for future contributors.
Description
Added a PowerShell-based linter that enforces structural consistency between Architecture Decision Record frontmatter and body sections in docs/planning/adrs/. The linter is shipped as two reusable modules, a CLI entry script, a JSON rule registry, and three Draft-07 JSON Schemas, with editor schema bindings, a markdownlint exclusion for fixture corpora, an
npm run lint:adr-consistencyentry, and a Pester 5.x test suite covering all nine rules through a paired pass/fail fixture pattern.The change is additive. No existing linter was modified except a one-line tolerance added to skill structure validation so skills can carry a templates/ subdirectory without warning. The new
lint:adr-consistencyscript is invokable on its own and is not wired into thelint:allchain in this PR.Linter core
Invoke-AdrConsistencyValidationdispatcher and nine privateTest-*rule functions coveringADR-CONSISTENCY-001throughADR-CONSISTENCY-009. The module declares#Requires -Version 7.0and pinsPowerShell-Yamlto version0.4.7, loads../rules/adr-consistency-rules.jsoninto$script:RuleRegistrykeyed by rule id, and routes all violation construction through a singleNew-AdrViolationfactory that owns rule lookup,{token}substitution, and severity assignment.Get-AdrBodySectionsparser. The parser splits an ADR body into structured sections — AffectedComponents, DecisionDrivers, DecisionOutcomeMatrixDrivers, BadConsequences, RisksAndMitigationsRisks, Confirmation, and three path-token sets — and exposes only the parser; helpers likeGet-AdrH2Section,Get-AdrH3SectionInH2(for MADR v4### Consequencesunder## Decision Outcome),Get-AdrTableRows,Get-AdrPathTokens,Get-AdrBadConsequenceBullets, and a statefulRemove-AdrFencedCodeBlockswalker remain private.-Paths,-Files,-ExcludePaths,-WarningsAsErrors,-ChangedFilesOnly,-BaseBranch, and-OutputPath(defaulting to logs/adr-consistency-results.json). The script importsModules/AdrConsistency.psm1,Modules/LintingHelpers.psm1, and../lib/Modules/CIHelpers.psm1; resolves the repo root viagit rev-parse --show-toplevelwith a$PSScriptRootfallback; and uses an ordinal-ignore-caseStartsWithboundary check to reject paths that escape the repository. Each violation prints to host output, and on CI the script also emitsWrite-CIAnnotationper finding plus a singleWrite-CIStepSummary. Exit code is1when error count is non-zero, or when-WarningsAsErrorsis set and warning count is non-zero.Rule registry, schemas, and editor integration
id,severity,scope,check,message, anddescriptionfields. Eight rules areerrorseverity and one (ADR-CONSISTENCY-007, numeric-claim-generalized) iswarn. Scopes split asfrontmatter(002, 008),body(003, 004, 005, 006, 007), andcross-region(001, 009). The kebab-casecheckslug maps mechanically to theTest-<PascalCase>PowerShell function name.ruleswithminItems: 1, pinsidto^ADR-CONSISTENCY-\d{3}$, restrictsseveritytoerror|warnandscopetofrontmatter|body|cross-region, and requirescheckto be kebab-case.docs/planning/adrs/NNNN-{kebab-case}.mdfrontmatter. Required fields areid,title,status,proposed_date,deciders, andaffected_components. The schema models the six-value status taxonomy (proposed,accepted,rejected,deprecated,superseded,withdrawn), constrainsefforttoS|M|L|XL, modelssuccess_criteriaitems withmetric/target/measurement_window/source, modelsrelateditems aspath/relationpairs (with relation enuminformational|influenced-by|influences), pairs eachasr_triggersitem with requiredevidence, and adds anallOfconditional that requiresaccepted_datewhenstatusisaccepted.docs/planning/adrs/.adr-config.yml. Required keys areproject_slug,owner,default_status,decision_id_format,template_source, andlast_decision_id.decision_id_formatis constrained to the literalNNNN;last_decision_idis pinned to^\d{4}$.docs/planning/adrs/[0-9][0-9][0-9][0-9]-*.mdagainst the adr-frontmatter schema, and the rule registry against the adr-consistency-rules schema../scripts/linting/schemas/adr-config.schema.jsonagainst**/.adr-config.ymland**/.adr-config.yamlunderyaml.schemasso editor validation matches the file-on-disk checker.Tests and fixtures
Unit. ABeforeDiscoveryblock declares a nine-element$RuleCasesarray mapping each rule id to its fixture directory; threeDescribeblocks then run data-drivenIt -ForEachiterations over the cases, asserting that eachpass.mdproduces zero violations, eachfail.mdfires its target rule, no violation message contains an unsubstituted{token}template, and the validator's return shape exposesFile/Violationsproperties with consistent fields. TheBeforeAllblock force-imports the consistency module and mocksWrite-Host; theAfterAllblock tears down both AdrConsistency and its AdrBodyParser dependency.pass.mdandfail.md. All fixtures derive from a single canonical 76-line Fixture base ADR 9999 baseline that follows MADR v4 layout:## Context,## Decision Drivers,## Considered Options,## Decision Outcome(with a driver matrix and an H3 Consequences block split into Good/Bad/Neutral H4 subsections),## Risks and Mitigations,## Confirmation,## More Information, and## Affected Components. Eachfail.mdmutates the baseline along exactly one axis to trigger a single target rule (for example, risks-consequences-pairing/fail.md swaps the Bad consequence to a risk-shaped statement absent from the Risks and Mitigations table).scripts/tests/linting/fixtures/**to the markdownlintignoreslist so the deliberately defective fail-fixture markdown does not trip project-wide linting.Configuration touches
lint:adr-consistency(pwsh -NoProfile -File scripts/linting/Validate-AdrConsistency.ps1 -Paths docs/planning/adrs) betweenlint:frontmatterandlint:collections-metadata. The script is not added to thelint:allchain in this PR.'templates'to$script:RecognizedSubdirectoriesalongside the existingscripts,references,assets,examples, andtestsentries.Related Issue(s)
Closes #1551.
Type of Change
Select all that apply:
Code & Documentation:
Infrastructure & Configuration:
AI Artifacts:
prompt-builderagent and addressed all feedback.github/instructions/*.instructions.md).github/prompts/*.prompt.md).github/agents/*.agent.md).github/skills/*/SKILL.md)Other:
.ps1,.sh,.py)Sample Prompts (for AI Artifact Contributions)
For detailed contribution requirements, see:
Testing
Ran
npm run lint:adr-consistencyagainst the working tree. The validator discovered the existing .adr-config.yml style entries under docs/planning/adrs/ and reportedADR consistency: 1 file(s) | 0 error(s) | 0 warning(s)with exit code0. The Pester suite at scripts/tests/linting/AdrConsistency.Tests.ps1 exercises all nine rules through paired pass/fail fixtures, validates the validator's return contract, and asserts that no rule message contains an unsubstituted{token}placeholder.Checklist
Required Checks
AI Artifact Contributions
/prompt-analyzeto review contributionprompt-builderreviewRequired Automated Checks
The following validation commands must pass before merging:
npm run lint:mdnpm run spell-checknpm run lint:frontmatternpm run validate:skillsnpm run lint:md-links— 7 pre-existing baseline failures onmainnot introduced by this PR (.github/skills/security/secure-by-design/SKILL.md,docs/agents/code-review/README.md,docs/agents/rai-planning/agent-overview.md,docs/agents/rai-planning/phase-reference.md,docs/architecture/README.md,docs/getting-started/collections.md,docs/getting-started/mcp-configuration.md)npm run lint:psnpm run plugin:generatenpm run docs:testSecurity Considerations
Additional Notes
The new
lint:adr-consistencyscript is intentionally not wired intolint:allin this PR. Thedocs/planning/adrs/corpus does not yet contain ADR markdown files, so the linter is a no-op against the current tree. A follow-up PR will introduce the ADR Planner upgrade that produces ADR documents and enable the linter as part oflint:all.The
Validate-SkillStructure.ps1change addstemplatesto$script:RecognizedSubdirectories, allowing skills that ship author-facing templates to validate without warnings. No existing skill relies on atemplates/directory being rejected.