Skip to content

feat(scripts)(settings): add ADR consistency lint infrastructure#1552

Open
WilliamBerryiii wants to merge 1 commit into
mainfrom
feat/adr-consistency-lint
Open

feat(scripts)(settings): add ADR consistency lint infrastructure#1552
WilliamBerryiii wants to merge 1 commit into
mainfrom
feat/adr-consistency-lint

Conversation

@WilliamBerryiii
Copy link
Copy Markdown
Member

Description

Added a PowerShell-based linter that enforces structural consistency between Architecture Decision Record frontmatter and body sections in docs/planning/adrs/. The linter is shipped as two reusable modules, a CLI entry script, a JSON rule registry, and three Draft-07 JSON Schemas, with editor schema bindings, a markdownlint exclusion for fixture corpora, an npm run lint:adr-consistency entry, and a Pester 5.x test suite covering all nine rules through a paired pass/fail fixture pattern.

The change is additive. No existing linter was modified except a one-line tolerance added to skill structure validation so skills can carry a templates/ subdirectory without warning. The new lint:adr-consistency script is invokable on its own and is not wired into the lint:all chain in this PR.

Linter core

  • Added scripts/linting/Modules/AdrConsistency.psm1 (857 lines) with the public Invoke-AdrConsistencyValidation dispatcher and nine private Test-* rule functions covering ADR-CONSISTENCY-001 through ADR-CONSISTENCY-009. The module declares #Requires -Version 7.0 and pins PowerShell-Yaml to version 0.4.7, loads ../rules/adr-consistency-rules.json into $script:RuleRegistry keyed by rule id, and routes all violation construction through a single New-AdrViolation factory that owns rule lookup, {token} substitution, and severity assignment.
  • Added scripts/linting/Modules/AdrBodyParser.psm1 (445 lines) with the public Get-AdrBodySections parser. The parser splits an ADR body into structured sections — AffectedComponents, DecisionDrivers, DecisionOutcomeMatrixDrivers, BadConsequences, RisksAndMitigationsRisks, Confirmation, and three path-token sets — and exposes only the parser; helpers like Get-AdrH2Section, Get-AdrH3SectionInH2 (for MADR v4 ### Consequences under ## Decision Outcome), Get-AdrTableRows, Get-AdrPathTokens, Get-AdrBadConsequenceBullets, and a stateful Remove-AdrFencedCodeBlocks walker remain private.
  • Added scripts/linting/Validate-AdrConsistency.ps1 as the CLI entry script. Parameters cover -Paths, -Files, -ExcludePaths, -WarningsAsErrors, -ChangedFilesOnly, -BaseBranch, and -OutputPath (defaulting to logs/adr-consistency-results.json). The script imports Modules/AdrConsistency.psm1, Modules/LintingHelpers.psm1, and ../lib/Modules/CIHelpers.psm1; resolves the repo root via git rev-parse --show-toplevel with a $PSScriptRoot fallback; and uses an ordinal-ignore-case StartsWith boundary check to reject paths that escape the repository. Each violation prints to host output, and on CI the script also emits Write-CIAnnotation per finding plus a single Write-CIStepSummary. Exit code is 1 when error count is non-zero, or when -WarningsAsErrors is set and warning count is non-zero.

Rule registry, schemas, and editor integration

  • Added scripts/linting/rules/adr-consistency-rules.json declaring all nine rules with id, severity, scope, check, message, and description fields. Eight rules are error severity and one (ADR-CONSISTENCY-007, numeric-claim-generalized) is warn. Scopes split as frontmatter (002, 008), body (003, 004, 005, 006, 007), and cross-region (001, 009). The kebab-case check slug maps mechanically to the Test-<PascalCase> PowerShell function name.
  • Added scripts/linting/schemas/adr-consistency-rules.schema.json as a Draft-07 schema for the rule registry. It requires rules with minItems: 1, pins id to ^ADR-CONSISTENCY-\d{3}$, restricts severity to error|warn and scope to frontmatter|body|cross-region, and requires check to be kebab-case.
  • Added scripts/linting/schemas/adr-frontmatter.schema.json as a Draft-07 schema for docs/planning/adrs/NNNN-{kebab-case}.md frontmatter. Required fields are id, title, status, proposed_date, deciders, and affected_components. The schema models the six-value status taxonomy (proposed, accepted, rejected, deprecated, superseded, withdrawn), constrains effort to S|M|L|XL, models success_criteria items with metric/target/measurement_window/source, models related items as path/relation pairs (with relation enum informational|influenced-by|influences), pairs each asr_triggers item with required evidence, and adds an allOf conditional that requires accepted_date when status is accepted.
  • Added scripts/linting/schemas/adr-config.schema.json as a Draft-07 schema for docs/planning/adrs/.adr-config.yml. Required keys are project_slug, owner, default_status, decision_id_format, template_source, and last_decision_id. decision_id_format is constrained to the literal NNNN; last_decision_id is pinned to ^\d{4}$.
  • Modified scripts/linting/schemas/schema-mapping.json to register two new entries: ADR markdown files via the glob docs/planning/adrs/[0-9][0-9][0-9][0-9]-*.md against the adr-frontmatter schema, and the rule registry against the adr-consistency-rules schema.
  • Modified .vscode/settings.json to register ./scripts/linting/schemas/adr-config.schema.json against **/.adr-config.yml and **/.adr-config.yaml under yaml.schemas so editor validation matches the file-on-disk checker.

Tests and fixtures

  • Added scripts/tests/linting/AdrConsistency.Tests.ps1 as a Pester 5.x suite tagged Unit. A BeforeDiscovery block declares a nine-element $RuleCases array mapping each rule id to its fixture directory; three Describe blocks then run data-driven It -ForEach iterations over the cases, asserting that each pass.md produces zero violations, each fail.md fires its target rule, no violation message contains an unsubstituted {token} template, and the validator's return shape exposes File/Violations properties with consistent fields. The BeforeAll block force-imports the consistency module and mocks Write-Host; the AfterAll block tears down both AdrConsistency and its AdrBodyParser dependency.
  • Added 18 fixture files under scripts/tests/linting/fixtures/adr-consistency/ — one folder per rule, each with pass.md and fail.md. All fixtures derive from a single canonical 76-line Fixture base ADR 9999 baseline that follows MADR v4 layout: ## Context, ## Decision Drivers, ## Considered Options, ## Decision Outcome (with a driver matrix and an H3 Consequences block split into Good/Bad/Neutral H4 subsections), ## Risks and Mitigations, ## Confirmation, ## More Information, and ## Affected Components. Each fail.md mutates the baseline along exactly one axis to trigger a single target rule (for example, risks-consequences-pairing/fail.md swaps the Bad consequence to a risk-shaped statement absent from the Risks and Mitigations table).
  • Modified .markdownlint-cli2.jsonc to add scripts/tests/linting/fixtures/** to the markdownlint ignores list so the deliberately defective fail-fixture markdown does not trip project-wide linting.

Configuration touches

  • Modified package.json to register lint:adr-consistency (pwsh -NoProfile -File scripts/linting/Validate-AdrConsistency.ps1 -Paths docs/planning/adrs) between lint:frontmatter and lint:collections-metadata. The script is not added to the lint:all chain in this PR.
  • Modified scripts/linting/Validate-SkillStructure.ps1 to add 'templates' to $script:RecognizedSubdirectories alongside the existing scripts, references, assets, examples, and tests entries.

Related Issue(s)

Closes #1551.

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

For detailed contribution requirements, see:

Testing

Ran npm run lint:adr-consistency against the working tree. The validator discovered the existing .adr-config.yml style entries under docs/planning/adrs/ and reported ADR consistency: 1 file(s) | 0 error(s) | 0 warning(s) with exit code 0. The Pester suite at scripts/tests/linting/AdrConsistency.Tests.ps1 exercises all nine rules through paired pass/fail fixtures, validates the validator's return contract, and asserts that no rule message contains an unsubstituted {token} placeholder.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links — 7 pre-existing baseline failures on main not introduced by this PR (.github/skills/security/secure-by-design/SKILL.md, docs/agents/code-review/README.md, docs/agents/rai-planning/agent-overview.md, docs/agents/rai-planning/phase-reference.md, docs/architecture/README.md, docs/getting-started/collections.md, docs/getting-started/mcp-configuration.md)
  • PowerShell analysis: npm run lint:ps
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

The new lint:adr-consistency script is intentionally not wired into lint:all in this PR. The docs/planning/adrs/ corpus does not yet contain ADR markdown files, so the linter is a no-op against the current tree. A follow-up PR will introduce the ADR Planner upgrade that produces ADR documents and enable the linter as part of lint:all.

The Validate-SkillStructure.ps1 change adds templates to $script:RecognizedSubdirectories, allowing skills that ship author-facing templates to validate without warnings. No existing skill relies on a templates/ directory being rejected.

- add ADR consistency module, parser, CLI, rules JSON, schemas, and Pester tests
- register schemas, markdownlint excludes, and 'templates' skill subdirectory
- wire lint:adr-consistency npm script (excluded from lint:all)
- extend cspell and frontmatter validator excludes; regenerate plugin outputs

🔍 - Generated by Copilot
@WilliamBerryiii WilliamBerryiii requested a review from a team as a code owner May 10, 2026 16:15
@github-actions
Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.91723% with 130 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.92%. Comparing base (e158d88) to head (43d4da5).

Files with missing lines Patch % Lines
scripts/linting/Validate-AdrConsistency.ps1 0.00% 104 Missing ⚠️
scripts/linting/Modules/AdrConsistency.psm1 92.07% 18 Missing ⚠️
scripts/linting/Modules/AdrBodyParser.psm1 93.04% 8 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1552      +/-   ##
==========================================
- Coverage   85.46%   84.92%   -0.55%     
==========================================
  Files          82       85       +3     
  Lines       11802    12248     +446     
==========================================
+ Hits        10087    10402     +315     
- Misses       1715     1846     +131     
Flag Coverage Δ
pester 82.87% <70.91%> (-0.73%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/linting/Validate-MarkdownFrontmatter.ps1 78.22% <ø> (ø)
scripts/linting/Validate-SkillStructure.ps1 94.46% <100.00%> (ø)
scripts/linting/Modules/AdrBodyParser.psm1 93.04% <93.04%> (ø)
scripts/linting/Modules/AdrConsistency.psm1 92.07% <92.07%> (ø)
scripts/linting/Validate-AdrConsistency.ps1 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@katriendg katriendg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full PR. Clean additive change — no regressions, no extension/plugin impact, conventions followed. Two non-blocking observations below.


#region Helpers

function Remove-AdrFencedCodeBlocks {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality / Maintainability (Medium)

Remove-AdrFencedCodeBlocks is defined privately in both AdrConsistency.psm1 (here, regex-based) and AdrBodyParser.psm1 (line 15, stateful line-by-line parser) with divergent behavior:

Module Approach Handles ~~~ fences Handles inline code spans
AdrConsistency.psm1 Regex (?ms) No Yes
AdrBodyParser.psm1 Line-by-line state machine Yes No

The AdrConsistency version misses ~~~-delimited fences (valid CommonMark). Consider consolidating into one shared function that handles both ~~~ fences and inline code spans, either exported from AdrBodyParser or placed in a shared helper module.

Comment on lines +8 to +11
- id: sc-1
description: Validator runs in CI
source: README.md
decisionMetadata:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Quality (Low / Informational)

All 18 fixture files use id/description/source fields in success_criteria, but the ADR frontmatter schema (adr-frontmatter.schema.json) requires metric/target/measurement_window/source with additionalProperties: false.

The fixtures aren't schema-matched at runtime (their paths don't match docs/planning/adrs/[0-9][0-9][0-9][0-9]-*.md), so this doesn't break anything. Updating them to use the schema-defined shape would make them canonical examples for future contributors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(scripts): add ADR consistency lint infrastructure

3 participants