[Travis Angevine (trangevi)]

Extension command additions can be missed when registration and constructor changes are in different files. Repeated reviews can also request an experiences spec again for contracts already covered by earlier threads.

The review procedure now traces base/head command registrations across files, records each contract item, and matches earlier threads even when they are grouped, unmarked, resolved, or outdated. Merged findings must name every covered item so later reviews can recognize that coverage. Findings must be anchored on changed diff lines; unchanged constructors are inspected only for context. The companion skill version is bumped to 1.3.

Seven new eval tasks include six contract-review regressions covering cross-file command registration, multiple independent commands, command-tree-preserving refactors, grouped existing threads, partial thread coverage with a new flag, and explicit item coverage in merged findings, plus a JSON-extraction regression for leading diagnostics, trailing logs, nested objects, and escaped strings.

Validation: the full 15-task suite passed after the initial review fixes. Subsequent targeted duplicate-matching and merged-finding evals passed. waza check, spelling, and whitespace checks passed.

Fixes #10821

[azure-pipelines]

Azure Pipelines:
27 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟡 Changes recommended

The evaluation fixtures require an out-of-scope service-API thread match and allow comments with incorrect file/side anchors or missing experiences-spec requests to pass.

2 open issues

• Use an authored YAML schema for the trace-window thread · New
• Validate comment path, side, and spec-request text · New

What changed in this PR

This PR strengthens azd’s extension contract-review tooling to catch commands registered separately from their constructors while avoiding false positives for command-tree-preserving refactors.

Changes:

• Adds cross-file tracing of base/head registrations, constructors, and parent commands.
• Expands per-contract coverage tracking and the parent reviewer’s completion audit.
• Adds three regression evaluations and bumps the companion skill to version 1.3.

Extension user-contract review: not-applicable (0 extension files inspected; 0 new contract comments; 0 existing threads).

File	Description
.github/​skills/​extension-user-contract-review/​SKILL.md	Bumps the companion skill version to 1.3.
.github/​skills/​extension-user-contract-review/​references/​review.md	Defines cross-file command tracing and distinct-contract tracking.
.github/​skills/​extension-user-contract-review/​evals/​tasks/​multiple-command-registrations.yaml	Tests separate findings for two new commands.
.github/​skills/​extension-user-contract-review/​evals/​tasks/​command-tree-refactor.yaml	Tests that registration refactors produce no contract findings.
.github/​skills/​extension-user-contract-review/​evals/​tasks/​command-registration.yaml	Tests command discovery alongside an existing review thread.
.github/​skills/​azd-code-reviewer/​SKILL.md	Aligns parent reviewer guidance and completion audits.

_{Review effort: Max}

[Copilot]

🔵 Needs a closer look

The evaluation cases lack changed-line coordinates and a registration-only scenario, leaving inline-anchor behavior and command-path tracing through unchanged source untested.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Test tracing command paths through unchanged declarations

.github/​skills/​extension-user-contract-review/​evals/​tasks/​command-registration.yaml:15

The prompt supplies the resolved command path and places the constructor in an added file, so it does not exercise tracing into unchanged source. Add a registration-only case that supplies the unchanged constructor and parent declarations without spelling out the resolved path. Assert the full command path in the resulting comment.

Add changed-line coordinates to the refactor case

.github/​skills/​extension-user-contract-review/​evals/​tasks/​command-tree-refactor.yaml:16

This refactor case supplies no changed-line coordinates, although the harness says tasks provide them. That leaves the zero-comment check untested against valid inline anchors for the extracted registrations. Add LEFT coordinates for the removed calls and RIGHT coordinates for the replacement call and helper.

2 resolved since last review

• Validate comment path, side, and spec-request text
• Use an authored YAML schema for the trace-window thread

_{Review effort: Max}

[Copilot]

🟡 Changes recommended

Merged findings can omit covered inputs, causing prior-thread matching to treat unchanged inputs as uncovered and generate duplicate spec requests.

1 open issue

• Require merged findings to name every covered input · New

_{Review effort: Max}

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

2 open issues

• PR description says 'Three regression evals' were added, but this change set adds more than three… · New
• This row suggests anchoring a finding on a constructor line, but earlier guidance requires evidence… · New

1 resolved since last review

• Require merged findings to name every covered input

_{Review effort: Lite}

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open issues

• The regex used to extract the JSON (/\\{\\s*\"body\"[\\s\\S]*\\}/) is greedy and can over-match… · New
• This step requires comparing “base and head registrations,” but the procedure doesn’t describe what… · New
• The ledger definition mixes two different concepts (“every changed extension file” vs “each… · New

2 resolved since last review

• This row suggests anchoring a finding on a constructor line, but earlier guidance requires evidence…
• PR description says 'Three regression evals' were added, but this change set adds more than three…

_{Review effort: Lite}

[Copilot]

🟡 Changes recommended

The shared grader’s greedy extraction fails when output contains an earlier diagnostic object, and the regression checks only the task-local selector rather than exercising that shared extraction path.

1 open issue

• Shared grader still uses greedy review-object extraction · New

3 resolved since last review

• The ledger definition mixes two different concepts (“every changed extension file” vs “each…
• This step requires comparing “base and head registrations,” but the procedure doesn’t describe what…
• The regex used to extract the JSON (/\\{\\s*\"body\"[\\s\\S]*\\}/) is greedy and can over-match…

_{Review effort: Max}

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

3 open issues

• The new parsing strategy assumes the review JSON object runs all the way to the end of output… · New
• The PR description says “Six new regression evals”, but this change set appears to add 7 new task… · New
• These assertions repeatedly recompute the same startIndex and re-JSON.parse(...) the same… · New

1 resolved since last review

• Shared grader still uses greedy review-object extraction

_{Review effort: Lite}

[Copilot]

🟡 Changes recommended

A balanced but invalid trailing JSON-like candidate can make JSON.parse throw, rejecting an earlier valid review instead of continuing to the previous candidate.

1 open issue

• Ignore invalid trailing JSON candidates · New

3 resolved since last review

• The new parsing strategy assumes the review JSON object runs all the way to the end of output…
• These assertions repeatedly recompute the same startIndex and re-JSON.parse(...) the same…
• The PR description says “Six new regression evals”, but this change set appears to add 7 new task…

_{Review effort: Max}

[Copilot]

🔵 Needs a closer look

Human assessment is needed because the regression can pass despite missing required shorthand aliases, leaving its coverage unreliable.

No open issues 🎉

1 resolved since last review

• Ignore invalid trailing JSON candidates

_{Review effort: Max}

[Travis Angevine (trangevi)]

/check-enforcer override