fix(ci): use GITHUB_TOKEN installation auth + model fallback for Copilot workflows

[takyyon]

Commit Type

• feature - New functionality
• fix - Bug fix
• refactor - Code restructuring without behavior change
• perf - Performance improvement
• docs - Documentation update
• test - Test-related changes
• chore - Maintenance/tooling

Risk Level

• Low - Minor changes, limited scope
• Medium - Moderate changes, some user impact
• High - Major changes, significant user/system impact

What & Why

The Copilot-powered CI workflows added in #9385 fail on every PR whose author
does not personally hold a GitHub Copilot seat (e.g. #9393). Two checks —
Test Copilot CLI and Copilot: PR Validation — went red on unrelated PRs.

Root cause. Every workflow invoked the CLI as
COPILOT_GITHUB_TOKEN="$GITHUB_TOKEN" copilot .... Setting COPILOT_GITHUB_TOKEN
selects the CLI's personal-token auth path, which authenticates as the
triggering actor and draws AI credits from that user's Copilot seat. So it
only worked for PR authors who happen to have a seat, and failed for everyone
else, for forks, and for bots. GitHub's documented robust path is the plain
GITHUB_TOKEN installation auth: it authenticates as the installation and
is metered directly to the organization, independent of the author — with a
short-lived token and nothing to rotate.

This PR switches every direct-CLI Copilot workflow to the installation path
(GITHUB_TOKEN env + --yolo, no COPILOT_GITHUB_TOKEN), adds an ordered
model-fallback loop so an unavailable model no longer fails a run, and removes
two now-unneeded workflows: a redundant validator that collided with the
existing one and hard-failed PRs, and a dispatch-only smoke test that
collided with the unit-test check.

What changed

Workflow	Change
pr-ai-validation.yml	Installation auth + model fallback. Kept as the single PR validator (native Copilot primary, Logic App only as last-resort fallback).
copilot-pr-validation.yml	Deleted. Redundant duplicate: its validate-pr job collided with pr-ai-validation.yml, and its deterministic fallback hard-failed PR bodies.
copilot-test-cli.yml	Deleted. Was a workflow_dispatch diagnostic that also collided with the unit-test test check on pull_request. Used to verify auth/model availability during this fix, then removed — no ongoing gating purpose.
copilot-investigate.yml	Installation auth + model fallback.
copilot-design-pass.yml	Installation auth + model fallback.
copilot-flake-review.yml	Installation auth + model fallback.

Auth flow (before → after)

BEFORE:  COPILOT_GITHUB_TOKEN=$GITHUB_TOKEN copilot ...   → personal-token path
         → authenticates as the PR author → needs the author to own a Copilot seat
         → author has seat: pass ; author has no seat / fork / bot: "Authentication failed"

AFTER:   GITHUB_TOKEN=$GITHUB_TOKEN copilot --yolo ...     → installation path
         → authenticates as the installation, metered to the org
         → passes for every author, fork, and bot; short-lived token, nothing to rotate

Model-fallback chain

claude-opus-4.8 → claude-opus-4.6 → gpt-5-mini. The first model that
returns a successful response wins; an unavailable model logs a warning and the
loop advances. Chain is overridable per-workflow via the COPILOT_MODELS env.

Impact of Change

• Users: None. CI-only; no product, UX, runtime, or API change.
• Developers: PRs from any contributor (seat or no seat), fork, or bot now
  get Copilot validation instead of a spurious red check. A Copilot/model hiccup
  no longer hard-blocks a PR. The Test Copilot CLI check no longer runs on
  every PR (dispatch-only) and no longer collides with the unit-test test check.
• System: Copilot AI-credit usage for these workflows is now metered to the
  org (installation billing) rather than an individual author's seat.

Test Plan

• Unit tests added/updated
• E2E tests added/updated
• Manual testing completed
• Tested in: GitHub Actions on this branch

Verified live in Actions on fix/copilot-cli-actions-auth via the
Test Copilot CLI smoke workflow:

• Installation auth works org-wide — copilot --yolo with only GITHUB_TOKEN
  (no COPILOT_GITHUB_TOKEN) returned COPILOT_CLI_WORKING, confirming the org
  policy "Allow use of Copilot CLI billed to the organization" is enabled and the
  path is author-independent.
• Model fallback works — a deliberately invalid primary model
  (definitely-not-a-real-model) logged a warning and the loop fell through to
  claude-opus-4.8, which succeeded.
• Model availability probed — confirmed claude-opus-4.8, claude-opus-4.6,
  claude-sonnet-4.5, and gpt-5-mini are available to the CLI in this org;
  gpt-5, gpt-5.x, and *-codex ids are not, so the chain uses gpt-5-mini
  as the GPT fallback.

[github-actions]

🤖 AI PR Validation Report

PR Review Results

Thank you for your submission! Here's detailed feedback on your PR title and body compliance:

✅ PR Title

• Current: fix(ci): use GITHUB_TOKEN installation auth + model fallback for Copilot workflows
• Issue: None — valid fix(ci): prefix and clearly descriptive of the change.
• Recommendation: No change needed.

---

✅ Commit Type

• Exactly one type selected: fix - Bug fix.
• Matches the title prefix and the nature of the change (fixing broken CI auth).

---

✅ Risk Level

• Exactly one box selected (Medium) and it matches the risk:medium label. This also matches the advised estimate: the change alters repository-governance automation — it swaps the Copilot CLI auth path (personal-token → installation/org-billed) and removes a pull_request_target validator, affecting the repo's security/permissions posture while remaining CI-only with no shipped-product impact. Medium is correct.

---

✅ What & Why

• Current: Detailed root-cause analysis (personal-token vs installation auth), a per-workflow change table, before/after auth flow, and the model-fallback chain rationale.
• Issue: None.
• Recommendation: No change needed.

---

✅ Impact of Change

• All three audiences addressed: Users (none — CI-only), Developers (validation now works for any author/fork/bot; no hard-block on model hiccups), System (AI-credit metering moves to org billing).
• Recommendation:
  • Users: As stated — no product/UX/runtime/API change.
  • Developers: As stated — clear.
  • System: As stated — clear.

---

✅ Test Plan

• Manual testing checked with a clear, adequate explanation of why automated tests don't apply: these are GitHub Actions workflow YAML changes, verified live in Actions on the branch (installation auth returned COPILOT_CLI_WORKING, model fallback exercised, model availability probed). No unit/E2E tests are meaningful for workflow config.

---

⚠️ Contributors

• Section is blank. Not required, but if PMs/designers/reviewers helped shape this fix, consider crediting them.

---

✅ Screenshots/Videos

• Not required. The diff only touches .github/workflows/ files — no user-facing UI under libs/designer-ui/src/, libs/designer/src/, or apps/vs-code-react/src/.

---

Summary Table

Section	Status	Recommendation
Title	✅	No change needed
Commit Type	✅	No change needed
Risk Level	✅	Medium is correct and matches label
What & Why	✅	No change needed
Impact of Change	✅	No change needed
Test Plan	✅	Manual testing justified for CI-only change
Contributors	⚠️	Optionally credit collaborators
Screenshots/Videos	✅	N/A — no UI changes

---

✅ All required checks pass. This PR is compliant with the team template and is cleared to merge. Great, thorough write-up.

---

Powered by: Copilot CLI (claude-opus-4.8) | Last updated: Tue, 14 Jul 2026 15:01:56 GMT

[github-actions]

📊 Coverage Check

No source files changed in this PR.

[Copilot]

Pull request overview

Updates the repository’s Copilot CLI GitHub Actions workflows to use installation authentication (via GITHUB_TOKEN, avoiding the personal-token COPILOT_GITHUB_TOKEN path) and adds an ordered model fallback chain to reduce CI flakiness when a preferred model is unavailable. This is CI/automation-only and primarily affects reliability and billing/auth behavior of Copilot-powered workflows.

Changes:

• Switched Copilot CLI invocations in several workflows from COPILOT_GITHUB_TOKEN=... copilot ... to copilot --yolo ... with GITHUB_TOKEN installation auth.
• Added a space-separated, ordered model fallback loop (defaulting to claude-opus-4.8 → claude-opus-4.6 → gpt-5-mini).
• Removed the redundant copilot-pr-validation.yml workflow and adjusted the Copilot CLI smoke workflow to be workflow_dispatch-only with a non-colliding job name.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
.github/workflows/pr-ai-validation.yml	Switches PR validator Copilot CLI call to installation auth + model fallback while preserving Logic App fallback path.
.github/workflows/copilot-test-cli.yml	Makes CLI smoke test dispatch-only, adds model fallback input, renames job to avoid check-name collisions.
.github/workflows/copilot-pr-validation.yml	Deleted redundant PR validation workflow to avoid duplication/collisions.
.github/workflows/copilot-investigate.yml	Switches investigation workflow to installation auth + model fallback.
.github/workflows/copilot-flake-review.yml	Switches flake-fix workflow to installation auth + model fallback.
.github/workflows/copilot-design-pass.yml	Switches design-pass workflow to installation auth + model fallback.