Skip to content

chore(drift-sync): needs-human model-family decision (2026-11-13) - #616

Closed
copilotkit-bot[bot] wants to merge 1 commit into
mainfrom
drift-needs-human/rolling-2026-11-13-43378372062
Closed

copilotkit-bot[bot] wants to merge 1 commit into
mainfrom
drift-needs-human/rolling-2026-11-13-43378372062

Conversation

@copilotkit-bot

Copy link
Copy Markdown
Contributor

Needs a human decision (drift-sync)

The deterministic, zero-LLM drift-sync found a model-family change it must
NOT auto-apply (a genuinely new/unclassified family, or a registry
structural mismatch). It wrote the note file(s) below and opened this PR so
the decision is REACHABLE in the repo.

This is the ONE open needs-human PR. A later run whose family set differs
rewrites this PR's branch and this body in place instead of opening another.
A commit by anyone but the drift bot on this branch stops those rewrites.

Not auto-merged — a human decides, and applies it by hand.
For a new-family note, DO NOT expect merging this PR to apply anything:
includeFamilies is checksum-pinned, and adding a family moves that pin,
which reds the freeze test gate-2 re-runs. drift-sync cannot re-pin — the
gate-1 allowlist permits only model-registry.ts and drift-proposals/.
So a follow-up run would report gate-failed, not ok-applied.

The procedure that actually works, in ONE commit: add the family to
includeFamilies (or excludeFamilies to reject), re-pin BOTH checksums,
resolve the note file(s) below, and update the offline /models wave.
Then close this PR. Prior examples: 72f85f8, 936b59c.

Note file(s) in this PR

  • drift-proposals/gemini-gemini-4-hydrogen-new-family.md

drift-sync outcome

npm warn Unknown project config "minimum-release-age". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown project config "block-exotic-subdeps". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
needs-human note(s) written this run — routed to human without a live re-collect (no registry edit to re-verify)
  [needs-human-new-family] gemini/gemini-4-hydrogen: "gemini-4-hydrogen" is unclassified — routed to human (drift-proposals/gemini-gemini-4-hydrogen-new-family.md)
npm warn Unknown env config "block-exotic-subdeps". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown env config "minimum-release-age". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown project config "minimum-release-age". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
npm warn Unknown project config "block-exotic-subdeps". This will stop working in the next major version of npm. See `npm help npmrc` for supported config options.
�[33m[STARTED]�[39m Backing up original state...
�[32m[COMPLETED]�[39m Backed up original state in git stash (29fe247)
�[33m[STARTED]�[39m Running tasks for staged files...
�[33m[STARTED]�[39m package.json�[0;90m — 1 file�[0m
�[33m[STARTED]�[39m *.{ts,mts,js,mjs,cjs,json,html,css,md}�[0;90m — 1 file�[0m
�[33m[STARTED]�[39m *.{ts,mts,js,mjs}�[0;90m — 0 files�[0m
�[33m[SKIPPED]�[39m *.{ts,mts,js,mjs}�[0;90m — no files�[0m
�[33m[STARTED]�[39m prettier --write
�[32m[COMPLETED]�[39m prettier --write
�[32m[COMPLETED]�[39m *.{ts,mts,js,mjs,cjs,json,html,css,md}�[0;90m — 1 file�[0m
�[32m[COMPLETED]�[39m package.json�[0;90m — 1 file�[0m
�[32m[COMPLETED]�[39m Running tasks for staged files...
�[33m[STARTED]�[39m Applying modifications from tasks...
�[32m[COMPLETED]�[39m Applying modifications from tasks...
�[33m[STARTED]�[39m Cleaning up temporary files...
�[32m[COMPLETED]�[39m Cleaning up temporary files...
[fix/drift-2026-11-13-43378372062 38993a4] fix(drift-sync): mechanical model-family sync (needs-human note file(s))
 1 file changed, 20 insertions(+)
 create mode 100644 drift-proposals/gemini-gemini-4-hydrogen-new-family.md
unchecked-providers=
reason=needs-human
changeset-key=35c4c44fc26cdc03

@pkg-pr-new

pkg-pr-new Bot commented Nov 13, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@616

commit: 38993a4

jpr5 added a commit that referenced this pull request Nov 13, 2026
The scheduled Drift Tests run 43376684576 flagged gemini/gemini-4-hydrogen
as an unclassified family in the live /models canary (needs-human PR
#616).

Classified INCLUDE on the live listing and a live probe, following the
gemini-4-argon precedent: GA displayName "Gemini 4 Hydrogen" with no
preview, EAP or confidential marker; supportedGenerationMethods
[generateContent, countTokens, batchGenerateContent]; a text
generateContent call returned 200 with a TEXT-modality text part and
finishReason STOP, the same shape as gemini-4-argon in the same run.

- includeFamilies.gemini entry with evidence, and its checksum re-pin
- Gemini static /models wave and behavioural coverage with negative
  controls
- drift-proposals note resolved: Decision INCLUDE

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jpr5 added a commit that referenced this pull request Nov 13, 2026
## What

Classifies the new Gemini family `gemini-4-hydrogen` as **INCLUDE**
(`includeFamilies.gemini`), so the live `/models` family canary stops
reporting it as unclassified.

Supersedes #616 (the Fix Drift needs-human PR, which only added the
proposal note with `Decision: pending`).

Fixes the critical in scheduled Drift Tests run
[43376684576](https://github.com/CopilotKit/aimock/actions/runs/43376684576)
(Thu Nov 12, 10:09 PM PST, main `78342949`):
`Google Gemini model-family availability — live /models contains no
unclassified family` → `models/gemini-4-hydrogen`.

## Why INCLUDE

The approved rule: include when the live entry has no
preview/EAP/`[Confidential]` marker, lists `generateContent`, and a text
generateContent call returns 200 with text. This matches the
`gemini-4-argon` precedent. Otherwise exclude, as with `gemini-4-carbon`
(#614). Hydrogen meets every include condition. Google's Cloud guide
also calls it a "short-term generally available (GA) release", with no
allowlist.

## Probe evidence (redacted)

There was no Gemini key on the local machine, so the probe ran in CI.
Commit `0b8ac15b` added a temporary diagnostic to `models.drift.ts`. It
wrote a key-free summary to `drift-evidence/` in the drift-live-pr head
artifact (run
[43472488931](https://github.com/CopilotKit/aimock/actions/runs/43472488931),
Fri Nov 13, 9:27 AM PST). Commit `5acf0bb8` then reverted it, with no
force-push. The artifact was checked for key material, and none was
found.

`GET /v1beta/models/gemini-4-hydrogen` → **200**
```json
{
  "name": "models/gemini-4-hydrogen",
  "version": "3.0",
  "displayName": "Gemini 4 Hydrogen",
  "description": "Gemini 4 Hydrogen",
  "inputTokenLimit": 1048576,
  "outputTokenLimit": 65536,
  "supportedGenerationMethods": ["generateContent", "countTokens", "batchGenerateContent"],
  "thinking": true
}
```
- The paged `/models` list contains `models/gemini-4-argon`,
`models/gemini-4-carbon` and `models/gemini-4-hydrogen`.
- `POST /v1beta/models/gemini-4-hydrogen:generateContent` ("Reply with
the single word: pong") → **200**. The response has the text part
`"pong"`, `finishReason: "STOP"`, `modelVersion: "gemini-4-hydrogen"`
and `candidatesTokensDetails: [{modality: "TEXT"}]`.
- Control: `gemini-4-argon` (already included) returned the same shape
in the same run: 200, STOP, TEXT.
- The entry has no `bidiGenerateContent`, `embedContent` or `predict`,
so it is not a voice/audio family and needs no `knownVoiceModelFamilies`
edit.

## Changes (copied from the #610 include and #614 exclude shapes)

- `src/__tests__/drift/model-registry.ts`: the `gemini-4-hydrogen` entry
in `includeFamilies.gemini`, with an evidence comment.
- `src/__tests__/drift/logic-pin.test.ts`: the `includeFamilies.gemini`
checksum re-pin (`2ce8c838…` → `7d064a63…`, cross-checked by recomputing
sha256 of the sorted members).
- `src/__tests__/drift/models.drift.ts`: the static Gemini `/models`
wave entry.
- `src/__tests__/drift/text-drift.test.ts`: the 2026-11-13 behavioural
block, with negative controls (`gemini-4-helium` and
`gemini-4-hydrogen-tts` are still reported).
- `drift-proposals/gemini-gemini-4-hydrogen-new-family.md`: resolved
with `Decision: INCLUDE` and the rationale.

## RED / GREEN (real surface: live Gemini `/models` canary)

**RED** (before the fix, on main code):
- Scheduled run 43376684576 report: a critical `Unclassified model
family "gemini-4-hydrogen" in gemini /models`, path
`models/gemini-4-hydrogen`.
- drift-live-pr run 43472488931 (diagnostic head, main classification
code): the base report (fresh live, 9:27 AM PST) and the head report
both contain the same `models/gemini-4-hydrogen` critical.

**GREEN** (after the fix):
- drift-live-pr run
[43473964234](https://github.com/CopilotKit/aimock/actions/runs/43473964234)
on head `2017bfdb`: the head report (9:39 AM PST) has **zero**
`hydrogen` mentions and no Gemini `/models` family-canary entry. The
base report from the same run (fresh live main, 9:36 AM PST) still has
the `models/gemini-4-hydrogen` critical, so the canary fired live in the
same run and the fix cleared it. drift-live-pr: pass.
- The head report has 6 remaining criticals, all native-authentication
value divergences (`Anthropic native authentication and version`:
status/type/error.type; `Gemini native authentication`:
status/error.code/error.status). They are identical in the base report
and in scheduled run 43376684576, they predate this PR, and this PR does
not touch them.

Local, non-live:
- `vitest run src/__tests__/drift/`: 20 files, 256 passed, 6 skipped.
- `pnpm typecheck`: clean.
- eslint and `prettier --check` on the changed files: clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@jpr5

jpr5 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Superseded by #617, which classifies gemini-4-hydrogen as INCLUDE based on a live listing and probe (GA, text generateContent, same shape as gemini-4-argon).

@jpr5 jpr5 closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant