Repository navigation
chore(drift-sync): needs-human model-family decision (2026-11-13) - #616
Closed
copilotkit-bot[bot] wants to merge 1 commit into
Closed
copilotkit-bot[bot] wants to merge 1 commit into
copilotkit-bot[bot] wants to merge 1 commit into
Conversation
commit: |
jpr5
added a commit
that referenced
this pull request
Nov 13, 2026
The scheduled Drift Tests run 43376684576 flagged gemini/gemini-4-hydrogen as an unclassified family in the live /models canary (needs-human PR #616). Classified INCLUDE on the live listing and a live probe, following the gemini-4-argon precedent: GA displayName "Gemini 4 Hydrogen" with no preview, EAP or confidential marker; supportedGenerationMethods [generateContent, countTokens, batchGenerateContent]; a text generateContent call returned 200 with a TEXT-modality text part and finishReason STOP, the same shape as gemini-4-argon in the same run. - includeFamilies.gemini entry with evidence, and its checksum re-pin - Gemini static /models wave and behavioural coverage with negative controls - drift-proposals note resolved: Decision INCLUDE Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jpr5
added a commit
that referenced
this pull request
Nov 13, 2026
## What Classifies the new Gemini family `gemini-4-hydrogen` as **INCLUDE** (`includeFamilies.gemini`), so the live `/models` family canary stops reporting it as unclassified. Supersedes #616 (the Fix Drift needs-human PR, which only added the proposal note with `Decision: pending`). Fixes the critical in scheduled Drift Tests run [43376684576](https://github.com/CopilotKit/aimock/actions/runs/43376684576) (Thu Nov 12, 10:09 PM PST, main `78342949`): `Google Gemini model-family availability — live /models contains no unclassified family` → `models/gemini-4-hydrogen`. ## Why INCLUDE The approved rule: include when the live entry has no preview/EAP/`[Confidential]` marker, lists `generateContent`, and a text generateContent call returns 200 with text. This matches the `gemini-4-argon` precedent. Otherwise exclude, as with `gemini-4-carbon` (#614). Hydrogen meets every include condition. Google's Cloud guide also calls it a "short-term generally available (GA) release", with no allowlist. ## Probe evidence (redacted) There was no Gemini key on the local machine, so the probe ran in CI. Commit `0b8ac15b` added a temporary diagnostic to `models.drift.ts`. It wrote a key-free summary to `drift-evidence/` in the drift-live-pr head artifact (run [43472488931](https://github.com/CopilotKit/aimock/actions/runs/43472488931), Fri Nov 13, 9:27 AM PST). Commit `5acf0bb8` then reverted it, with no force-push. The artifact was checked for key material, and none was found. `GET /v1beta/models/gemini-4-hydrogen` → **200** ```json { "name": "models/gemini-4-hydrogen", "version": "3.0", "displayName": "Gemini 4 Hydrogen", "description": "Gemini 4 Hydrogen", "inputTokenLimit": 1048576, "outputTokenLimit": 65536, "supportedGenerationMethods": ["generateContent", "countTokens", "batchGenerateContent"], "thinking": true } ``` - The paged `/models` list contains `models/gemini-4-argon`, `models/gemini-4-carbon` and `models/gemini-4-hydrogen`. - `POST /v1beta/models/gemini-4-hydrogen:generateContent` ("Reply with the single word: pong") → **200**. The response has the text part `"pong"`, `finishReason: "STOP"`, `modelVersion: "gemini-4-hydrogen"` and `candidatesTokensDetails: [{modality: "TEXT"}]`. - Control: `gemini-4-argon` (already included) returned the same shape in the same run: 200, STOP, TEXT. - The entry has no `bidiGenerateContent`, `embedContent` or `predict`, so it is not a voice/audio family and needs no `knownVoiceModelFamilies` edit. ## Changes (copied from the #610 include and #614 exclude shapes) - `src/__tests__/drift/model-registry.ts`: the `gemini-4-hydrogen` entry in `includeFamilies.gemini`, with an evidence comment. - `src/__tests__/drift/logic-pin.test.ts`: the `includeFamilies.gemini` checksum re-pin (`2ce8c838…` → `7d064a63…`, cross-checked by recomputing sha256 of the sorted members). - `src/__tests__/drift/models.drift.ts`: the static Gemini `/models` wave entry. - `src/__tests__/drift/text-drift.test.ts`: the 2026-11-13 behavioural block, with negative controls (`gemini-4-helium` and `gemini-4-hydrogen-tts` are still reported). - `drift-proposals/gemini-gemini-4-hydrogen-new-family.md`: resolved with `Decision: INCLUDE` and the rationale. ## RED / GREEN (real surface: live Gemini `/models` canary) **RED** (before the fix, on main code): - Scheduled run 43376684576 report: a critical `Unclassified model family "gemini-4-hydrogen" in gemini /models`, path `models/gemini-4-hydrogen`. - drift-live-pr run 43472488931 (diagnostic head, main classification code): the base report (fresh live, 9:27 AM PST) and the head report both contain the same `models/gemini-4-hydrogen` critical. **GREEN** (after the fix): - drift-live-pr run [43473964234](https://github.com/CopilotKit/aimock/actions/runs/43473964234) on head `2017bfdb`: the head report (9:39 AM PST) has **zero** `hydrogen` mentions and no Gemini `/models` family-canary entry. The base report from the same run (fresh live main, 9:36 AM PST) still has the `models/gemini-4-hydrogen` critical, so the canary fired live in the same run and the fix cleared it. drift-live-pr: pass. - The head report has 6 remaining criticals, all native-authentication value divergences (`Anthropic native authentication and version`: status/type/error.type; `Gemini native authentication`: status/error.code/error.status). They are identical in the base report and in scheduled run 43376684576, they predate this PR, and this PR does not touch them. Local, non-live: - `vitest run src/__tests__/drift/`: 20 files, 256 passed, 6 skipped. - `pnpm typecheck`: clean. - eslint and `prettier --check` on the changed files: clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor
|
Superseded by #617, which classifies gemini-4-hydrogen as INCLUDE based on a live listing and probe (GA, text generateContent, same shape as gemini-4-argon). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Needs a human decision (drift-sync)
The deterministic, zero-LLM drift-sync found a model-family change it must
NOT auto-apply (a genuinely new/unclassified family, or a registry
structural mismatch). It wrote the note file(s) below and opened this PR so
the decision is REACHABLE in the repo.
This is the ONE open needs-human PR. A later run whose family set differs
rewrites this PR's branch and this body in place instead of opening another.
A commit by anyone but the drift bot on this branch stops those rewrites.
Note file(s) in this PR
drift-proposals/gemini-gemini-4-hydrogen-new-family.mddrift-sync outcome