Comprehensive Normalization. - #3655
Open
404oops wants to merge 14 commits into
Open
Conversation
Introduces a new `normalize` option for chat completions, plus release-date based default normalization (post-2026-09-01) to coerce provider-native outputs into a consistent OpenAI-style shape. Adds shared normalization utilities, extensive driver/provider consistency tests, and controller safeguards that pin provider-native output where route-specific translators are used. Also wires the option through puter.js (`chat` options and `puter.ai.normalize` default), updates AI/chat response docs and examples, and resolves related TypeScript typing issues reflected in the typecheck baseline.
Adds the upstream Hoonify provider (feat #3499, merged after this branch was cut) to the cross-provider output conformance suite. It speaks the OpenAI chat-completions dialect and conforms out of the box. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s through
Closes the reasoning gaps left open by the normalization work.
Reasoning replay. The coercer dropped Anthropic thinking-block signatures and
the Responses handler dropped reasoning item ids/encrypted_content, so a
normalized reasoning turn could not be replayed — Anthropic rejects an
extended-thinking tool-use continuation whose thinking blocks lost their
signature. Both now ride `message.reasoning_details` verbatim, and both input
paths accept them back: ClaudeProvider splices the blocks ahead of the content
(Anthropic requires them to lead), and the Responses input processor expands
them into standalone `reasoning` items. Output-only fields a replayed message
carries (`reasoning`, `refusal`, `normalized`) are stripped on both paths,
since neither upstream accepts them. The docs caveat recommending
`normalize: false` for agentic Claude loops is gone; it is no longer true.
Unmapped stop reasons. chatresponse.md promised a vendor `finish_reason` with
no OpenAI analog "passes through unchanged" — true for the Mistral remap, false
for the Anthropic coercer, which discarded it. Anthropic's `pause_turn` means
"continue this turn", so flattening it to `stop` destroyed the signal. The
coercer now passes unmapped values through verbatim, matching both the doc and
the Mistral path, and the docs gain the full Anthropic stop-reason table.
Reasoning summaries. Multi-part summaries joined with '' instead of a blank
line, and the streaming Responses path emitted no reasoning at all;
`response.reasoning_summary_text.delta` now feeds the same `reasoning` stream
channel the chat-completions handler uses.
Types. `text?: string & { verbosity?: ... }` was an uninhabitable intersection
(providers read `text?.verbosity` as an object), and the verbosity enum was
`'concise' | 'detailed'` where OpenAI accepts `'low' | 'medium' | 'high'`.
Adds `reasoning`, `reasoning_details`, and `refusal` to the SDK ChatMessage
typedef.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rules from the post-normalization review: don't silently change what existing models return, verify doc claims against every code path they cover, record self-disclosed defects in the PR draft, finish with a fresh build + suites, and check git stash before concluding an edit was lost. Plus the standing typecheck-baseline no-go. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`magistral-*` returns `message.content` as a ContentChunk[] rather than a string, with the thinking text nested one level deeper inside `thinking` chunks. The camelCase remap did not touch it, so a non-streamed magistral response reached the caller as an array with no `reasoning` — the one case left where a provider did not produce the equalized shape this branch promises. Streaming had the matching bug: the chunk array was handed to addText, which would have stringified it into the text stream. Both paths now split chunked content into a string `content` plus a `reasoning` string, joining multiple thinking chunks with a blank line as the Responses handler and the Anthropic coercer do. The streaming fix rides the existing Mistral-only `chunk_but_like_actually` hook, so no new deviation is introduced. The conformance matrix had no Mistral reasoning fixture, which is why it missed this; it now has one carrying chunked content, verified to fail without the flattening. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Commit 6b4f5b0 dropped 31 entries (49 error instances) from tools/typecheck-baseline.json as a side effect of fixing the AI-code type errors they suppressed. The file is a ledger the maintainer owns, and shrinking it inside a feature PR ships a CI-gate change nobody asked for. Restoring is free: tools/typecheck.mjs fails only on *regressions* (count > baseline), so stale entries are non-fatal — the gate prints "49 baselined error(s) fixed. Run npm run typecheck:update to lock that in." and exits 0. Nothing else in the repo reads the file, vitest included, and tsconfig.build.json's noCheck:true means emit is unaffected. The errors stay fixed in the source either way; only the ledger's own bookkeeping is deferred to whenever the maintainer chooses to regenerate it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ing caller messages Acts on a triple-check audit of this branch. Gate the Mistral dialect remap. The camelCase→snake_case rewrite and the chunked-content flattening were firing for every Mistral call regardless of `normalize` or the cutoff, deleting `finishReason` and `message.toolCalls` out from under any caller reading them. Both now sit behind the policy resolution the driver already used, extracted as `shouldPresentAsOpenAI` so the provider and the driver cannot drift. The streaming chunk-array split stays ungated: handing an array to `addText` is a plain bug, and streamed chunks are provider-uniform by design. The conformance matrix now passes `normalize: true`, which is the contract it was always testing. Unify the reasoning join. Three code paths produced two separators while one doc sentence described them all: the coercer joined thinking segments with '', the Responses handler and Mistral with '\n\n'. The coercer now matches, and chatresponse.md's claim is true for every path it covers. Text blocks still join with '' — Anthropic splits prose mid-sentence across them. finish_reason is an open set. chat.md's normalize bullet and the SDK ChatMessage typedef still declared a closed four-value set, contradicting the documented pass-through of unmapped vendor reasons and the coercer that implements it. Stop mutating caller messages. Both reasoning-replay input paths deleted output-only fields from the caller's own message objects, which the driver reuses across fallback attempts. Both strip a copy now; tests pass a frozen message through each. Drop three dead things the type cleanup left: the no-op ChatProvider checkModeration stub (no subclasses, no callers — its removal restores a pre-existing baselined TS2420), the redundant second normalizeReasoningContent call in BytePlus and ZAI, and the coercer's bare-string branch that no provider reaches. A bare string now passes through by reference instead of being coerced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rrect four doc claims Acts on a second triple-check audit. `normalized` means one thing. The driver stamped it on both the OpenAI-shape path and the legacy `response.normalize` path, which converts toward Anthropic blocks — so a caller could get `normalized: true` alongside array content, and the flag told them nothing they could branch on. The legacy branch no longer sets it. That branch is reachable only by a direct driver call with `response.normalize` and no `normalize`; the four wire routes pin `normalize: false`, which skips it. Mistral streaming, split by concern. Flattening a reasoning model's chunked `delta.content` to a string is a correctness floor and stays ungated — the shared handler passes the value straight to `addText`, so an array reaches the caller as stringified objects. Splitting the thinking text out into a `reasoning` delta is the dialect change and now sits behind the policy gate like the non-streaming remap. On the native path the thinking text is kept inline rather than dropped. Mistral `finishReason` is deleted only once its value carried over. The delete ran unconditionally, so a non-string `finishReason` with no `finish_reason` left the choice with no finish reason at all. Four doc claims corrected against the code paths they cover: `content` is string-or-null on normalized responses (tool-only turns carry no text, and the `// always a string` example comment was wrong); `reasoning_details` is not scoped to normalized responses, since Responses models emit it either way; and the release-date rule depends on the serving provider's own dates — OpenRouter derives them from its live API, so models newly listed there from 2026-09-01 normalize by default. Adds the test the copy-on-write fix was actually for: one messages array sent through two sequential calls, asserting the caller's array is untouched and both attempts carried the thinking signature. That is the fallback hazard; the harness wires one provider per model, so the fallback loop itself cannot be driven from a provider test. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This reverts commit f220276.
…conditionally Third triple-check round. Ten findings were put through independent skeptics first; six did not survive — pre-existing on main, inert, or resting on a false premise — and are not acted on here. Mistral streamed thinking now goes to `reasoning` on every path. The previous commit gated the split, which made this the only place in the repo where chain-of-thought reached the visible text channel, and made Mistral the only provider whose streamed chunk *types* depend on a response-format flag. Every other reasoning path routes thinking to `reasoning` unconditionally — ClaudeProvider's thinking_delta, the DeepSeek/OpenRouter rename, and this branch's own Responses summary-delta handler. Removing the gate restores that uniformity and makes the documented promise that streaming is unaffected by normalization true again; the two opposing tests collapse into one that runs the same fixture with and without `normalize` and asserts identical event streams. Docs stop claiming older models are unchanged. Four reasoning fields were made consistent across all models, ungated, and one of them removes a field: on non-streaming responses `message.reasoning_content` is now `message.reasoning`. chat.md gains a table naming all four so a caller reading `reasoning_content` learns why it disappeared, instead of reading that nothing changed for them. Adds the driver-level fallback test. Writing it surfaced that the invariant it was meant to assert is false and always was: the driver rewrites string `content` into text blocks in place on the caller's own messages (`normalize_single_message`, pre-existing) before any provider runs. The test now asserts what is true and load-bearing — both attempts receive the same array reference, and the reasoning artifacts survive attempt 1 so attempt 2 can still replay them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lback test The comment claimed the fallback loop cannot be driven from a test — false since ChatCompletionDriver.test.ts gained a two-attempt fallback test that drives the actual loop. Say where that test lives instead. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
…-date policy Resolves the flag's semantics per the requester: the SDK-wide flag is two-state and outward-facing — `true` (the default) means the release-date policy applies, so models released on or after 2026-09-01 are normalized and older models keep their vendor-native shape; `false` disables normalization for every call. The tri-state lives only on the per-call `normalize` option: unset defers to the SDK-wide flag, `true` forces the OpenAI shape regardless of release date, `false` forces the vendor-native shape. On the wire nothing changes: the default sends no `normalize` key (the server's policy resolution decides), and only an explicit SDK-wide `false` or a per-call value is transmitted. Consequences worth noting: assigning `true` now restores the date policy rather than force-normalizing everything (previously there was no boolean that meant "give me the policy back"), and SDK-wide force-all is no longer expressible — force-all is per-call only, as specified. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
… thinking
Live-probing magistral-small-latest showed the model inlines its reasoning as
answer prose in a flat string — no ThinkChunk content, no markers, nothing a
client can separate. Mistral's chunked thinking shape is requested via
`prompt_mode: 'reasoning'`, which the provider previously dropped on the
floor: there was no way to even ask for it.
`custom.prompt_mode` now forwards to the SDK's `promptMode`, following the
BytePlus custom-params precedent. Opt-in rather than a default because the
API rejects the mode where the account/model lacks it ('Reasoning prompt
mode is not enabled for this model', code 3051) — verified end-to-end: the
3051 travels back through the stack, which also proves the parameter is
delivered. The moment Mistral enables the mode, the ThinkChunk content flows
into the existing splitter and comes out as `message.reasoning` and
`reasoning` stream chunks with no further changes.
Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
A live audit of every keyed provider's /models endpoint against the hardcoded catalogs found no stale entries but large gaps. This backfills them under four rules: nothing vendor-deprecated, nothing without a price confirmed on the vendor's official pricing page (each entry's source was recorded during review), nothing absent from the live /models listing, and nothing that fails a live routing probe. Added: 46 Alibaba entries (qwen3/3.5/3.7/3.8 families, VL/omni/MT lines, and Model Studio's hosted GLM/DeepSeek/Kimi third-party models) plus 9 dated aliases; OpenAI chat-latest and gpt-4o-2024-11-20 plus 16 snapshot aliases; Gemini gemma-4-31b-it and gemma-4-26b-a4b-it (vendor-documented free tier) plus rolling -latest aliases; Mistral-hosted zai-glm-5-2 and a mistral-medium-3.5 alias; deepseek-v4-flash-vision-exp; glm-5.3-flash. Culled by the rules: 15 vendor-deprecated OpenAI entries (the 3.5/4/4-turbo legacy line, gpt-4o-2024-05-13, o1-pro, four chat-latest predecessors, the 5.x codex line — deprecations page, most shut down 2026-10-23) and dated aliases onto the deprecated o1/o3-mini/o4-mini; qwen3-vl-flash-2025-10-15 (live routing probe returned upstream 400 twice). Tiered Alibaba prices are encoded at the base tier and busy-hour rates where time-of-day priced, noted in comments. Every surviving addition was verified end-to-end through a local deployment: 58/58 answered a live prompt, including all 46 Alibaba entries and every spot-checked alias. Not changed, flagged for maintainers: pre-existing o1, o3-mini and o4-mini entries are now vendor-deprecated (shutdown 2026-10-23); the pre-existing deepseek-v4-flash/-pro prices no longer match DeepSeek's current pricing page; gemma-4 emits its own <thought> markup inline in content. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
Contributor
Coverage Report for puter.js SDK
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Comprehensive Normalization.
This PR makes Puter's chat API return one response shape — the OpenAI
choices[0]format — across all vendors, on an opt-in/opt-out basis. Callers get a new tri-statenormalizeoption (per call and SDK-wide viaputer.ai.normalize); models released on or after 2026-09-01 are normalized by default, while older models keep their vendor-native shape unless the caller asks otherwise. Normalization is lossless for reasoning models: Anthropic thinking-block signatures and OpenAI Responses reasoning items are preserved on a newmessage.reasoning_detailsfield and can be replayed verbatim to continue an extended-thinking tool-use turn. Alongside the gated work, a small set of unconditional reasoning-field equalizations ships for existing models (documented inchat.md), and the provider catalogs get ~53 new model entries and ~23 new aliases. A 23-provider consistency test matrix pins the equalized contract.Changes by area
Response normalization core
src/backend/drivers/ai-chat/utils/normalizeToOpenAI.ts(new) — the whole policy in one module:OPENAI_SHAPE_CUTOFF = '2026-09-01',isPostCutoffRelease(timestamp compare, tolerant of month-precision dates, missing/unparseable ⇒ pre-cutoff),needsOpenAICoercion(Anthropic envelope or content-block array),shouldPresentAsOpenAI(the single precedence rule), andnormalizeResultToOpenAI— an idempotent coercer that joins text blocks into stringcontent, convertstool_use→tool_calls(arguments stringified), joinsthinkingintoreasoning(blank-line separated), preservesthinking/redacted_thinkingverbatim inreasoning_details, maps Anthropicstop_reason→ OpenAIfinish_reason, and passes unmapped vendor reasons (e.g.pause_turn) through verbatim. Already-OpenAI-shaped results return by reference.ChatCompletionDriver.ts— applies the coercer to non-streaming message results pershouldPresentAsOpenAI(args, model.release_date), wheremodelis the model that actually served the request (fallback/reroute included). Stampsnormalized: trueonly on OpenAI-shaped output.response.normalizepath (Anthropic-block normalization) no longer stampsnormalized: true— the field now exclusively means "OpenAI shape".PuterAIController.ts— all four wire-compat routes (/openai/chat/completions, completions, responses,/anthropic/messages) pinnormalize: false, so the release-date cutoff can never change what those routes' own translators receive.types.ts—ICompleteArguments.normalize?: booleandocumented as the tri-state.Per-provider equalization
MistralAiProvider.ts— (1) Gated on the policy: non-streaming remap of the SDK's camelCase dialect —finishReason→ mappedfinish_reason(unmapped values pass through),toolCalls→ OpenAItool_callswith stringified arguments, and magistral chunkedcontentarrays split into stringcontent+reasoning. Without the flag / pre-cutoff, callers keep seeing the SDK-native keys. (2) Unconditional: the streaming chunk splitter — chunkeddelta.contentis split into text andreasoningdeltas regardless of policy (previously the raw array reachedaddTextand stream consumers got stringified objects; streamed chunk types must not depend on a response-format flag). (3) New opt-incustom.prompt_mode→ SDKpromptModepassthrough.OpenAIUtil.jshandle_completion_output—normalizeReasoningContent(ret)now runs for every chat-completions provider, unconditionally:reasoning_content(DeepSeek wire convention) is renamed toreasoningand the vendor key deleted on all non-streaming responses. This removes a field from existing models and is not behind the normalize policy — the one field removal, called out in the docs.OpenAIUtil.jshandle_completion_output_responses_api— three unconditional changes to existing OpenAI/Azure Responses-API models:finish_reasonis now'tool_calls'on tool turns (was hardcoded'stop');reasoning: nullis replaced by string-or-absent (populated from reasoning summaries);reasoning_details(itemid,encrypted_content,summary) is added for replay.create_chat_stream_handler_responses_apinow routesresponse.reasoning_summary_text.deltaevents to thereasoningchannel (blank line between summary parts). Unconditional and additive: those deltas were previously dropped.Reasoning round-trip / replay
ClaudeProvider.ts— inbound messages carrying round-tripped artifacts are handled:thinking/redacted_thinkingblocks fromreasoning_detailsare spliced back ahead of the content array (Anthropic requires leading thinking blocks with intact signatures), and output-only fields (reasoning,refusal,reasoning_details) are stripped — on a copy, because the driver reuses onemessagesarray across fallback attempts. The copy-on-write invariant is pinned by tests at both the provider level (frozen objects, double-send) and the driver level (fallback attempts share the array by reference).OpenAIUtil.jsprocess_input_messages_responses_api— the mirror for the Responses API:reasoning_detailsitems expand into standalonereasoninginput items preceding their message, output-only fields (reasoning,refusal,normalized) stripped via destructuring rebind (caller objects untouched).SDK surface (puter-js)
modules/ai/index.js—puter.ai.normalize = truedefault. Note the asymmetry: SDK-widetruemeans "the release-date policy" (nothing sent on the wire), while per-calltruemeans "force OpenAI shape". Only SDK-widefalserides the wire (normalize: falseon every call). Per-call always wins.modules/ai/chat.js— the forwarding logic above;modules/ai/types.js— JSDoc fornormalize,finish_reason,normalized,reasoning,reasoning_details,refusal.Catalog backfill (billing-bearing — all unconditional)
alibaba/models.ts, +~1,000 lines): 46 new entries — Qwen Plus/Max/Flash dated snapshots, Qwen3 2507 open-weight refreshes, coder, VL, Omni, MT translation, plus 8 third-party models hosted on Model Studio (GLM-5.1/5.2/5.2-fast-preview, DeepSeek V3.2/V4 Flash/Pro + dated variants, Kimi K2.7 Code) at Alibaba's own rates. 4 dated-snapshot aliases added to existing entries.chat-latest,gpt-4o-2024-11-20).gemini-{flash,flash-lite,pro}-latestaliases (×2 spellings each) pinned to the newest catalog entry, + 2 free Gemma 4 entries.deepseek-v4-flash-vision-exp. Mistral:zai-glm-5-2(Mistral-hosted) + amistral-medium-3.5spelling alias. Z.AI:glm-5.3-flashat list price (50% promo deliberately not encoded).Backfill rules enforced: nothing vendor-deprecated (15 already-written OpenAI entries were culled against the deprecations page), nothing without a price confirmed on the vendor's official pricing page, nothing absent from the vendor's live
/modelslisting, and nothing that fails a live routing probe (one entry removed on that rule).Docs
chat.md— a full "Response Normalization" section: the flag, the cutoff, the SDK-wide switch, the served-model caveat, the OpenRouter live-release-date caveat, streaming unaffected, replay guidance, and a before/after table for the four unconditional reasoning-field changes.chatresponse.md—contentstring-or-array,finish_reasonmapping table (open set),normalized,reasoning,reasoning_details,usage. The Claude cache-control example (docs + playground) switched tonormalize: true+ string content.Tests
New:
providerConsistency.test.ts(23 providers × text/tool/reasoning against mocked upstreams — the equalized-contract matrix),normalizeToOpenAI.test.ts(21 cases), plus additions to the driver (12, including full precedence coverage and the fallback shared-array invariant), Claude (4), Mistral (6), OpenAIUtil (9), puter-js unit (7), and one live API-suite test. Controller tests grewnormalize: falseassertions.Type/lint cleanup
Annotation-only changes across ~12 files (casts, callback params, return types,
context.tstyping the pre-existingdriverNamekey). Two slightly more than formatting:ICompleteArguments.text/verbosityretyped from'concise' | 'detailed'to'low' | 'medium' | 'high'(matches what vendors accept; compile-time only), and the xAI STTBlobconstruction now copies the buffer into aUint8Array(type fix; runtime copy, bytes identical).Behavior contract after this PR
Precedence for non-streaming results, resolved in
shouldPresentAsOpenAI(used identically by driver and Mistral):normalize: true⇒ OpenAI shape, any model.normalize: false⇒ provider-native, any model.response.normalize(internal) ⇒ Anthropic-block normalization, suppressing the OpenAI presentation (and no longer stampingnormalized).release_date >= 2026-09-01⇒ OpenAI shape; earlier, missing, or unparseable ⇒ native. The served model decides — a fallback or content-block reroute to a date-less model yields native shape even if the requested model was post-cutoff.SDK:
puter.ai.normalizedefaults totrue= "the policy" (nothing on the wire); onlyfalseis transmitted; per-call overrides both ways.Normalized responses: string-or-null
message.content,message.tool_calls,refusal: null, optionalreasoning/reasoning_details, mappedfinish_reason(end_turn/stop_sequence→stop,max_tokens→length,tool_use→tool_calls,refusal→content_filter, unmapped values verbatim — treat as an open set),normalized: true,usageuntouched (key names remain provider-specific by design). Native responses are byte-for-byte what the provider path returned.Streaming is never touched by
normalize(the driver returns stream results before the coercion branch, test-pinned); the two unconditional streaming changes (Mistral chunk split, Responses reasoning-summary routing) exist precisely so chunk types don't depend on the flag.Provider spec audit
The precondition for this work was to establish which providers already speak
the OpenAI format and which are out of spec. Result: 20 of 23 needed no change
at all, because they funnel through
OpenAIUtil.handle_completion_outputandalready return
choices[0]-shaped results.The table has 21 rows for the 23 entries the conformance matrix drives: openai and
azure each register two providers, one per wire API.
api.openai.com/v1)config.apiURLhttps://api.meta.ai/v1https://generativelanguage.googleapis.com/v1beta/openai/https://dashscope-intl.aliyuncs.com/compatible-mode/v1https://ark.ap-southeast.bytepluses.com/api/v3https://api.deepseek.comreasoning_contentrenamed centrallygroq-sdkdefault (api.groq.com/openai/v1)https://api.hoonify.ai/v1https://llm.onerouter.pro/v1https://api.minimax.io/v1https://api.moonshot.ai/v1https://api.neuralwatt.com/v1http://localhost:11434/v1(configurable)https://openrouter.ai/api/v1release_dateis live-derived (see above)together-aiSDK default (api.together.xyz/v1)https://api.x.ai/v1https://api.z.ai/api/paas/v4@mistralai/mistralaidefault (api.mistral.ai/v1)@anthropic-ai/sdkdefault (api.anthropic.com/v1)stop_reason— the coercer's whole reason for existingMeta was the suspected edge case going in. It is not one: Meta
ships its own OpenAI-compatible layer at
api.meta.ai/v1and the providerdrives it through the standard OpenAI SDK, which is why Meta appears nowhere in
this diff. Anthropic's own OpenAI-compat beta endpoint was evaluated and
rejected for Claude — routing through it would cost prompt-caching control,
compaction, and the cache-token usage detail metering bills on. Hence a central
coercer rather than per-provider endpoint swaps.
Risk notes
reasoning_content→reasoningrename on all non-streaming chat-completions responses — a field removal; (2) Responses-APIfinish_reasonnowtool_callson tool turns; (3) Responses-APIreasoningnull → absent-or-string ('reasoning' in msgchanges); (4) Responses-APIreasoning_detailsadded; (5) Mistral streaming chunk split; (6) Responses streaming reasoning-summary deltas now emitted; (7) legacyresponse.normalizepath no longer setsnormalized: true. All are documented and bug-fix-shaped, and each was explicitly approved during review rather than shipped silently — but none is behind the policy, so sign off on them consciously.compareModelPreference): the Alibaba-hosted third-party entries reuse the direct vendors' ids, so they merge into shared routing buckets — which is this repo's deliberate vendor+reseller fallback design. Concretely:deepseek-v4-flash/-proandkimi-k2.7-codekeep the direct vendor primary on the cost tiebreak (Alibaba becomes a more expensive fallback route);glm-5.2ties Z.AI exactly (same price, same id length), so the serving provider falls to provider-registration order. Billing-neutral either way — both routes serve the same upstream model at the same price — but reviewers should confirm the fallback-bucket effect is wanted.glm-5.3-flashignores the current 50% promo (over-bills, deliberate); qwen3-30b-a3b thinking-mode output rate not modeled (under-bills); Gemma 4 entries are $0 per Google's pricing page.gpt-5.4-nanorelease2026-03-19vs alias...-2026-03-17(pre-existing quirk);qwen3.6-flashrelease2026-04-27vs alias...-2026-04-16;qwen3.7-plusrelease2026-06-02vs alias...-2026-05-26(snapshot-name-vs-GA-date is plausible, but worth confirming). New OpenAI entry with bare idchat-latest— verified live: the upstream API accepts exactly that id.gemini-*-latestandqwen-plus-latestare hot-swapped upstream; Puter's pinning goes stale (and mis-prices) the day the vendor moves them — a maintenance obligation, acknowledged in comments.normalize = truemeans "policy", per-calltruemeans "force" — correct and tested, but easy for users to misread.o1,o3-mini,o4-minientries are now vendor-deprecated (shutdown 2026-10-23) — untouched here since they predate this work; the pre-existingdeepseek-v4-flash/-proprices no longer match DeepSeek's current pricing page (billing drift found during review, not introduced or changed here); gemma-4 emits its own<thought>markup inline in content (model behavior).Known gaps
reasoning_detailsis not portable across vendors. The artifacts areprovider-specific (an Anthropic signature means nothing to OpenAI), so replay
only works against the model that produced them. Documented, not enforced — a
fallback reroute mid-conversation cannot replay a reasoning turn.
each upload buffer into a
Uint8Arraybefore wrapping it in aBlob. Node'sBufferdoes not satisfy the DOMBlobPartsignature, so the old code was atype error and a latent bug; the fix costs one extra in-memory copy per audio
upload. Not part of the normalization work — flagged so reviewers aren't
surprised by a non-chat driver in the diff.
ChatProviderno longer stubscheckModeration. A no-op stub addedduring the type cleanup has been removed: it had no subclasses and no
production callers, and it was suppressing a genuine design smell — the base
class does not implement
IChatProvider. That pre-existingTS2420is backand is covered by
origin/main's baseline entry, so the gate passes.tools/typecheck-baseline.jsonis untouched by this branch. An earliercommit had removed 31 entries (49 error instances) as a side effect of fixing
the AI-code type errors they suppressed; that removal has been reverted. The
errors stay fixed in the source, so
npm run typecheckprints "48 baselinederror(s) fixed" and exits 0. Regenerating the ledger is the maintainer's call.
renameReasoningContentdeletes
message.reasoning_contenteven whenmessage.reasoningwas alreadypresent, so a provider sending both keys loses the vendor one. That looks like
data loss, and changing it was requested during review — but
BytePlusProvider.test.tsandZAIProvider.test.ts, both pre-existing onmain, pin the current behavior deliberately, with a fixture value named'should-be-dropped'. Honouring the request would mean rewriting two teststhis branch does not own and changing BytePlus/ZAI behavior nobody asked to
change, so it was left alone. A reviewer should decide: keep the drop as those
tests specify, or change it and update them.
input paths now strip output-only fields from a copy, so a replayed message's
thinking signature survives a fallback retry. But the driver still normalizes
every inbound message in place before any provider runs
(
normalize_single_messageinutils/Messages.js, pre-existing), rewritingstring
contentinto[{type:'text'}]blocks on the caller's own objects. Thefix protects the reasoning artifacts, not whole-message immutability, and the
new driver-level test says so explicitly.
example (
src/docs/src/playground/examples/ai-claude-cache-control.html) wasswitched to
normalize: truewith stringcontent. Nobody asked for it; itwas changed because its
content[0].textreading goes stale the momentpost-cutoff Claude models normalize by default. Flagged so the diff's
user-facing surface is not a surprise.
earlier push of this branch:
contentis string-or-null on normalizedresponses (tool-only turns have no text),
reasoning_detailsis not scoped tonormalized responses,
finish_reasonis an open set, and the release-datetrigger depends on the serving provider's own dates.
.claude/worktrees/), concentrated insrc/backend/stores. Zero land in any file this branch touches; every file in this diff is clean.Test results
All runs on the final rebased tree (14 commits on top of
origin/main), fresh builds first, exit codes read directly — no output filtering.Automated suites
src/puter-jsbuild (webpack)npm run build:workerLibnpm run test:backend, 274 files)npm run test:puterjs— same tests on node, browser/Playwright, and workerd/miniflare, against the built bundle)vitest --config src/gui/vitest.config.js)npm run typecheck, read-only)tools/typecheck-baseline.jsonbyte-identical tomain)npm run check:puterjs:types)src/backend/drivers/ai-chatanywarnings)New coverage added by this branch: ~90 tests — the 23-provider × text/tool/reasoning conformance matrix (
providerConsistency.test.ts, driven withnormalize: trueso the equalized contract is what's asserted), 25 coercer unit tests, 13 driver precedence/cutoff tests, reasoning round-trip tests on both input paths (thinking-signature preservation sabotage-verified: the test fails if the copy-on-write is reverted), a driver-level two-attempt fallback test that exercises the real fallback loop, Mistral chunk-splitting tests (both paths, bothnormalizevalues asserted identical for streams), Responses-API reasoning tests, SDK flag-precedence tests, and an e2enormalize: truecase that runs in all three API-suite runners.Live end-to-end verification (local Docker deployment, real provider APIs)
The branch was built into the self-hosted Docker image and exercised over HTTP through Caddy against real vendor keys:
Normalization matrix — 25/25 meaningful checks passed:
normalize: true→ string content /finish_reason: stop/normalized: true/refusal: null;normalize: false→ native; tools →tool_callswith stringified arguments,finish_reason: tool_calls,content: nullon the tool-only turn; extended thinking →reasoningstring +reasoning_detailswith signatures; replaying the normalized message verbatim was accepted by the live Anthropic API and answered correctly — the round-trip contract holds against the real vendor.finishReason; force-on maps, cleans camelCase, stampsnormalized).normalized: truestamp only when asked.custom.prompt_modepassthrough was verified delivered end-to-end (Mistral's own3051mode-not-enabled error returns through the stack).Catalog backfill — 58/58 surviving additions answered a live prompt through the deployment (all 46 Alibaba entries, both gemma-4 models,
chat-latest,gpt-4o-2024-11-20,zai-glm-5-2,deepseek-v4-flash-vision-exp,glm-5.3-flash, plus spot-checked snapshot/rolling aliases). The one entry that failed live routing (qwen3-vl-flash-2025-10-15, upstream 400 twice) was removed. A post-edit re-audit of every keyed provider's/modelsendpoint shows zero stale catalog entries across all 11 verifiable catalogs.Numbers
normalize: falseassertions added to 4 existing controller tests; the provider-consistency matrix expands to ~67 runs over 23 providers.normalizeToOpenAI.tsat 227 lines, plus its 309-line test); largest single file change isalibaba/models.ts(+1,031).