fix(ai-image): refresh model catalogs, unify txt2img requests, expand docs - #3889
Merged
Merged
Conversation
Model sync against every vendor listing and a live generation sweep: Gemini stable ids replace the retired preview spellings (2.5 Flash Image delisted ahead of its 2026-10-02 shutdown), OpenAI gpt-image-1/-1-mini/-1.5 are delisted but routable until their shutdown dates with gpt-image-2 as the default, xAI gains grok-imagine-image-2.0 with its quality tiers, BytePlus gains the 3K/4K tiers and per-model pixel bounds, Cloudflare gains SDXL Lightning/Base and SD 1.5 Inpainting, Replicate gains a schema-driven catalog of 88 additional models with version-pinned community predictions, and every Together image route is excluded for the third-party data-sharing requirement. Request normalization: the driver collapses ratio/width/height/aspect_ratio into one imageSize with aspect-versus-pixel intent, validates prompt, quality and resolution once, resolves provider hints (short names or full driver ids) and aliases with exact ids winning over resellers, and hands each provider an immutable copy of the caller's args. Providers share prompt validation, aspect snapping, and content-sniffed data URIs so responses carry the right MIME type. Replicate predictions are created once, cancelled on abort or deadline, and bounded in input fan-out. SDK: txt2img copies caller options, rejects blank prompts with prompt_required in every call form, and documents that normalize has no effect because the image return shape is already uniform. Docs: txt2img options per provider, provider defaults and discovery, availability notes, a new image model catalog and pricing page, and the image-generation limits in the quotas page. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
Coverage Report for puter.js SDK
File Coverage
|
||||||||||||||||||||||||||||||||||||||
Contributor
…sions
imageDataUri only decoded the first 24 base64 chars (18 bytes) before
sniffImageMime, which cannot see an <svg> root behind an XML prolog -
SVG outputs from recraft-v*-svg models were being labeled image/png.
Decode enough of the payload to cover the 8 KB SVG sniff window.
dimension() accepted string number literals ('0x10', '1e3') as if they
were decimal dimensions; restrict to plain decimal notation.
404oops
changed the title
404oops
marked this pull request as ready for review
September 16, 2026 20:24
404oops
requested review from
ProgrammerIn-wonderland,
Salazareo and
reynaldichernando
as code owners
September 16, 2026 20:24
reynaldichernando
requested changes
Sep 17, 2026
The image model catalog and pricing page duplicated the always up to date model directory at developer.puter.com/ai/models. Link to the directory from txt2img-related docs and keep the Together data-sharing exclusion note self-contained.
Resolves conflicts with the hardening change (#3904): - ImageGenerationDriver: keep the driver-scoped metering getter from main and the async onServerStart from this branch. - OpenAI / xAI image providers: keep this branch's 401 guard on a missing actor, then derive the upstream user identifier via upstreamUserIdentifier() from main. - xAI: drop the now-unused assertInputImageString import in favour of toUrlOrDataUri. - Provider tests build the actor fixture with makeActor() so effectiveApp is derived from app, matching how the helper resolves it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ProgrammerIn-wonderland
approved these changes
Sep 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Image-generation maintenance for the standard AI checklist: refresh the model catalogs, normalize how
puter.ai.txt2img()requests are built, and expand the docs. Independent of #3887, which carries the chat/video pieces split off this tree — the two touch disjoint areas.Model catalogs
Every provider's model list was checked against what the vendor actually offers (re-probed on 2026-09-16), and for paid models a live generation sweep across the catalog (114 of 117 succeeded; 3 recorded failures are surfaced when requested):
gemini-3-pro-image,gemini-3.1-flash-image); retired-previewspellings kept as aliases;gemini-3.1-flash-imageis now the default.gemini-2.5-flash-imagestays available until Google's 2026-10-02 shutdown. Token tables and prices updated.gpt-image-1and related ids are deprecated upstream, so they're hidden from listing but still usable until their shutdown dates;gpt-image-2is the new default.grok-imagine-image-2.0with itslow/mediumquality tiers and separateresolutionoption; edit requests now accept up to 5 reference images (previously truncated silently).thirdPartySharingdata policy) because Together rejects all image models unless the org opts into third-party data sharing; requests fail withbad_requestbefore any upstream call and the models are hidden from discovery. Retired ids are blocked so they can't silently route to Cloudflare.Request normalization
What you pass is now what each provider interprets by the same rules: the driver turns
ratio/width+height/aspect_ratiointo oneimageSize, validatesprompt,qualityandresolutiononce, accepts short provider hints ('openai','gemini', …) as well as full driver ids, and hands every provider an immutable copy of your options. Shared helpers standardize prompt checks, aspect snapping, and content-sniffed data URIs so the returned image MIME type is correct on every provider. Replicate predictions are created once, cancelled on abort or deadline, and input fan-out is bounded by count, bytes, concurrency, and time.About
normalize: the image response shape is already uniform across browsers and Node (a single image element, or{ src }), so there is no vendor-specific shape to convert. The SDK doesn't sendnormalizefortxt2img(), and the docs say so.SDK (backward compatible)
txt2img()keeps all three call forms. It now copies your options instead of mutating them (frozen objects work), and rejects missing, blank, or non-string prompts withprompt_requiredin every form. No option, response field, or error code was renamed.Docs
txt2img.mdrewritten per provider with defaults, discovery, availability notes, and output-size rules; new catalog and pricing page atAI/image-models.md; image-generation limits added torate-limits-and-quotas.md.Security notes
No auth or permission changes. Caller-supplied image URLs stay behind
secureFetch; Replicate fetches are capped at 30 MB, 30 s, and 10 references. Error messages name the model and provider only.Verification
npx vitest run --config src/backend/vitest.config.ts src/backend/drivers/ai-image src/backend/drivers/util/imageInput.test.ts(597 passed, 3 skipped)npx vitest run src/puter-js/src/modules/ai/ai.test.js(62 passed)npm run build:workerLib && npm run test:puterjs:node(507 passed, 67 skipped)npm run typecheckclean;npm run check:puterjs:typesclean; eslint clean on touched sourceschatgpt-image-latest, Cloudflaredreamshaper-8-lcm, BytePlusseedream-4-0-20260415(no published price).