You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Feature request: provider-native model listing, required-header validation, and non-chat endpoints (Anthropic / Gemini) #519
We use aimock (1.44.0, strict mode, in a Bun test suite) to test an app that talks to user-configured providers in four formats: OpenAI Chat Completions, OpenAI Responses, Anthropic Messages and Gemini. Generation goes through the Vercel AI SDK and aimock covers it well. The gaps we hit are the calls around generation, which our app makes with plain fetch.
A concrete miss: our model-discovery call to an Anthropic endpoint sent x-api-key but no anthropic-version. The real API answers that with
{"type":"error","error":{"type":"invalid_request_error","message":"anthropic-version: header is required"}}
(HTTP 400), but our aimock-backed tests passed, so we only found it with a real key.
What we would like
Provider-native model listing. Today GET /v1/models returns one OpenAI-shaped list ({object: "list", data: [{id, object: "model", owned_by}]}) to every client, and there is no GET /v1beta/models. It would help to have:
Anthropic GET /v1/models in its own shape (data[{type: "model", id, display_name, created_at}], has_more, first_id, last_id), with limit / after_id paging, selected e.g. by the presence of anthropic-version or x-api-key.
Gemini GET /v1beta/models (models[{name: "models/…", displayName, inputTokenLimit, outputTokenLimit, supportedGenerationMethods}], nextPageToken).
Paging that can be forced from fixtures or options (e.g. a small page size), so clients that ignore has_more / nextPageToken fail in tests.
Required-header validation under strict mode. On Anthropic routes (/v1/messages, /v1/models), reject a request without anthropic-version with the real 400 body above. More generally, strict mode could enforce each provider's documented required headers, since that is exactly the class of rule an SDK-based happy path never exercises.
Other non-generation endpoints, so they can be tested next to chat:
Anthropic GET /v1/models/{model_id} and POST /v1/messages/count_tokens.
Gemini GET /v1beta/models/{model} and POST /v1beta/models/{model}:countTokens.
Anthropic prompt-cache usage in fixtures. As far as we can tell, an Anthropic fixture cannot return cache_creation_input_tokens, cache_read_input_tokens or the cache_creation.ephemeral_5m_input_tokens / ephemeral_1h_input_tokens breakdown. Being able to set these (or having replay serve recorded values) would let us test cost and context accounting for cached prompts without a live key.
Happy to provide more detail or test a pre-release. Thanks for aimock.
This landed in 1.46.0. These work out of the box now: Gemini model list/detail, Anthropic model detail (/v1/models/{id}), the Anthropic and Gemini token-count routes, and Anthropic prompt-cache usage in fixtures and record mode.
The stricter behavior is opt-in through policy (or --policy, or the X-AIMock-Policy header). That way the defaults match 1.45 and upgrading won't break anyone.
Activity
jpr5 commentedon Oct 21, 2026
This landed in 1.46.0. These work out of the box now: Gemini model list/detail, Anthropic model detail (
/v1/models/{id}), the Anthropic and Gemini token-count routes, and Anthropic prompt-cache usage in fixtures and record mode.The stricter behavior is opt-in through
policy(or--policy, or theX-AIMock-Policyheader). That way the defaults match 1.45 and upgrading won't break anyone.requiredHeaders: "enforce"rejects requests missinganthropic-versionmodelCatalog: "native"serves the Anthropic list onGET /v1/models, which you need for the SDK'smodels.list()authErrors: "native"https://aimock.copilotkit.dev/aimock-cli#required-header-policy
Closing, but open a new issue if anything is still missing.