Expose a custom LLM provider on the evaluator config (bring-your-own provider, e.g. Vertex/Bedrock)

[smnji]

Problem

The SDK's public evaluator config (BaseEvaluatorConfig, and BatchConfig) only supports modelOverride: { provider: OpenAI | Google | Anthropic, model } — i.e. the three built-in API-key adapters. There's no way to route the evaluators through a custom provider.

Our setup uses Google Vertex AI (service-account auth via @ai-sdk/google-vertex), not the Google AI Studio API key that @ai-sdk/google expects, and we don't use OpenAI. So today we cannot run the evaluators on our existing (Vertex) LLM path — the only option we have a credential for is direct Anthropic, which also pulls in a different @ai-sdk/anthropic major than the rest of our app.

Internally the package already has exactly the right abstraction: ProviderConfig has customProvider?: LLMProvider, and createProvider({ type: 'custom', customProvider }) honors it — but customProvider isn't surfaced on BaseEvaluatorConfig/BatchConfig, and createConfiguredProvider only builds openai/google/anthropic types.

Proposed change

Surface the existing customProvider on the public config:

1. Add customProvider?: LLMProvider to BaseEvaluatorConfig (and BatchConfig).
2. In createConfiguredProvider, if config.customProvider is set, return it (skipping the API-key/modelOverride validation that assumes a built-in provider).

This is fully backward-compatible (opt-in) and lets consumers run every evaluator (including the multi-stage ones) through any AI-SDK model they already have wired — Vertex, Bedrock, a gateway, etc. — with no new keys.

Happy to open a PR if you're open to it. Thanks for the SDK!

[adnanrhussain]

@smnji - Thank you for your feedback on improving our evals. I have a PR up for internal review here - #115

I am also experimenting a similar approach for our Python SDK, the team will review and lock down the design/approach and then move forward with these changes.

This issue will be updated, once the PR is merged