fix(@workflow/ai): support provider-executed tools (AI SDK v6) #734
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🦋 Changeset detectedLatest commit: 0a7c30c The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (161 failed)mongodb (40 failed):
redis (40 failed):
starter (41 failed):
turso (40 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Express | Nitro workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds support for provider-executed tools in AI SDK v6 by preventing local execution of tools marked with providerExecuted: true and instead using results captured directly from the provider's stream.
Key Changes
- Captures provider-executed tool results from the stream rather than attempting local execution
- Separates tool calls into client-executed and provider-executed categories for proper handling
- Adds comprehensive test coverage for both pure provider-executed and mixed tool scenarios
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
packages/ai/src/agent/do-stream-step.ts |
Captures tool-result stream parts with providerExecuted: true and returns them in a Map |
packages/ai/src/agent/stream-text-iterator.ts |
Adds ProviderExecutedToolResult interface and passes provider results through the iterator |
packages/ai/src/agent/durable-agent.ts |
Filters tool calls by execution type, executes only client tools locally, and combines results in original order |
packages/ai/src/agent/durable-agent.test.ts |
Adds two new tests validating provider-executed tool handling in isolation and mixed with local tools |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
27
to
32
| * Provider-executed tool result captured from the stream. | ||
| */ | ||
| export interface ProviderExecutedToolResult { | ||
| toolCallId: string; | ||
| toolName: string; | ||
| result: unknown; |
There was a problem hiding this comment.
The JSDoc comment for the ProviderExecutedToolResult interface would benefit from more detailed documentation explaining when and how this interface is used, particularly clarifying the relationship with AI SDK v6's provider-executed tools feature and providing examples of which providers support this feature.
Suggested change
| * Provider-executed tool result captured from the stream. | |
| */ | |
| export interface ProviderExecutedToolResult { | |
| toolCallId: string; | |
| toolName: string; | |
| result: unknown; | |
| * Result of a provider-executed tool call captured from the model stream. | |
| * | |
| * In AI SDK v6 some providers can execute certain tools (sometimes called | |
| * "built‑in" or "provider‑hosted" tools) on their own infrastructure instead | |
| * of requiring your application to run the tool implementation. When a model | |
| * uses such a tool during a streaming call, the provider may stream back the | |
| * tool's result as part of the model response. | |
| * | |
| * The streaming agent layer detects these provider‑executed tool results and | |
| * exposes them via the {@link StreamTextIteratorYieldValue.providerExecutedToolResults} | |
| * map. Each entry is keyed by the tool call ID so you can correlate it with the | |
| * corresponding {@link LanguageModelV2ToolCall}. | |
| * | |
| * Not all models or providers support provider‑executed tools. Support depends | |
| * on the underlying provider's capabilities (for example, providers that expose | |
| * server‑side web search or code‑execution tools). Refer to the AI SDK and | |
| * individual provider documentation for an up‑to‑date list of supported | |
| * providers and tools. | |
| * | |
| * When {@link isError} is true, {@link result} contains provider‑specific error | |
| * information for the tool call instead of a successful tool payload. | |
| */ | |
| export interface ProviderExecutedToolResult { | |
| /** | |
| * Identifier of the tool call, matching the `id` of the | |
| * {@link LanguageModelV2ToolCall} emitted by the model. | |
| */ | |
| toolCallId: string; | |
| /** | |
| * Name of the tool as defined by the provider (for example, the tool | |
| * name used in the model's tool schema). | |
| */ | |
| toolName: string; | |
| /** | |
| * The value returned by the provider‑executed tool. The shape of this | |
| * object is provider‑ and tool‑specific and is passed through as‑is. | |
| */ | |
| result: unknown; | |
| /** | |
| * Indicates that the provider reported an error when executing the tool. | |
| * When set, {@link result} typically contains error details instead of a | |
| * successful payload. | |
| */ |
Comment on lines
+829
to
+874
| const toolResults = toolCalls.map((tc) => { | ||
| const clientResult = clientToolResults.find( | ||
| (r) => r.toolCallId === tc.toolCallId | ||
| ); | ||
| if (clientResult) return clientResult; | ||
| const providerResult = providerToolResults.find( | ||
| (r) => r.toolCallId === tc.toolCallId | ||
| ); | ||
| if (providerResult) return providerResult; | ||
| // This should never happen, but return empty result as fallback | ||
| return { | ||
| type: 'tool-result' as const, | ||
| toolCallId: tc.toolCallId, | ||
| toolName: tc.toolName, | ||
| output: { type: 'text' as const, value: '' }, | ||
| }; | ||
| }); |
There was a problem hiding this comment.
The logic for combining tool results has O(n²) complexity. For each tool call in toolCalls, it performs a linear search through both clientToolResults and providerToolResults arrays using find(). This could be inefficient when there are many tool calls.
Consider creating Maps indexed by toolCallId for O(1) lookups instead of using find() on arrays.
Comment on lines
815
to
855
| // If no result from stream, return an empty result | ||
| // This can happen if the provider didn't send a tool-result stream part | ||
| return { | ||
| type: 'tool-result' as const, | ||
| toolCallId: toolCall.toolCallId, | ||
| toolName: toolCall.toolName, | ||
| output: { | ||
| type: 'text' as const, | ||
| value: '', | ||
| }, | ||
| }; | ||
| }); |
There was a problem hiding this comment.
When a provider-executed tool call doesn't have a corresponding result in the stream, an empty string is returned as the result. This silent failure could make debugging difficult. Consider logging a warning when this fallback is used, as the comment indicates this scenario "can happen if the provider didn't send a tool-result stream part" but it might also indicate an unexpected error condition.
Collaborator
Author
|
a solution that seemingly work in AI sdk v5 and v6 
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| value: | ||
| typeof streamResult.result === 'string' | ||
| ? streamResult.result | ||
| : JSON.stringify(streamResult.result), |
Contributor
There was a problem hiding this comment.
Suggested change
| : JSON.stringify(streamResult.result), | |
| : JSON.stringify(streamResult.result) ?? '', |
JSON.stringify(undefined) returns undefined instead of a string, violating the type contract for the 'value' field
View Details
📝 Patch Details
diff --git a/packages/ai/src/agent/durable-agent.ts b/packages/ai/src/agent/durable-agent.ts
index 4795ea9..ba8e4a8 100644
--- a/packages/ai/src/agent/durable-agent.ts
+++ b/packages/ai/src/agent/durable-agent.ts
@@ -808,7 +808,7 @@ export class DurableAgent<TBaseTools extends ToolSet = ToolSet> {
value:
typeof streamResult.result === 'string'
? streamResult.result
- : JSON.stringify(streamResult.result),
+ : JSON.stringify(streamResult.result) ?? '',
},
};
}
Analysis
The issue was in packages/ai/src/agent/durable-agent.ts at line 811. When handling provider-executed tool results, the code did:
value: typeof streamResult.result === 'string'
? streamResult.result
: JSON.stringify(streamResult.result)The problem: JSON.stringify(undefined) returns undefined (not a string), which violates the type contract since the value field in LanguageModelV2ToolResultPart should be a string.
This was confirmed by testing:
JSON.stringify(undefined)returnsundefinedwith typeundefined- This causes the
valuefield to beundefinedinstead of a string
The fix applies the null coalescing operator (??) pattern that was already used elsewhere in the codebase (at line 1053 for local tool results):
value: typeof streamResult.result === 'string'
? streamResult.result
: JSON.stringify(streamResult.result) ?? ''This ensures:
- If
streamResult.resultis a string, use it directly - If
JSON.stringify()returns a string, use it - If
JSON.stringify()returns undefined (only when input isundefined), use empty string
The fix makes the provider-executed tool result handling consistent with the local tool result handling pattern used at line 1053.
Collaborator
Author
|
Oh this is a duplicate of #434 |
TooTallNate
approved these changes
Jan 9, 2026
Provider-executed tools (like Google's googleSearch, Claude's WebSearch, etc.) have providerExecuted: true on their tool calls and should NOT be executed locally. Instead, their results come from the provider via tool-result stream parts. This fix: - Captures provider-executed tool results from the stream in do-stream-step.ts - Passes these results through the iterator to the DurableAgent - Separates client-executed from provider-executed tool calls - Uses stream results for provider-executed tools instead of local execution - Adds tests for provider-executed tools and mixed scenarios Fixes #628
pranaygp
force-pushed
the
pranaygp/fix-628
branch
from
7851c7e to
4351102
Compare
Use 'json' type for object results and 'text' type for string results, matching the AI SDK spec (LanguageModelV2ToolResultOutput). Before: Always used 'text' with JSON.stringify(result) After: Use 'json' with raw value for objects, 'text' for strings This aligns with AI SDK types and fixes compatibility with writeToolOutputToUI which no longer needs to JSON.parse. Co-Authored-By: Claude Opus 4.5 <[email protected]>
pranaygp
added a commit
that referenced
this pull request
Jan 28, 2026
…s-and-commands * origin/main: fix(@workflow/ai): support provider-executed tools (AI SDK v6) (#734) Publish "workflow" and "@workflow/core" package versions in sync (#870) Add SDK version to workflow run executionContext for observability (#868) Allow recreateRun to accept an optional deploymentId parameter (#869) Add support for top-level `using` declarations inside of step / workflow functions (#866) docs: URL in docs was missing the docs/ prefix, 404 errors (#852) Add "classes" object to `manifest.json` file (#864) Fix Nest workbench app build (#865) Ignore Astro on local dev tests for source map e2e tests (#863) Enable custom class serialization transformations for "client" mode (#860) Submit request bodies with CBOR encoding (#844) [world-vercel] Update queue to use VQS v3 API (#799) NestJS framework support (#840) Fix resolve hook theming and token fetching. (#856) docs: rename Control Flow Patterns to Common Patterns and add new content (#846) docs: revamp World documentation pages (#763) Remove unused `getWritable` stub function (#855) # Conflicts: # packages/core/package.json # pnpm-lock.yaml
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes issue #628 - provider-executed tools (tools with
providerExecuted: true) now work correctly with the Workflow DevKit.Problem
Provider-executed tools (like Google's
googleSearch, Claude'sWebSearch, OpenAI's web search tools, etc.) were causing errors because:providerExecuted: trueDurableAgenttried to execute them locallytoolsmap, execution failed with "Tool not found" errorsSolution
The fix modifies three files in the
@workflow/aipackage:do-stream-step.tstool-resultstream parts that haveproviderExecuted: trueMap<string, ProviderExecutedToolResult>keyed bytoolCallIdstream-text-iterator.tsProviderExecutedToolResultinterfaceStreamTextIteratorYieldValueto includeproviderExecutedToolResultsdurable-agent.tsclientToolCalls(noproviderExecutedflag) andproviderToolCalls(providerExecuted: true)clientToolCallslocallyproviderToolCalls, uses the results captured from the streamTesting
Added two new tests:
should skip local execution for provider-executed tools- verifies provider-executed tools are NOT executed locallyshould handle mixed provider-executed and local tools- verifies mixed scenarios work correctlyAll 64 tests pass.
Affected Providers
This fix enables proper support for:
@ai-sdk/googlewithgoogle.tools.googleSearch()@ai-sdk/openaiwith web search toolsai-sdk-provider-claude-code(Claude Code's Bash, Read, Write, WebSearch, etc.)providerExecuted: trueFixes #628
Fixes #433
Closes #434 (duplicate)