feat: Cloudflare-native AI tracing (agents/observability/ai)

[mattzcarey]

Cloudflare-native tracing for the repository's installed AI SDK v6 using the Workers runtime tracing API. Spans use the scalar subset of the current OpenTelemetry GenAI semantic conventions that Workers custom spans can represent and flow directly to Workers Observability and configured OTLP destinations. No OTel SDK, exporter, or collector dependency is added.

AI SDK v6 instrumentation

Enable Worker traces:

{
  "observability": { "traces": { "enabled": true } }
}

Wrap the namespace once and use it normally:

import * as ai from "ai";
import { wrapAISDK } from "agents/observability/ai";

const { generateText, streamText } = wrapAISDK(ai);

await streamText({
  model,
  prompt: "book a table for two",
  tools: { searchRestaurants, reserve },
  experimental_telemetry: {
    functionId: "booking-agent", // AI SDK canonical mapping: gen_ai.agent.name
    metadata: { conversationId: "conv-42" }
  }
});

The wrapper instruments generateText, streamText, generateObject, and streamObject:

invoke_agent booking-agent
├── chat gpt-5.5
├── execute_tool searchRestaurants
└── chat gpt-5.5

Model objects are instrumented through wrapLanguageModel, so provider work and its automatic subrequests run under chat {model}. Tool execution runs under execute_tool {tool}. Streams close on completion, cancellation, an in-band error, or early consumer return. Async-generator tools stay open until iteration and cleanup finish.

AI SDK v6 exposes experimental_context; selected scalar keys can be emitted with wrapAISDK(ai, { includeRuntimeContext: [...] }).

Think: traced out of the box

Think routes every turn through the wrapper with no new customer-facing option. Its default identity is:

gen_ai.agent.name      = this.constructor.name
gen_ai.agent.id        = this.name
gen_ai.conversation.id = this.ctx.id.toString()

These are defaults, not restrictions: beforeTurn may override functionId or identity metadata for applications with another identity model. Think also adds exact cloudflare.agents.turn.* metadata for request ID, trigger, admission, channel, continuation, and generation. Ordinary customer keys such as requestId remain ordinary metadata and are not silently reclassified as Think turn fields.

Drain loops consume the abandoned AI SDK tee after early exit (in-stream error, stall abort, or user abort), preventing open operation spans.

Attribute contract

• invoke_agent: agent/conversation identity, output type, request settings, aggregate standard token usage, dashboard-ready total/tool counts, and scalar finish reason.
• chat: provider, requested model/settings, actual response ID/model when reported, model-call usage and total/tool counts, and gen_ai.response.time_to_first_chunk in seconds.
• execute_tool: tool name/type/call ID and real execution duration via span duration.
• Original SDK operation fields use cloudflare.agents.* where no standard attribute exists.

Semantic corrections included in the final audit:

• functionId → gen_ai.agent.name is retained because it is the AI SDK's canonical OpenTelemetry projection.
• gen_ai.response.finish_reasons and gen_ai.request.stop_sequences are omitted: OTel defines arrays, while Workers Span.setAttribute currently accepts only scalar values. A single finish reason is exposed accurately as cloudflare.agents.response.finish_reason instead of JSON text under an array-typed key.
• Response ID/model and first-chunk timing are placed on model-call spans rather than duplicated onto the enclosing operation when a model child exists.
• Requested modelId is never presented as the served gen_ai.response.model.
• cloudflare.agents.tool.count and cloudflare.agents.usage.total_tokens remain as useful precomputed dashboard dimensions; only low-value output-presence flags were removed.
• Span names retain the Workers Observability 64 UTF-8-byte guard, falling back to the bare operation while preserving the full target in attributes.
• Provider aliases match the AI SDK/OpenTelemetry mapping (azure vs azure-openai, google-vertex, bedrock, etc.).
• Failures emit error.type; no fake otel.status_code attribute is created because OTel status is span state and Workers exposes no custom-span status setter. Cancellations remain non-errors with cloudflare.agents.canceled: true.
• No prompts, messages, tool inputs/outputs, schemas, headers, provider options, raw model output, or raw error messages are recorded.

Public surface

agents/observability/ai exports only:

• wrapAISDK
• the v6 wrapper option type

The Workers tracing adapter, tracer/span handles, builders, and runtime seams remain internal. agents/observability continues to expose diagnostics-channel events only.

The AI SDK v7 prototype was removed because this v6 repository cannot integration-test it. It remains available as an explicitly unverified public design artifact: https://gist.github.com/mattzcarey/cc7265966dd8eb58e7b230688a3f4140

Testing

• 71 observability tests pass across tracer internals and real/fake AI SDK v6 wrappers
• complete Think Workers suite: 42 files / 895 tests
• pnpm check: exports, formatting, lint, and all 116 TypeScript projects
• agents and @cloudflare/think package builds pass
• generated declarations expose no generic tracer/span API

Production validation

Compared raw Workers Observability events for the same agent-think smoke shape:

• baseline: de7395981c140629b3219f5f41cb9d29
• audited build: 6f3737d3788029ae4b23dff165ed3383
• topology unchanged: 1 invoke_agent ThinkAgent, 4 chat gpt-5.5, 3 execute_tool; all children correctly parented
• model TTFC corrected from four 0 values to 9.462s, 6.144s, 4.356s, 1.011s
• invalid JSON-string gen_ai.response.finish_reasons removed
• scalar application metadata (repository, issueNumber, requestedBy) visible on the root
• turn reached done, added no tool errors, and terminal cleanup left exactly one warm container on container-application image version 7

Evidence: #1883 (comment)

Provenance

Ported from @msmps's feat/ai-tracing branch with Co-authored-by credit, then folded into agents, aligned with current GenAI semantics, and integrated into Think.

[changeset-bot]

🦋 Changeset detected

Latest commit: 05ef7ce

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
agents	Minor
@cloudflare/think	Minor
@cloudflare/agent-think	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@1860

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@1860

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@1860

create-think

npm i https://pkg.pr.new/create-think@1860

hono-agents

npm i https://pkg.pr.new/hono-agents@1860

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@1860

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@1860

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@1860

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@1860

commit: 05ef7ce

[abhagsain]

@mattzcarey Would it show a tracing UI like langsmith/braintrust and also show sub-agent calls separately?

[devin-ai-integration]

Devin Review found 1 potential issue.

[mattzcarey]

Production trace comparison

Final v6-only build deployed to agent-think Worker version a612d7b1-d349-4b9b-8601-31901ca8b93e (container application image version 7, digest sha256:1ceff71cad30f3207e4380de6ceaff1b7f89c6908505f49aeba49574344a9d3b).

Raw Workers Observability comparison:

• baseline: de7395981c140629b3219f5f41cb9d29
• final v6-only build: 6ca72a82c7813bfc313c765ea3e86eeb
• topology unchanged: 1 invoke_agent ThinkAgent, 4 chat gpt-5.5, 3 execute_tool; all children correctly parented
• real model TTFC: 8.575s, 2.093s, 4.758s, 1.452s rather than four 0 values
• invalid JSON-string gen_ai.response.finish_reasons removed
• response ID/model/TTFC retained on model spans rather than copied to the root
• dashboard counts retained: root cloudflare.agents.tool.count=3, cloudflare.agents.usage.total_tokens=86336; per-model total/tool counts also present
• scalar application metadata (repository, issueNumber, requestedBy) visible on the root
• turn reached done, added no tool errors, and terminal cleanup left exactly one warm container

The untestable AI SDK v7 adapter was removed from the package and preserved as an explicitly unverified design gist: https://gist.github.com/mattzcarey/cc7265966dd8eb58e7b230688a3f4140

Validation evidence: #1883 (comment)