feat(tracing): span each tool call on the sync agent path - #494
Closed
mohammadatallah-scale wants to merge 1 commit into
Closed
feat(tracing): span each tool call on the sync agent path#494mohammadatallah-scale wants to merge 1 commit into
mohammadatallah-scale wants to merge 1 commit into
Conversation
The sync path only spans Model.stream_response, the inference call. The Runner executes tools outside it, so tool time falls into gaps between spans and nothing accounts for it. On dev-sgp an agent with slow tools reported a third of its real wall-clock time. SyncTracingHooks ports the span lifecycle the Temporal plugin already uses, minus the activity plumbing, so both paths emit the same trace shape.
|
This PR is targeting The
See |
2 tasks
Author
|
Closing unmerged. The approach stands, it just needs a live dev-sgp run behind it before it should land. Kept as the starting point on OVE-694. Branch |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Important
Pending Mohammad's self-review. This note is removed by a human, not automation.
The bug: on the sync agent path the only span wraps the LLM inference call, so time an agent spends in tools is not covered by any span and disappears from the SGP dashboard.
The fix:
SyncTracingHooksopens a span when a tool starts and closes it when the tool returns, so span width is the tool's real duration. It ports the lifecycle the Temporal plugin already uses, minus the activity plumbing.Not in this PR: nothing calls these hooks yet. The turn helper that wires them up is #495.
Measured on dev-sgp, 14 days,
analyst-agent. Span width is what a timeline renders, so a duration recorded only as an attribute is invisible.