feat(serve): restore tool turns on daemon slots - #753
Merged
Kaden-Schutt merged 1 commit intoSep 13, 2026
Merged
Conversation
Signed-off-by: alpineq <alpineq@protonmail.com>
alpineQ
force-pushed
the
fix/multislot-tool-parity-beta
branch
from
September 12, 2026 07:14
dfdb742 to
ad6004a
Compare
Kaden-Schutt
added a commit
that referenced
this pull request
Sep 13, 2026
…, slot tool turns (#753)
Collaborator
|
Merged to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(serve): restore tool turns on daemon slots
Summary
Implement Track A of #646 for the daemon-owned multi-slot path. Slot requests now carry the projected tool schemas and typed tool-choice policy, render tool messages through Qwen Jinja, stage canonical calls at
commit_ready/done, and resume through the existing typedContinuation::ToolResultspath.The daemon owns session-scoped call IDs and pending tool-turn state. Reentry requires the complete parallel result set and matching session, conversation, call ID, and tool name; stale, ambiguous, aborted, malformed, or failed turns are rejected and cleared. The ordinary non-slot path remains gated out of this implementation.
Which surface(s) does this touch?
crates/hipfire-quantize/ quant formatshipfire-clirequest projectionTest plan
cargo build --release --workspace --all-targets --lockedcleancargo test --lib --workspace --lockedpasses./scripts/speed-gate.sh --fastpasses on gfx1100; a non-slot neutrality check is reported belowValidation
cargo test -p hipfire-daemon --bin daemon: 21 passedcargo test -p hipfire-enginepassedcargo test -p hipfire-cli: 247 passedcargo check -p hipfire-daemon -p hipfire-cli -p hipfire-engineupstream/beta, changed-file rustfmt, and diff checks passedspeed-gate.sh --fast: 1,492.2 tok/s pp32 prefill and 177.9 tok/s decode (floors 1,014.0 and 178.0)serve_harness.py --mode battery: 5/5 turns ended atstop, with no empty output, runaway, or attractor; average decode was 133.8 tok/sThe local
no-gpu-ci.shRust phases pass. Its Python phase is baseline-red at 381 passed / 6 failed: the same Kernel Atlas case and fivetest_mq4c_repack.pycases fail on cleanbetaatc88a1ba0dc3fb2104ab78e6cb65b3067973e41ac.Hardware validation
Hardware: RX 7900 XTX, gfx1100, HIP/ROCm 7.2, Linux 6.18.37. Model: Qwen3.5-9B MQ4, MD5
296092bf1e6a45d78c1acf815eb93366, SHA-256ba83acf5bfd5d4e334b0afc26d779734e31623bb7f74e807c3581dfecb3128ad; Q8 KV, two slots, one weight load, 31 function schemas, fixed seed 17.read_fileandread_manifestcalls with distinct daemon IDs; session 1 emitted an interleavedgenerate_titlecallfinish_reason=aborted; the abandoned ID was rejected as stale, and a subsequent request reused the slot successfullyThe public OpenAI HTTP path also completed a specific-tool round-trip with
multi_slot=true: both requests returned 200; the first response contained normalizedread_file({"path":"Cargo.toml"})with a daemon-ownedcall_hf_...ID; the tool result was accepted;tool_choice=nonewas enforced on reentry; and the final response was coherent withfinish_reason=stopand 340/370 prompt tokens cached.The ordinary backend produced the same tool name and arguments and completed the result turn with
stop. Backend-owned call IDs and free-form final prose are intentionally not compared byte-for-byte.Performance neutrality
With
multi_slot=false, four fresh processes per arm and three samples per process measured 130.65 tok/s on beta and 130.70 tok/s on this branch by median of process medians (+0.04%, noise-level neutral). The byte-identical prompt MD5 was253c7ac50857fe6d0e10fb0d2c5e35c0.Baseline:
c88a1ba0dc3fb2104ab78e6cb65b3067973e41ac; candidate:dfdb7425d504547e4e68ebb64a25cc40bfa0c978. Candidate daemon MD5a9a5650bce8f49ae7f4965f71289d093; CLI MD5f24c3be9f212ddf9c38c95ad8df05b60.Scope
This PR implements #646 Track A only. It retains the existing slot-engine limits: Qwen3.5-family models, Q8 KV, non-speculative decode, and one physical GPU.
Revalidation on current beta
Rebased onto
betaat65a4d826a5577f926be11116f43c18906583351e; current headad6004ac08b9f4d01c4e8c9dd4894d21064dc212. The only source conflict was the post-accept terminal ticket handoff inslots.rs; the port keeps the stricter invariant that an accepted session must already own its lane ticket. Required build, unit, gates, cargo-deny, and changed-file rustfmt CI jobs pass. The advisory clippy job still reports the pre-existing beta lints.Fresh gfx1100 validation passed after the port:
tool_choice=nonereentry, coherentstop, 340/370 prompt tokens cached;aborted, abandoned call ID rejected, next request reused the slot;speed-gate.sh --fast: 2,012.1 tok/s pp32 prefill and 171.2 tok/s decode against 1,014.0 / 178.0 floors with the gate's 5% tolerance.Rebased binary MD5s: daemon
2470c1d933385649a01009f5cc0f64e1; CLI9a3a008c2def764d9077ca42d9ea1e11.Architecture-trait change?
No.