Skip to content

feat(serve): restore tool turns on daemon slots - #753

Merged
Kaden-Schutt merged 1 commit into
warpfront:betafrom
alpineQ:fix/multislot-tool-parity-beta
Sep 13, 2026
Merged

Kaden-Schutt merged 1 commit into
warpfront:betafrom
alpineQ:fix/multislot-tool-parity-beta

Conversation

@alpineQ

@alpineQ alpineQ commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

feat(serve): restore tool turns on daemon slots

Summary

Implement Track A of #646 for the daemon-owned multi-slot path. Slot requests now carry the projected tool schemas and typed tool-choice policy, render tool messages through Qwen Jinja, stage canonical calls at commit_ready / done, and resume through the existing typed Continuation::ToolResults path.

The daemon owns session-scoped call IDs and pending tool-turn state. Reentry requires the complete parallel result set and matching session, conversation, call ID, and tool name; stale, ambiguous, aborted, malformed, or failed turns are rejected and cleared. The ordinary non-slot path remains gated out of this implementation.

Which surface(s) does this touch?

  • kernel
  • load
  • serve — daemon slots, engine emission, and typed tool-result reentry
  • arch crate(s)
  • crates/hipfire-quantize / quant formats
  • control plane — hipfire-cli request projection
  • docs / CI / scripts only
  • policy files

Test plan

  • Required no-GPU CI jobs are green on the rebased head
  • cargo build --release --workspace --all-targets --locked clean
  • cargo test --lib --workspace --locked passes
  • Public OpenAI HTTP round-trip, production-shape two-slot run, deterministic replay, and abort/stale/reuse recovery passed on gfx1100
  • ./scripts/speed-gate.sh --fast passes on gfx1100; a non-slot neutrality check is reported below
  • ratchet raise: not applicable

Validation

  • cargo test -p hipfire-daemon --bin daemon: 21 passed
  • cargo test -p hipfire-engine passed
  • cargo test -p hipfire-cli: 247 passed
  • focused broker, Jinja projection, call-ID, request-gate, canonical-emission, and pre-policy identity tests passed
  • cargo check -p hipfire-daemon -p hipfire-cli -p hipfire-engine
  • cleanup ratchets (21/21), crate maps (44), ratchet diff against upstream/beta, changed-file rustfmt, and diff checks passed
  • speed-gate.sh --fast: 1,492.2 tok/s pp32 prefill and 177.9 tok/s decode (floors 1,014.0 and 178.0)
  • serve_harness.py --mode battery: 5/5 turns ended at stop, with no empty output, runaway, or attractor; average decode was 133.8 tok/s

The local no-gpu-ci.sh Rust phases pass. Its Python phase is baseline-red at 381 passed / 6 failed: the same Kernel Atlas case and five test_mq4c_repack.py cases fail on clean beta at c88a1ba0dc3fb2104ab78e6cb65b3067973e41ac.

Hardware validation

Hardware: RX 7900 XTX, gfx1100, HIP/ROCm 7.2, Linux 6.18.37. Model: Qwen3.5-9B MQ4, MD5 296092bf1e6a45d78c1acf815eb93366, SHA-256 ba83acf5bfd5d4e334b0afc26d779734e31623bb7f74e807c3581dfecb3128ad; Q8 KV, two slots, one weight load, 31 function schemas, fixed seed 17.

  • two interleaved conversations remained warm
  • session 0 emitted parallel read_file and read_manifest calls with distinct daemon IDs; session 1 emitted an interleaved generate_title call
  • tool-result reentry reused 4,507/4,550 and 4,475/4,504 prompt tokens; worst processed/necessary work ratio was 0.99
  • an independent fresh-daemon replay matched call names, arguments, IDs, decoded previews, token counts, and finish reasons exactly
  • the daemon performed one model load for both sessions
  • abort returned finish_reason=aborted; the abandoned ID was rejected as stale, and a subsequent request reused the slot successfully

The public OpenAI HTTP path also completed a specific-tool round-trip with multi_slot=true: both requests returned 200; the first response contained normalized read_file({"path":"Cargo.toml"}) with a daemon-owned call_hf_... ID; the tool result was accepted; tool_choice=none was enforced on reentry; and the final response was coherent with finish_reason=stop and 340/370 prompt tokens cached.

The ordinary backend produced the same tool name and arguments and completed the result turn with stop. Backend-owned call IDs and free-form final prose are intentionally not compared byte-for-byte.

Performance neutrality

With multi_slot=false, four fresh processes per arm and three samples per process measured 130.65 tok/s on beta and 130.70 tok/s on this branch by median of process medians (+0.04%, noise-level neutral). The byte-identical prompt MD5 was 253c7ac50857fe6d0e10fb0d2c5e35c0.

Baseline: c88a1ba0dc3fb2104ab78e6cb65b3067973e41ac; candidate: dfdb7425d504547e4e68ebb64a25cc40bfa0c978. Candidate daemon MD5 a9a5650bce8f49ae7f4965f71289d093; CLI MD5 f24c3be9f212ddf9c38c95ad8df05b60.

Scope

This PR implements #646 Track A only. It retains the existing slot-engine limits: Qwen3.5-family models, Q8 KV, non-speculative decode, and one physical GPU.

Revalidation on current beta

Rebased onto beta at 65a4d826a5577f926be11116f43c18906583351e; current head ad6004ac08b9f4d01c4e8c9dd4894d21064dc212. The only source conflict was the post-accept terminal ticket handoff in slots.rs; the port keeps the stricter invariant that an accepted session must already own its lane ticket. Required build, unit, gates, cargo-deny, and changed-file rustfmt CI jobs pass. The advisory clippy job still reports the pre-existing beta lints.

Fresh gfx1100 validation passed after the port:

  • public OpenAI HTTP tool call/result round-trip: 200/200, daemon-owned ID, normalized arguments, tool_choice=none reentry, coherent stop, 340/370 prompt tokens cached;
  • two-session, 31-schema production-shape gate: parallel and interleaved calls completed, warm reentry preserved, worst processed/necessary work ratio 1.00;
  • abort/stale/reuse gate: correlated aborted, abandoned call ID rejected, next request reused the slot;
  • speed-gate.sh --fast: 2,012.1 tok/s pp32 prefill and 171.2 tok/s decode against 1,014.0 / 178.0 floors with the gate's 5% tolerance.

Rebased binary MD5s: daemon 2470c1d933385649a01009f5cc0f64e1; CLI 9a3a008c2def764d9077ca42d9ea1e11.

Architecture-trait change?

No.

Signed-off-by: alpineq <alpineq@protonmail.com>
@alpineQ
alpineQ force-pushed the fix/multislot-tool-parity-beta branch from dfdb742 to ad6004a Compare September 12, 2026 07:14
@Kaden-Schutt
Kaden-Schutt merged commit 01b7576 into warpfront:beta Sep 13, 2026
6 of 7 checks passed
Kaden-Schutt added a commit that referenced this pull request Sep 13, 2026
@Kaden-Schutt

Copy link
Copy Markdown
Collaborator

Merged to beta for v0.3.1 (cc3dfebb2) with two maintainer fixes on top: slot grammar now reads the documented HIPFIRE_QWEN35_GRAMMAR through hipfire_config::developer_var (was a raw std::env::var("HIPFIRE_TOOL_GRAMMAR")), and include_reasoning_content recognises the qwen3_5/qwen3_5_moe arch spellings the slot daemon emits so rich history keeps reasoning_content. Fresh tool round-trip on the post-#755 runtime (Ornith-1.5-35B-A3B, 2 slots): tool_call id survives into tool_call_id, t2 cached=510, correct decoded answer. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants