Skip to content

Execute oversized pure-read batches in bounded waves #111

Description

@yablokolabs

Summary

The prompt now states the exact maxParallelToolCalls value clearly, but a model can still emit an otherwise valid pure-read batch larger than the configured cap. Atomic rejects it and spends a second LLM call asking for a repaired batch.

This is the residual form of finding §8 from Yabloko Labs' evaluation. The observed Qwen response contained 14 independent file reads with a configured maximum of eight.

Current behavior

The stable prompt correctly tells the model to emit up to the configured limit:

If the response still exceeds the cap, validation produces batch exceeds maxParallelToolCalls, and oversized batches go through the LLM repair path:

Proposed behavior

When every call has been preflight-validated as a registered, schema-valid pure_read call, execute the response deterministically in waves of at most maxParallelToolCalls rather than regenerating it.

For example, 14 independent reads with a cap of 8 become waves of 8 and 6. This issue intentionally does not generalize wave splitting to browser, memory, tasks, vision, writes, approval-gated calls, or terminal calls; those classes can be stateful or require dependency-aware scheduling.

Current approval-only classification does not consistently enforce this boundary because the batch-level cap error is separate from per-call approval errors. The implementation must therefore route an excluded oversized batch directly to fail-closed repair before either wave splitting or approval trimming/retained-barrier handling can dispatch an original call.

Acceptance criteria

  • Wave splitting activates only when every call is registered, argument-schema-valid, and classified pure_read.
  • An otherwise valid pure-read batch larger than the cap is split without another LLM call.
  • No wave exceeds maxParallelToolCalls.
  • Original global batchIndex values and result correlation are preserved across waves.
  • Waves execute in original order; calls within each wave use the existing pure-read executor.
  • An oversized batch containing any approval-gated, terminal, unknown, malformed, or non-pure_read call bypasses both wave splitting and approval trimming/retained-barrier handling, triggers parse_retry, and dispatches no original call before a valid repaired response.
  • Events/metrics report the original size, cap, wave count, and per-wave boundaries.
  • Tests cover 14 reads at cap 8, cap 1, exact-cap input, schema-invalid reads, each excluded resource class, and an explicit oversized [approval_gated, pure_read, ...] batch.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions