P4b: invert non-row bounded tokens to sync TdsCore step_token driver

[Saurabh Singh (saurabh500)]

P4b — Invert the non-row RECEIVE / token-consume paths to a sans-I/O sync core

Base: dev/saurabh/sans-io-p4a-tdscore-row-driver (frozen P4a tip 07099e0c). Draft; do not merge — coordinator will verify from git ground truth + a real nextest and append to stack #192.

Mirrors P4a's row inversion for the non-row receive path. Every guaranteed-bounded category-(a) non-row token is now parsed by a pure-sync body over PacketBuffer, driven by one async shell that owns the sole refill .await. Value-carrying / login tokens stay on an explicit async seam. Zero behavior change; public API frozen async-only.

What changed

• TokenStep + TdsCore::step_token (io/tds_core.rs) — peeks the token byte, ensures the length-bounded body, and parses in place; returns NeedBytes on entry underflow (nothing consumed, restartable) or AsyncToken(tt) for seamed tokens.
• io/sync_token.rs (new) — the pure-sync parse leaf: one production parse body per category-(a) token (DONE/DONEINPROC/DONEPROC, RETURNSTATUS, ORDER, ERROR, INFO, ENVCHANGE), byte-for-byte mirroring the async parsers.
• drive_token_over_buffer + resolve_header_token (io/token_stream.rs) — the single async shell; rewires the three production receive_token sites in network_transport.rs.
• Oracle retained — receive_token_internal is gated #[cfg(any(test, fuzzing))] as a differential byte-identity oracle; grep-proven no production path reaches it (its only non-test callers are #[cfg(fuzzing)]).

Scope honesty (deferred work)

COLMETADATA / RETURNVALUE / SESSIONSTATE / FEATUREEXTACK (category-(b), value/PLP-carrying) and the login/handshake tokens (LOGINACK/TABNAME/COLINFO/SSPI/FEDAUTHINFO) remain on the AsyncToken async seam. Their pure-sync inversion is deferred to a later, parent-gated layer that would introduce a sanctioned mid-token TokenPauseState cursor. The receive path is not fully sync after P4b — this is stated in the tds_core.rs module doc and enforced by leaving those tokens on the seam. No new resumable machine is introduced here.

Two folded P4a-hardening items (carried here, not in #198)

1. RowStep::AsyncColumn load-bearing doc comment (io/tds_core.rs) — bakes in the exact rationale that yielding an unbounded PLP column preserves L4b bounded residency and must never be collapsed into NeedBytes{shortfall}.
2. Strict production-path eager-PLP residency-CEILING test (tdscore_step_row_eager_plp_residency_ceiling_on_production_driver) — decodes a >8192 B eager-PLP LOB through drive_row_over_buffer (production) and asserts peak PacketBuffer residency stays ≲ one buffer/chunk via a #[cfg(test)] peak_length high-water hook (off the public surface). It FAILS if AsyncColumn is ever collapsed to a collect-whole / NeedBytes(full-len) path.

Note: P4a's own eager-PLP residency guarantee is asserted one layer up (here, in P4b) rather than in #198, to keep the frozen P4a tip 07099e0c stable. A reviewer of #198 should not be confused about why that assertion lives in P4b.

Tests (through the new sync driver)

• DONE/DONEINPROC/DONEPROC, ERROR/INFO, ORDER, ENVCHANGE — swept at every interior split vs the oracle (byte-identical resume, no consume-on-shortfall, no hang).
• COLMETADATA split across a refill boundary — proves the AsyncToken seam handles a token spanning a refill.
• COLMETADATA → ROW handoff — proves the seam hands off to the row driver over the shared buffer.

Gate (from this worktree)

• cargo nextest run -p mssql-tds --lib --no-fail-fast → fail-set == exactly the 7 known cert fixtures (Compare-Object of sorted fail-name sets == EMPTY), no hang.
• cargo bfmt + cargo bclippy (-D warnings) clean; scripts/bfmt.ps1 + scripts/bclippy.ps1 clean incl. mssql-py-core; cargo build --all-features in mssql-py-core OK.
• Public-API diff-scan == EMPTY (all new symbols pub(crate)/cfg(test)).

[Copilot]

Pull request overview

Moves bounded non-row token parsing into the synchronous TdsCore path while retaining async seams for value-carrying tokens.

Changes:

• Adds synchronous token classification and parsing.
• Rewires production token reception through one async refill driver.
• Adds differential and PLP residency tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
mssql-tds/src/io/token_stream.rs	Adds token drivers and boundary tests.
mssql-tds/src/io/tds_core.rs	Adds synchronous token stepping.
mssql-tds/src/io/sync_token.rs	Implements synchronous token parsers.
mssql-tds/src/io/packet_buffer.rs	Adds test-only residency tracking.
mssql-tds/src/io.rs	Registers the new module.
mssql-tds/src/connection/transport/network_transport.rs	Uses the new production driver.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.