Skip to content

Commit f4a45e0

Browse files
test: measure client-observed refresh latency (Refs #531) (#542)
## Summary Measure refresh latency at the JSONRPC client boundary instead of treating server discovery duration as total request latency. Refs #531; part of #528. Initial hosted calibration is now recorded in `31551ea`; this draft remains open for CI and review before closing #531. - Record client round-trip, request-relative time to first environment, process-start-relative time to first environment, and server discovery duration separately, including cold distributions. - Preserve exact-base schema-v2 semantic comparisons; never compare old discovery-only samples as client RTT. Schema-v3 artifacts require all new distributions, and v3-to-v3 comparisons activate empirically calibrated client gates without changing existing discovery/startup/coverage budgets. - Exercise the real `PetClient.refresh` serialization/write/buffered-read path using a deterministic channel fixture: 80ms/50ms client RTT despite 2ms/1ms server duration, with TTFE resetting to 40ms/7ms. Malformed environment notifications fail before timestamps/inventory are recorded. ## Validation Local Windows validation: - `cargo test --offline --locked -p pet --test e2e_performance`: **8 passed**. - `python -m unittest discover -s scripts\tests -p 'test_*.py' -q`: **55 passed**. - `scripts\rust-precommit.ps1`: formatting and workspace warnings-as-errors Clippy passed. - `cargo clippy --offline --locked --workspace --all-targets --all-features -- -D warnings`: passed. - `git diff --check`: passed. - Independent repository Reviewer follow-up confirmed malformed-notification feedback resolved. ## Draft gates and compatibility Calibration now uses three successful hosted baseline runs plus the original PR measurement at identical source `9c1b003`, giving 40 cold/warm pairs per platform. Run links, stable inventories, observed ranges, and new client-only budgets are documented in `docs/QUALITY_SNAPSHOTS.md`; replay tests cover unchanged-source variance and material regressions. Client absolute budgets retain at least twice observed statistic ranges; existing discovery/startup/coverage gates are unchanged. The v2-to-v3 transition retains old gates and explicitly reports new client metrics as diagnostics when the exact base lacks comparable data. Environment notifications lack request IDs. The measured summary uses one refresh per fresh process to avoid post-response/concurrent notification ambiguity; no general concurrent TTFE attribution is claimed. Existing buffered stdout reading and bounded stderr draining are retained. This PR is independent of #541's runtime glob fix and does not include #529's EOF changes. Preserve both changes when integrating the benchmark-client shutdown work later. No merge or auto-merge requested. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 22e37af commit f4a45e0

5 files changed

Lines changed: 911 additions & 177 deletions

File tree

0 commit comments

Comments
 (0)