feat: add native iOS review journey - #5971
Open
jedwards27 wants to merge 11 commits into
Open
Conversation
added 6 commits
August 14, 2026 20:54
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
…urneys * origin/main: fix(desktop): hide the offcanvas-collapsed sidebar so it stops painting over the community rail (block#5947) Polish mobile message threads and composer (block#5645) chore(release): release Buzz Desktop version 0.5.14 (block#5917) ci(release): remove desktop smoke gate (block#5914) Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Fail closed on non-standard fixture ports, use fixed loopback database coordinates, scrub repository-controlled subprocess environments, and remove generated review keys when fixture seeding fails. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Apply absolute maximum budgets to the worst candidate sample while retaining median-based relative regression comparisons. Cover the catastrophic-outlier case directly. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Create a unique run-owned simulator, scrub credentials from iOS review subprocesses, and delete only the owned device during cleanup. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Default harness commands to a credential-free environment and isolate cleanup HOME without reconstructing the host environment. Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz> (cherry picked from commit 3635687)
themiguelamador
suggested changes
Aug 16, 2026
themiguelamador
left a comment
There was a problem hiding this comment.
Blocking findings:
- The documented harness tests and desktop runner import PyYAML without installing or pinning it, so they fail from a clean checkout after Hermit activation.
- The AVAssetWriter loop leaves its frame clock unchanged during backpressure, making its sleep deadline remain in the past and causing a busy-spin that can perturb native performance results.
scrollaccepts a locator but posts at the pointer's prior screen position, so the declared target is ignored and the wrong view can scroll.- The new iOS receipt does not conform to the checked-in receipt schema: it omits flow/timestamps/isolation/steps/measurements/performance and adds an undeclared device object. Publication and comparison consumers therefore receive incomplete provenance.
- Journey/performance validation accepts invalid durations, duplicate cohort receipts, booleans, and non-finite values, allowing late crashes or non-independent samples to satisfy the cohort minimum.
- The harness README still says iOS Simulator is a future phase and omits its owned-device safety contract, despite this PR adding the lane.
Fixed in Complear/buzz@2efeddec8 (branch review/pr-5971-fix): locked uv/PyYAML provisioning, backpressure-safe recording, locator-targeted scrolling, schema-compatible iOS receipts, fail-closed validation with tests, and corrected iOS documentation. Verification: native-review 39/39, Swift release build, wrapper validation, Python/JSON/shell syntax checks. The same stacked mobile/desktop changes were additionally validated on descendant #5972 with 1,417 mobile and 4,954 desktop tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stack
Stacked on #5968 (
feat/native-review-journeys) so the iOS layer can be reviewed independently.Validation
At clean HEAD
61410c429fc67bd31c105dfbf1b7f68ed11b18d4:test-results/native-review/61410c429fc6/ios_pairing/ios-20260815T100106-f971e4/receipt.json(31.625s video)test-results/native-review/61410c429fc6/ios_pairing/ios-20260815T100219-ddcdaa/receipt.json(30.310s video)Evidence semantics
The runner records the selected runtime and UDID, source SHA and dirty status, Flutter stdout/stderr, final screenshot, MP4, failure cause, and cleanup errors. A journey or cleanup failure returns nonzero. Native evidence is intentionally local because recordings are large and machine-specific.