Skip to content

feat: add native iOS review journey - #5971

Open
jedwards27 wants to merge 11 commits into
block:mainfrom
jedwards27:feat/native-review-ios
Open

feat: add native iOS review journey#5971
jedwards27 wants to merge 11 commits into
block:mainfrom
jedwards27:feat/native-review-ios

Conversation

@jedwards27

Copy link
Copy Markdown

Summary

  • add an isolated iOS Simulator runner that erases/boots a selected runtime, drives a real Flutter integration journey, records the full simulator display, captures logs/screenshots, and writes SHA/dirty/cleanup receipts
  • cover pairing-code reveal, edit, and hide through stable semantic widget keys
  • preserve production badge-permission behavior while suppressing the unautomatable SpringBoard prompt only for native-review child processes
  • add fail-closed runner tests for device/runtime selection, recorder finalization/timeouts, failure receipts, and cleanup

Stack

Stacked on #5968 (feat/native-review-journeys) so the iOS layer can be reviewed independently.

Validation

At clean HEAD 61410c429fc67bd31c105dfbf1b7f68ed11b18d4:

  • pre-push hooks passed: branch skew, Desktop check/typecheck/test, mobile tests, Rust tests, and Tauri checks
  • mobile analyze: no issues
  • full mobile suite: 1,417 passed
  • native-review Python suite: 24/24 passed
  • two consecutive clean exact-SHA iPhone 17 Pro / iOS 26.5 journeys passed with visible Buzz screenshots, finalized H.264 MP4s, and shutdown/erase cleanup:
    • test-results/native-review/61410c429fc6/ios_pairing/ios-20260815T100106-f971e4/receipt.json (31.625s video)
    • test-results/native-review/61410c429fc6/ios_pairing/ios-20260815T100219-ddcdaa/receipt.json (30.310s video)
  • deliberate missing pairing-toggle mutation failed with exact Flutter finder/stack diagnostics while still finalizing a 28.3s MP4, capturing the visible Buzz screen, writing a failed receipt, and passing simulator cleanup

Evidence semantics

The runner records the selected runtime and UDID, source SHA and dirty status, Flutter stdout/stderr, final screenshot, MP4, failure cause, and cleanup errors. A journey or cleanup failure returns nonzero. Native evidence is intentionally local because recordings are large and machine-specific.

Carl added 6 commits August 14, 2026 20:54
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
…urneys

* origin/main:
  fix(desktop): hide the offcanvas-collapsed sidebar so it stops painting over the community rail (block#5947)
  Polish mobile message threads and composer (block#5645)
  chore(release): release Buzz Desktop version 0.5.14 (block#5917)
  ci(release): remove desktop smoke gate (block#5914)

Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
wesbillman and others added 5 commits August 15, 2026 15:22
Fail closed on non-standard fixture ports, use fixed loopback database
coordinates, scrub repository-controlled subprocess environments, and
remove generated review keys when fixture seeding fails.

Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Apply absolute maximum budgets to the worst candidate sample while retaining
median-based relative regression comparisons. Cover the catastrophic-outlier
case directly.

Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Create a unique run-owned simulator, scrub credentials from iOS review subprocesses, and delete only the owned device during cleanup.

Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>

Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Default harness commands to a credential-free environment and isolate cleanup HOME without reconstructing the host environment.

Co-authored-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>

Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Signed-off-by: Carl <5f365698229751c0461f57bb03a4e93134e6e936bd7039ebe7b737282a43c754@buzz.block.builderlab.xyz>
(cherry picked from commit 3635687)

@themiguelamador themiguelamador left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking findings:

  1. The documented harness tests and desktop runner import PyYAML without installing or pinning it, so they fail from a clean checkout after Hermit activation.
  2. The AVAssetWriter loop leaves its frame clock unchanged during backpressure, making its sleep deadline remain in the past and causing a busy-spin that can perturb native performance results.
  3. scroll accepts a locator but posts at the pointer's prior screen position, so the declared target is ignored and the wrong view can scroll.
  4. The new iOS receipt does not conform to the checked-in receipt schema: it omits flow/timestamps/isolation/steps/measurements/performance and adds an undeclared device object. Publication and comparison consumers therefore receive incomplete provenance.
  5. Journey/performance validation accepts invalid durations, duplicate cohort receipts, booleans, and non-finite values, allowing late crashes or non-independent samples to satisfy the cohort minimum.
  6. The harness README still says iOS Simulator is a future phase and omits its owned-device safety contract, despite this PR adding the lane.

Fixed in Complear/buzz@2efeddec8 (branch review/pr-5971-fix): locked uv/PyYAML provisioning, backpressure-safe recording, locator-targeted scrolling, schema-compatible iOS receipts, fail-closed validation with tests, and corrected iOS documentation. Verification: native-review 39/39, Swift release build, wrapper validation, Python/JSON/shell syntax checks. The same stacked mobile/desktop changes were additionally validated on descendant #5972 with 1,417 mobile and 4,954 desktop tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants