Retry only failed Android E2E flows

[cipolleschi]

Summary:

Android Maestro E2E jobs currently stop after the first flow that exhausts its in-process retries. The workflow-level retry then starts the whole suite again, including flows that already passed.

This changes the retry model so that:

• each CI attempt runs every selected flow once and continues after individual failures;
• per-flow passed, failed, and pending state is saved atomically after every flow;
• cumulative results and attempt counts are shown in the GitHub job summary and uploaded as an artifact;
• retry workflows download that state and skip flows that already passed, running only failed or unfinished flows;
• a flavor whose downloaded state is already fully passed skips emulator startup and APK installation entirely;
• a missing state artifact falls back to running the full suite, so infrastructure failures remain retryable.

The Android E2E timeout is increased from 60 to 90 minutes to give the initial all-flows attempt enough time to finish. The existing three workflow attempts now provide up to three executions per failing flow instead of multiplying workflow retries by in-process retries.

CI follow-ups

• RNTester Debug completed all 40 flows with 38 passes and 2 failures. Retry 1 downloaded its state, skipped exactly those 38 passing flows, and executed only the two failures, validating the selective retry behavior.

• RNTester Release became unhealthy after an early flow failure and timed out while later flows were still pending. The state file was being written incrementally, but its artifact upload was inside the timed composite action and was killed by the same timeout. Moved state upload into the outer reusable workflow so if: always() preserves passed, failed, and pending results after an E2E timeout.

• The next Release retry downloaded the preserved state, skipped its 7 prior passes, reran the remaining flows, and recovered to 40/40. A later matrix retry could still start that completed flavor and fail during an unnecessary APK installation, so retry jobs now skip the entire E2E action when downloaded state is fully passed.

• Both test_js variants failed because the repository Jest preset throws whenever console.error is called, and the new failure-path unit test intentionally exercised a production error log. The test now mocks that expected log while continuing to assert the aggregate flow failure.

• RNTester Debug consistently failed image-wide-gamut.yml with exactly 83.974% screenshot similarity and legacy-native-module.yml before the APIs search became visible, while Release passed both immediately. Reproducing Debug with an offline Android emulator showed the actual cause: importing ImageExample.js eagerly starts a Facebook image prefetch, and its promise could reject before the Image Loading Events example attached its rejection callback. That produced an Uncaught (in promise): UnknownHostException LogBox notification, which covered the screenshot and intercepted the APIs tab. The prefetch now attaches a rejection handler immediately while preserving the existing example-level success/failure reporting. No LogBox dismissal or E2E-specific workaround remains.

• The remaining image-wide-gamut.yml failure was separate from LogBox: the flow explicitly accepted P3: error but then compared against a golden screenshot containing the successfully loaded Display-P3 fixture. The same ICC-profiled WebKit sample is now embedded as a compact data URI, removing the external network dependency, and the flow requires P3: loaded before screenshot comparison.

Changelog:

[INTERNAL] [CHANGED] - Retry only failed or unfinished Android Maestro E2E flows.

Test Plan:

• git diff --check (passed)

• node --check .github/workflow-scripts/maestro-android.js (passed)

• yarn jest .github/workflow-scripts/__tests__/maestro-android-test.js --runInBand --config '{"testEnvironment":"node","transform":{}}' (passed)

• ruby -ryaml -e "ARGV.each { |file| YAML.parse_file(file) }" .github/actions/maestro-android/action.yml .github/workflows/e2e-android-rntester.yml .github/workflows/e2e-android-templateapp.yml .github/workflows/test-all.yml (passed)

• yarn prettier --check .github/workflow-scripts/maestro-android.js .github/workflow-scripts/__tests__/maestro-android-test.js .github/actions/maestro-android/action.yml .github/workflows/e2e-android-rntester.yml .github/workflows/e2e-android-templateapp.yml .github/workflows/test-all.yml (passed)

• Verified the retry-state jq predicate returns false only when every recorded flow has passed.

• ./node_modules/.bin/prettier --check packages/rn-tester/js/examples/Image/ImageExample.js (passed)

• Reproduced on a local Android Debug emulator with external DNS unavailable. Before the fix, logcat reported Uncaught (in promise): UnknownHostException and LogBox covered the bottom navigation. After the fix, no unhandled rejection or LogBox appeared, the APIs tab displayed explorer_search, and the wide-gamut example was unobstructed.

• Verified on a local Android Debug emulator with external DNS unavailable that the embedded Display-P3 fixture reaches P3: loaded and renders without a network request.

• The next Debug run passed the stricter P3: loaded gate but still reproduced the exact 83.974% screenshot mismatch, followed by the legacy APIs-search failure. This confirms the embedded Display-P3 fixture is loading and that a second Debug-only UI state remains. The previous action skipped its Maestro-log upload because the emulator wrapper masked the inner script failure, and it pulled screen.mp4 before stopping the recorder, leaving an unplayable artifact. The runner now captures a PNG, UI hierarchy, logcat, and Metro output when each flow fails, stops screenrecord before pulling the MP4, and always uploads the diagnostic bundle. The next run will expose the actual obstructing UI so it can be fixed at its source.

• The first diagnostic run exposed a flaw in the new evidence collection itself: Debug remained inside the E2E action far beyond its normal duration because the uiautomator/ADB capture commands were unbounded. Every diagnostic subprocess now has a 15-second timeout, so a problem collecting evidence is logged but can never stall later flows or prevent state persistence.

• The captured screenshots and hierarchy identified the root cause shared by all remaining Debug failures (including one additional FlatList flow): StaticViewConfigValidator reported that AndroidTextInput.validAttributes.fontVariationSettings had the wrong value. The object-syntax change in Accept object font variation settings (#57929) #57929 added a JS processor to the static TextInput config, but runtime native view-config reflection still derived plain true from the native String prop. Debug validation therefore emitted a LogBox; its notifications covered the wide-gamut screenshot and FlatList controls, and its expanded console blocked the legacy APIs tab. Native view-config construction now recognizes fontVariationSettings and installs the same processor as the static config. This both makes validation agree and preserves object-to-string normalization when reflected native configs are active. A regression test verifies the reflected attribute matches the static processor and serializes object settings deterministically.

• Targeted regression test: yarn test packages/react-native/Libraries/ReactNative/__tests__/getNativeComponentAttributes-test.js --runInBand (passed).

• Both test_js variants then failed at ESLint before running tests because the new regression test lacked the required Flow file annotation and its CommonJS imports were not in repository order. Added @flow strict-local and reordered the requires; the targeted Jest test and Prettier check remain green.

• Final Android validation on bd55f0aa4226: the initial RNTester emulators lost ADB after three passing flows, so the outer state artifacts preserved those passes and marked the remaining work for retry. Retry 1 started fresh emulators, skipped the three prior passes, ran the remaining 37 flows, and finished with 40/40 passed in both Debug and Release; retry 2 was skipped. image-wide-gamut.yml, legacy-native-module.yml, and flatlist-inverted-recycle-maintainvisible.yml each passed on their first execution after the view-config fix. All Android checks are green, including Android builds, HelloWorld, TemplateApp E2E, and RNTester E2E.

[meta-codesync]

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this in D116463690.

[meta-codesync]

@cipolleschi merged this pull request in 4c022bb.