Skip to content

Harden the Flutter E2E flows against CI timing flakiness - #36

Open
brionmario wants to merge 1 commit into
thunder-id:mainfrom
brionmario:fix-flaky-e2e-flows
Open

Harden the Flutter E2E flows against CI timing flakiness#36
brionmario wants to merge 1 commit into
thunder-id:mainfrom
brionmario:fix-flaky-e2e-flows

Conversation

@brionmario

Copy link
Copy Markdown
Member

Purpose

The Flutter E2E suite is flaky. The nightly run on main failed on 2026-09-11 and passed on 2026-09-10 against the same commit (3312feb) and the same server release (v1.0.1, unchanged since August), and the run on #35, which changes only AGENTS.md, failed both flows:

[Failed] Sign up a new user, then sign in as them (2m 33s) (Assertion is false: "Get started" is visible)
[Failed] Sign in and sign out (1m 31s) (Assertion is false: "Session active" is visible)

Nothing in the SDK changed in that window, so these are timing failures rather than a regression.

The cause is that the Flutter flows never received the hardening the iOS and Android flows already carry. Comparing the three suites:

Repository retry blocks waitForAnimationToEnd
ios-sdks Yes, in both flows Yes
android-sdks Yes, in both flows Yes
flutter-sdks None None

Approach

Port the two mitigations the sibling repositories already use, with the same structure and the same reasoning recorded in comments.

waitForAnimationToEnd before the first input on each freshly rendered step. extendedWaitUntil ... visible fires the instant the field is on screen, which on a loaded runner can be before the widget has finished settling. Typing into a field mid-settle binds an empty string even though the keystrokes display correctly, because the input is discarded when the widget finishes building under it.

retry around each fill-and-submit block. A slow runner can take long enough filling the form that the server's credentials-step challenge expires before submission. The form silently resets instead of authenticating, so the assertion that follows never sees its target. That is an expired challenge, not a regression, so the block retries rather than failing outright. Each retry calls eraseText before typing, because a failed submission can leave the previous attempt's text in the field rather than coming back blank.

Applied to three places: the sign-in fill-and-submit in signin.yaml, and both the registration and the subsequent sign-in fill-and-submit blocks in signup.yaml. The thunderid-field-email wait after registration also moves from 20s to 30s, matching iOS, since CI renders that step noticeably slower than a local machine.

No product code changes, and no change to what the flows assert.

Related Issues

  • N/A

Related PRs

SDK parity

This brings flutter-sdks in line with hardening that ios-sdks and android-sdks already have, so no sibling pull requests are needed. The reverse direction is worth noting for a future change: the retry counts and timeouts now match across all three mobile suites, and should be kept that way.

Checklist

  • Followed the contribution guidelines.
  • Manual test round performed and verified.
    • Both flow files parse as valid Maestro YAML.
    • The real verification is the E2E job on this pull request, which exercises the changed flows directly.
  • Documentation provided. (Add links if there are any)
  • Tests provided. (Add links if there are any)
    • This pull request changes the tests themselves.
    • Unit Tests
    • Integration Tests
  • Breaking changes. (Fill if applicable)
    • Breaking changes section filled.
    • breaking change label added.

Security checks

  • Followed secure coding standards.
  • Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.

Signed-off-by: Brion <info@brionmario.com>
@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 726d9bdb-9a30-464d-b2ab-91c994c13052


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant