Skip to content

Integration tests using capybara (with selenium chromedriver) seem flaky #53

@laugmanuel

Description

@laugmanuel

It seems like integration tests using capybara are flaky and fail regularly if not run sequentially.

Currently, I suspect the problem in Github Actions environment and/or the fact, that tests run in parallel. When restarting the failed job one after the other, they complete successfully.

When running the tests with DEBUG_JS_TEST=1, I see the following erorr:

[MinitestRetry] retry 'test_0001_the index page works' count: 1,  msg: Selenium::WebDriver::Error::SessionNotCreatedError: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /opt/google/chrome/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
    test/integration_test_helper.rb:302:in `login_admin'
    test/test_helper.rb:84:in `before_setup'
    test/integration_test_helper.rb:331:in `before_setup'

I found the following issue in selenium: SeleniumHQ/selenium#12841
However, the recommended fix (adding -remote-debugging-pipe) did not help for me.

I noticed the problem here: dm-drogeriemarkt/foreman_dlm#67
A quick look into foreman_ansible also showed Actions runs showing the same behaviour:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions