Skip to content

test: Declare thread usage of tests, to fix flakiness and speedup - #350

Open
furtib wants to merge 2 commits into
Ericsson:mainfrom
furtib:fix-too-flaky-test
Open

furtib wants to merge 2 commits into
Ericsson:mainfrom
furtib:fix-too-flaky-test

Conversation

@furtib

@furtib furtib commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Why:
Currently, the store tests are flaky.
This is because in them we start a CodeChecker server with a timeout of 10sec.
This normally wouldn't be an issue, but when competing for resources with other tests (that also might use more threads than their fair share), the CodeChecker server often times out, resulting in a test failure.

Tests that use bazel inside bazel calls are a notorious example of tests that take more than their fair share. (An extra Bazel server process.)

What:

  • Added a cpu=2 tag to store tests. -- This solved the flakiness.
  • Added a cpu=2 tag to unit tests that use nested Bazel calls. -- This in some cases halved the length of tests. (In CI, external tests passed in 12-20 seconds instead of 40-60sec)

Addresses:
none

@furtib
furtib requested a review from Szelethus October 9, 2026 12:50
@furtib furtib self-assigned this Oct 9, 2026
@furtib furtib added non-functional change ☮️ The patch doesn't change any functionality, e.g. refactoring, documentation, test-only. test ☑️ Adding or refactoring tests labels Oct 9, 2026
],
local = True,
tags = ["unit"] + tags,
# Each test runs a nested Bazel invocation, so it needs a second

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest this does not sound reasonable so far.

Comment thread test/unit/parse/BUILD
@@ -113,7 +113,13 @@ py_test(
data = [":report_files"],
local = True,
main = "store_check.py",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another question... What do we actually test in store_check.py?

size = size,
tags = tags,
# Each test runs a nested Bazel invocation, so it needs a second
# thread. Declaring 2 CPUs makes Bazel reserve the capacity instead of

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then... why only 2 CPU? Not 3 or 4? 100?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-functional change ☮️ The patch doesn't change any functionality, e.g. refactoring, documentation, test-only. test ☑️ Adding or refactoring tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants