Skip to content

feat(ci): Last CI step checking for non-exercised tests#4533

Merged
pfcoperez merged 4 commits into
mainfrom
ci/missing-tests
Jul 6, 2026
Merged

feat(ci): Last CI step checking for non-exercised tests#4533
pfcoperez merged 4 commits into
mainfrom
ci/missing-tests

Conversation

@pfcoperez

@pfcoperez pfcoperez commented Jul 3, 2026

Copy link
Copy Markdown
Member

These changes add a last step in the build that compares the list of executed top level test functions with the list of existing top-level test functions in the source branch code.

📝 Doesn't detect missing runs of run-time generated tests like table based test cases.
📝 Only relevant for test cases defined under ./flow
📝 Future test selection implementations should propagate run selected to tests to the filter of this command. Related PoC: #4510

@pfcoperez
pfcoperez requested a review from a team as a code owner July 3, 2026 11:52
@claude

claude Bot commented Jul 3, 2026

Copy link
Copy Markdown

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

@pfcoperez

Copy link
Copy Markdown
Member Author

QA Validation

  1. Artificially excluded TestApiMongo in f5edf6c
  2. Confirmed the build failure:
image image

@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

❌ Test Failure

Analysis: Not flaky — the tests passed, but a deterministic CI meta-check failed identically across all matrix jobs because the newly added test TestApiMongo is never exercised in CI, indicating a missing CI wiring/config for that test.
Confidence: 0.96

⚠️ This appears to be a real bug - manual intervention needed

View workflow run

@pfcoperez pfcoperez added the ci-dev Improvements, new features or bug fixes in CI flow label Jul 6, 2026
@pfcoperez
pfcoperez enabled auto-merge (squash) July 6, 2026 07:53
@pfcoperez
pfcoperez merged commit 3cd6c9c into main Jul 6, 2026
14 checks passed
@pfcoperez
pfcoperez deleted the ci/missing-tests branch July 6, 2026 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-dev Improvements, new features or bug fixes in CI flow

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants