Skip to content

Conversation

@emosbaugh
Copy link
Member

@emosbaugh emosbaugh commented Nov 27, 2025

Description, Motivation and Context

The cluster resources collector is writing replicaset resources to a directory called statefulsets-errors.json, instead of replicasets.

Fixes: #1916

Also fixes issues:

  • Disable auto-updates for regression suite. Previously suite ran against current release rather than the current change.
  • Runs each spec on different cluster to reduce flakiness.
  • ./scripts/update_baselines.sh "gh run download" command fails with "fatal not a git repository" error.

Checklist

  • New and existing tests pass locally with introduced changes.
  • Tests for the changes have been added (for bug fixes / features)
  • The commit message(s) are informative and highlight any breaking changes
  • Any documentation required has been added/updated. For changes to https://troubleshoot.sh/ create a PR here

Does this PR introduce a breaking change?

  • Yes
  • No

- name: Reorganize artifacts
run: |
# Move artifacts from nested directories to test/output
find test/output/test-results-* -type f -exec mv {} test/output/ \;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Reorganize artifacts step fails without downloaded artifacts

The find test/output/test-results-* command fails when no artifacts match the pattern because bash expands the glob to a literal string that doesn't exist. This prevents the "Check for regressions" and "Generate summary report" steps from running when test jobs fail before uploading artifacts. Adding || true or using if: always() on subsequent steps would ensure proper failure reporting even when artifacts are missing.

Fix in Cursor Fix in Web

fi
if [ "${{ steps.compare-supportbundle.outcome }}" == "failure" ] && [ "${{ steps.compare-supportbundle.outputs.baseline_missing }}" != "true" ]; then
if [ "${{ needs.test-supportbundle.result }}" == "failure" ]; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Skipped test jobs not detected as failures

The regression check only detects jobs with result failure but ignores jobs with result skipped. When the build-binaries job fails, test jobs are skipped, and the workflow incorrectly reports success instead of failure. The check should also count skipped jobs as failures to ensure the workflow fails when tests don't run.

Fix in Cursor Fix in Web

fi
if [ "${{ steps.compare-supportbundle.outcome }}" == "failure" ] && [ "${{ steps.compare-supportbundle.outputs.baseline_missing }}" != "true" ]; then
if [ "${{ needs.test-supportbundle.result }}" == "failure" ]; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Baseline update step skipped after regression failures

The Update baselines step won't execute when regressions are detected because the preceding Check for regressions step exits with code 1, terminating the workflow. The step condition lacks always() to ensure execution regardless of prior step failures. This prevents users from updating baselines when regressions exist, which defeats the purpose of the manual baseline update feature.

Fix in Cursor Fix in Web

@emosbaugh emosbaugh changed the title fix(ci): disable auto-updates for regression test suite fix(collect): cluster resource errors json file has wrong name Nov 27, 2025
@xavpaice xavpaice merged commit 9343b43 into main Nov 27, 2025
24 checks passed
@xavpaice xavpaice deleted the emosbaugh/fix-regression-test branch November 27, 2025 21:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cluster resources collector: replicasets output to wrong directory

3 participants