feat: improve acceptance tests

[psturc]

User description

https://issues.redhat.com/browse/KFLUXDP-596

• use native Go instrumentation
• report code coverage correctly per flag (one coverage output file per flag):

see the coverage report in PR against my fork

---

PR Type

Enhancement, Tests

---

Description

• Implement native Go instrumentation for acceptance tests

• Replace legacy coverage merging with Go's native covdata tool

• Add per-flag coverage reporting to codecov uploads

• Enable conditional instrumentation via E2E_INSTRUMENTATION flag

---

Diagram Walkthrough

flowchart LR
  A["E2E_INSTRUMENTATION flag"] -->|"enables coverage"| B["Build with -cover flags"]
  B --> C["Run acceptance tests"]
  C --> D["GOCOVERDIR stores raw data"]
  D --> E["covdata textfmt converts to output"]
  E --> F["codecov uploads per-flag reports"]

Loading

File Walkthrough

	Relevant files
Enhancement	cli.go Add GOCOVERDIR environment variable                                            acceptance/cli/cli.go Add GOCOVERDIR environment variable to pass coverage directory to acceptance tests Enables native Go coverage instrumentation for test execution +1/-0      Makefile Replace legacy coverage with native Go instrumentation      Makefile Add E2E_INSTRUMENTATION_FLAGS variable that conditionally adds -cover -covermode atomic flags Include instrumentation flags in binary build command when E2E_INSTRUMENTATION=true Replace legacy gocovmerge tool with native go tool covdata textfmt for coverage conversion Create coverage directory in acceptance test workspace Export GOCOVERDIR environment variable pointing to coverage data directory Remove -coverprofile flag from acceptance test execution +5/-4
Configuration changes	checks-codecov.yaml Configure codecov uploads with per-flag reporting                .github/workflows/checks-codecov.yaml Add disable_search: true to all three codecov upload steps (unit, generative, integration) Prevents codecov from searching for additional coverage files Add CODECOV_TOKEN environment variable to acceptance test upload step Enable E2E_INSTRUMENTATION=true flag for acceptance test execution +8/-1

---

[dheerajodha]

/ok-to-test

[qodo-merge-pro]

PR Compliance Guide 🔍

(Compliance updated until commit 783e7d2)

Below is a summary of compliance checks for this PR:

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: No audit logs: The new code adds environment variable propagation for coverage but does not introduce or modify any audit logging of critical actions, and it is unclear from the diff whether such logging exists elsewhere. Referred Code environment := []string{ "PATH=" + os.Getenv("PATH"), "GOCOVERDIR=" + os.Getenv("GOCOVERDIR"), // directory where the Go coverage raw data is stored "HOME=/tmp", } Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Missing error checks: The acceptance target chains commands with ';' and runs tools like 'go tool covdata' without explicit error checks or short-circuiting, which may mask failures during coverage generation. Referred Code export GOCOVERDIR="$${ACCEPTANCE_WORKDIR}/coverage"; \ cd acceptance && go test -timeout $(ACCEPTANCE_TIMEOUT) ./... ; go tool covdata textfmt -i=$${GOCOVERDIR} -o="$(ROOT_DIR)/coverage-acceptance.out" Learn more about managing compliance generic rules or creating your own custom rules

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

---

Previous compliance checks

Compliance check up to commit 6193854

Security Compliance
🟢	No security concerns identified No security vulnerabilities detected by AI analysis. Human verification advised for critical code.
Ticket Compliance
⚪	🎫 No ticket provided Create ticket/issue
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
🟢	Generic: Meaningful Naming and Self-Documenting Code Objective: Ensure all identifiers clearly express their purpose and intent, making code self-documenting Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Error Handling Objective: To prevent the leakage of sensitive system information through error messages while providing sufficient detail for internal debugging. Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Security-First Input Validation and Data Handling Objective: Ensure all data inputs are validated, sanitized, and handled securely to prevent vulnerabilities Status: Passed Learn more about managing compliance generic rules or creating your own custom rules
⚪	Generic: Comprehensive Audit Trails Objective: To create a detailed and reliable record of critical system actions for security analysis and compliance. Status: No audit logs: The new build/test steps and environment configuration introduce no logging of critical security-relevant actions, but this PR appears focused on coverage tooling so lack of audit trails may be acceptable. Referred Code acceptance: ## Run all acceptance tests @ACCEPTANCE_WORKDIR="$$(mktemp -d)"; \ cleanup() { \ cp "$${ACCEPTANCE_WORKDIR}"/features/__snapshots__/* "$(ROOT_DIR)"/features/__snapshots__/; \ }; \ mkdir -p "$${ACCEPTANCE_WORKDIR}/coverage"; \ trap cleanup EXIT; \ cp -R . "$$ACCEPTANCE_WORKDIR"; \ cd "$$ACCEPTANCE_WORKDIR" && \ $(MAKE) build && \ export COVERAGE_FILEPATH="$$ACCEPTANCE_WORKDIR"; \ export COVERAGE_FILENAME="-acceptance"; \ export GOCOVERDIR="$${ACCEPTANCE_WORKDIR}/coverage"; \ cd acceptance && go test -args -tags=@focus ./... ; go tool covdata textfmt -i=$${GOCOVERDIR} -o="$(ROOT_DIR)/coverage-acceptance.out" Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Robust Error Handling and Edge Case Management Objective: Ensure comprehensive error handling that provides meaningful context and graceful degradation Status: Missing error checks: The new acceptance target chains multiple commands without explicit failure handling or conditional checks, which may mask partial failures in coverage export. Referred Code acceptance: ## Run all acceptance tests @ACCEPTANCE_WORKDIR="$$(mktemp -d)"; \ cleanup() { \ cp "$${ACCEPTANCE_WORKDIR}"/features/__snapshots__/* "$(ROOT_DIR)"/features/__snapshots__/; \ }; \ mkdir -p "$${ACCEPTANCE_WORKDIR}/coverage"; \ trap cleanup EXIT; \ cp -R . "$$ACCEPTANCE_WORKDIR"; \ cd "$$ACCEPTANCE_WORKDIR" && \ $(MAKE) build && \ export COVERAGE_FILEPATH="$$ACCEPTANCE_WORKDIR"; \ export COVERAGE_FILENAME="-acceptance"; \ export GOCOVERDIR="$${ACCEPTANCE_WORKDIR}/coverage"; \ cd acceptance && go test -args -tags=@focus ./... ; go tool covdata textfmt -i=$${GOCOVERDIR} -o="$(ROOT_DIR)/coverage-acceptance.out" Learn more about managing compliance generic rules or creating your own custom rules
	Generic: Secure Logging Practices Objective: To ensure logs are useful for debugging and auditing without exposing sensitive information like PII, PHI, or cardholder data. Status: Token exposure risk: The workflow passes CODECOV_TOKEN via environment which is standard but ensure it is masked in logs; this PR adds steps using the secret yet no explicit redaction is shown in logs. Referred Code env: CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }} with: Learn more about managing compliance generic rules or creating your own custom rules

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
acceptance	57.39% <ø> (?)
generative	19.27% <ø> (-49.54%)	⬇️
integration	28.56% <ø> (-40.24%)	⬇️
unit	68.37% <ø> (-0.44%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 49 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Fix invalid build flag Remove the trailing comma from the E2E_INSTRUMENTATION_FLAGS variable in the Makefile to fix an invalid build flag that would cause the build to fail. Makefile [16] -E2E_INSTRUMENTATION_FLAGS := $(if $(filter $(E2E_INSTRUMENTATION),true),-cover -covermode atomic,) +E2E_INSTRUMENTATION_FLAGS := $(if $(filter $(E2E_INSTRUMENTATION),true),-cover -covermode atomic) Apply / Chat Suggestion importance[1-10]: 10 __ Why: The suggestion correctly identifies a typo (a trailing comma) in the E2E_INSTRUMENTATION_FLAGS variable that would cause the go build command to fail, breaking the build for acceptance tests.	High
	Ensure all acceptance tests run Remove the -args -tags=@focus from the go test command in the acceptance target in the Makefile to ensure all acceptance tests are executed, not just a focused subset. Makefile [132] -cd acceptance && go test -args -tags=@focus ./... ; go tool covdata textfmt -i=$${GOCOVERDIR} -o="$(ROOT_DIR)/coverage-acceptance.out" +cd acceptance && go test ./... ; go tool covdata textfmt -i=$${GOCOVERDIR} -o="$(ROOT_DIR)/coverage-acceptance.out" Apply / Chat Suggestion importance[1-10]: 9 __ Why: The suggestion correctly points out that the acceptance make target is incorrectly configured to run only a subset of tests (@focus), which would lead to an incomplete and misleading CI run.	High
Update

[dheerajodha]

/ok-to-test

[dheerajodha]

/ok-to-test

[simonbaird]

The Conforma failure is unrelated to this change. I'll aim to merge something that will fix it shortly.

[simonbaird]

Okay, a rebase on upstream/main now should eliminate that CI failure. Specifically #3036 is the relevant change.

[dheerajodha]

/ok-to-test

[dheerajodha]

/ok-to-test

[dheerajodha]

lgtm