Skip to content

GCP-431: feat: Add CNCC support for GCP WIF in HyperShift#7824

Open
apahim wants to merge 8 commits intoopenshift:mainfrom
apahim:cncc
Open

GCP-431: feat: Add CNCC support for GCP WIF in HyperShift#7824
apahim wants to merge 8 commits intoopenshift:mainfrom
apahim:cncc

Conversation

@apahim
Copy link
Copy Markdown
Contributor

@apahim apahim commented Feb 27, 2026

What this PR does / why we need it:

Adds CNCC (Cloud Network Config Controller) support for GCP Workload Identity Federation in HyperShift HCP mode. Currently HyperShift does not provision WIF credentials
for CNCC on GCP, so CNCC cannot authenticate to GCP APIs to manage egress IPs and subnets.

This PR:

  • Adds a Network field to GCPServiceAccountsEmails API for the CNCC service account
  • Adds a cloud-network service account entry to iam-bindings.json with fine-grained CNCC permissions (compute.instances.get, compute.subnetworks.use, etc.)
  • Provisions a cloud-network-config-controller-creds secret in ReconcileCredentials when the Network GSA email is configured
  • Sets GCP_CNCC_CREDENTIALS_FILE env var on the CNO deployment for GCP platform, so CNO can wire GOOGLE_APPLICATION_CREDENTIALS on the CNCC container

Which issue(s) this PR fixes:

Fixes GCP-431

Special notes for your reviewer:

This PR is part of a cross-repo effort:

The Network field is +required, consistent with the other GSA fields. The iam-bindings.json entry uses predefined roles (roles/compute.instanceAdmin.v1 + roles/compute.networkUser), following the same pattern as the other service accounts.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • New Features
    • Added a GCP network service-account field for hosted clusters with immutability and project-scoped validation.
    • Enabled Cloud Network Config Controller (CNCC) support: new cloud-network service account, automatic CNCC credential provisioning, and a control-plane env var to expose CNCC credentials.
    • Added a CLI flag to supply the network service account during cluster creation.
  • Tests
    • Added serialization/compatibility and controller tests for the new network field.

@openshift-ci-robot
Copy link
Copy Markdown

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 27, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Feb 27, 2026

@apahim: This pull request references GCP-431 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Adds CNCC (Cloud Network Config Controller) support for GCP Workload Identity Federation in HyperShift HCP mode. Currently HyperShift does not provision WIF credentials
for CNCC on GCP, so CNCC cannot authenticate to GCP APIs to manage egress IPs and subnets.

This PR:

  • Adds a Network field to GCPServiceAccountsEmails API for the CNCC service account
  • Adds a cloud-network service account entry to iam-bindings.json with fine-grained CNCC permissions (compute.instances.get, compute.subnetworks.use, etc.)
  • Provisions a cloud-network-config-controller-creds secret in ReconcileCredentials when the Network GSA email is configured
  • Sets GCP_CNCC_CREDENTIALS_FILE env var on the CNO deployment for GCP platform, so CNO can wire GOOGLE_APPLICATION_CREDENTIALS on the CNCC container

Which issue(s) this PR fixes:

Fixes GCP-431

Special notes for your reviewer:

This PR is part of a cross-repo effort:

The Network field is +optional (unlike the other GSA fields which are +required) since existing clusters won't have it set. The iam-bindings.json entry uses permissions
instead of roles because CNCC needs fine-grained permissions that don't map to a single predefined GCP role.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Feb 27, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a Network service account field to GCP workload identity (immutable, optional) and a validation ensuring it is scoped to the same GCP project; adds a new cloud-network IAM binding entry mapping to a Kubernetes SA; sets a CNO env var for CNCC credentials when platform is GCP; reconciles an additional CNCC credentials secret and exposes CNCCCredsSecret(...) constructor; introduces a CLI flag to set the network service account and updates tests for JSON serialization compatibility and fixtures.

Changes

Cohort / File(s) Summary
GCP Platform Configuration
api/hypershift/v1beta1/gcp.go
Added Network field to GCPServiceAccountsEmails with json:"network,omitempty" and immutability XValidation; added XValidation on GCPPlatformSpec enforcing that workloadIdentity.serviceAccountsEmails.network (when present) ends with @<project>.iam.gserviceaccount.com.
GCP IAM Bindings
cmd/infra/gcp/iam-bindings.json
Added cloud-network service account entry (displayName, description) with permissions for network interface, subnet, and zone operations; maps to Kubernetes SA openshift-cloud-network-config-controller/cloud-network-config-controller.
CNO Deployment Configuration
control-plane-operator/controllers/hostedcontrolplane/v2/cno/deployment.go
When platform is GCP, add env var GCP_CNCC_CREDENTIALS_FILE=application_default_credentials.json to the CNO deployment.
GCP Credential Management
hypershift-operator/controllers/hostedcluster/internal/platform/gcp/gcp.go
ReconcileCredentials now conditionally includes the Network workload identity mapping and adds CNCCCredsSecret(controlPlaneNamespace string) *corev1.Secret to construct the CNCC credentials secret named cloud-network-config-controller-creds.
CLI / Tests
cmd/cluster/gcp/create.go, cmd/cluster/gcp/create_test.go, api/hypershift/v1beta1/gcp_test.go, hypershift-operator/.../gcp_test.go
Added CLI flag --network-service-account and RawCreateOptions.NetworkServiceAccount; ApplyPlatformSpecifics populates WorkloadIdentity.ServiceAccountsEmails.Network; added JSON serialization compatibility test and updated test fixtures to include the new Network field.
Lint config
api/.golangci.yml
Excluded _test.go files from the kubeapilinter linter via an additions to linters.exclusions.rules.

Sequence Diagram(s)

sequenceDiagram
    participant CLI as CLI / CreateOptions
    participant Controller as HostedCluster Controller
    participant Platform as GCP Platform Spec
    participant CredMgr as GCP Credential Manager
    participant K8s as Kubernetes API
    participant CNO as Cluster Network Operator

    CLI->>Controller: ApplyPlatformSpecifics(with network SA)
    Controller->>Platform: Set WorkloadIdentity.ServiceAccountsEmails.Network
    Controller->>CredMgr: ReconcileCredentials()
    alt Network SA configured
        CredMgr->>K8s: Ensure Secret "cloud-network-config-controller-creds" in control-plane namespace
        K8s-->>CredMgr: Secret created/reconciled
        CredMgr-->>Controller: Credentials mapping added
        Controller->>CNO: Deploy with env GCP_CNCC_CREDENTIALS_FILE=application_default_credentials.json
        CNO->>K8s: Mount credentials secret
    else Network SA not configured
        Controller->>CNO: Deploy without CNCC credentials env/secret
    end
Loading

Estimated code review effort

Moderate (~20–40 minutes)

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: adding Cloud Network Config Controller (CNCC) support for GCP Workload Identity Federation in HyperShift, matching the PR objectives.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Feb 27, 2026

@apahim: This pull request references GCP-431 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Adds CNCC (Cloud Network Config Controller) support for GCP Workload Identity Federation in HyperShift HCP mode. Currently HyperShift does not provision WIF credentials
for CNCC on GCP, so CNCC cannot authenticate to GCP APIs to manage egress IPs and subnets.

This PR:

  • Adds a Network field to GCPServiceAccountsEmails API for the CNCC service account
  • Adds a cloud-network service account entry to iam-bindings.json with fine-grained CNCC permissions (compute.instances.get, compute.subnetworks.use, etc.)
  • Provisions a cloud-network-config-controller-creds secret in ReconcileCredentials when the Network GSA email is configured
  • Sets GCP_CNCC_CREDENTIALS_FILE env var on the CNO deployment for GCP platform, so CNO can wire GOOGLE_APPLICATION_CREDENTIALS on the CNCC container

Which issue(s) this PR fixes:

Fixes GCP-431

Special notes for your reviewer:

This PR is part of a cross-repo effort:

The Network field is +optional (unlike the other GSA fields which are +required) since existing clusters won't have it set. The iam-bindings.json entry uses permissions
instead of roles because CNCC needs fine-grained permissions that don't map to a single predefined GCP role.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@apahim
Copy link
Copy Markdown
Contributor Author

apahim commented Feb 27, 2026

/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 27, 2026
@openshift-ci openshift-ci Bot requested review from muraee and patjlm February 27, 2026 14:20
@openshift-ci openshift-ci Bot added area/api Indicates the PR includes changes for the API area/cli Indicates the PR includes changes for CLI area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release area/platform/gcp PR/issue for GCP (GCPPlatform) platform and removed do-not-merge/needs-area labels Feb 27, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Feb 27, 2026

@apahim: This pull request references GCP-431 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Adds CNCC (Cloud Network Config Controller) support for GCP Workload Identity Federation in HyperShift HCP mode. Currently HyperShift does not provision WIF credentials
for CNCC on GCP, so CNCC cannot authenticate to GCP APIs to manage egress IPs and subnets.

This PR:

  • Adds a Network field to GCPServiceAccountsEmails API for the CNCC service account
  • Adds a cloud-network service account entry to iam-bindings.json with fine-grained CNCC permissions (compute.instances.get, compute.subnetworks.use, etc.)
  • Provisions a cloud-network-config-controller-creds secret in ReconcileCredentials when the Network GSA email is configured
  • Sets GCP_CNCC_CREDENTIALS_FILE env var on the CNO deployment for GCP platform, so CNO can wire GOOGLE_APPLICATION_CREDENTIALS on the CNCC container

Which issue(s) this PR fixes:

Fixes GCP-431

Special notes for your reviewer:

This PR is part of a cross-repo effort:

The Network field is +optional (unlike the other GSA fields which are +required) since existing clusters won't have it set. The iam-bindings.json entry uses permissions
instead of roles because CNCC needs fine-grained permissions that don't map to a single predefined GCP role.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

Release Notes

  • New Features
  • Added network configuration field for GCP hosted clusters with validation enforcement
  • Enabled Cloud Network Config Controller support for GCP platforms with automatic credential provisioning
  • Introduced new service account with granular permissions for cloud network configuration, including subnet and network interface management capabilities

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@apahim apahim marked this pull request as draft February 27, 2026 14:44
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 27, 2026
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 8, 2026
@openshift-merge-robot
Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci openshift-ci Bot added area/documentation Indicates the PR includes changes for documentation area/testing Indicates the PR includes changes for e2e testing needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Mar 30, 2026
apahim and others added 3 commits April 16, 2026 21:00
Add the --network-service-account CLI flag for specifying the CNCC
Google Service Account email when creating a GCP hosted cluster.
The flag is required. Includes unit tests and updated test fixture.

Signed-off-by: Amador Pahim <apahim@redhat.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Wire the --e2e.gcp-network-sa flag and GCPNetworkServiceAccount field
into the e2e test framework. Update API UX validation test with the
Network field.

Signed-off-by: Amador Pahim <apahim@redhat.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update GCP hosted cluster creation and IAM docs with the new
--network-service-account flag and cloud-network service account.
Includes generated API reference and aggregated docs.

Signed-off-by: Amador Pahim <apahim@redhat.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@apahim
Copy link
Copy Markdown
Contributor Author

apahim commented Apr 16, 2026

/retest-required

@apahim
Copy link
Copy Markdown
Contributor Author

apahim commented Apr 16, 2026

/test e2e-gke

@apahim
Copy link
Copy Markdown
Contributor Author

apahim commented Apr 16, 2026

/test e2e-v2-gke

@cblecker
Copy link
Copy Markdown
Member

/retest

…countDefinitions

The unparam linter flagged that customConfigPath always receives an
empty string. Remove the parameter and simplify the function to always
use the embedded default configuration.

Signed-off-by: Amador Pahim <apahim@redhat.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Amador Pahim <apahim@redhat.com>
@cblecker
Copy link
Copy Markdown
Member

/retest

1 similar comment
@cristianoveiga
Copy link
Copy Markdown
Contributor

/retest

@cblecker
Copy link
Copy Markdown
Member

/test e2e-v2-gke

@cblecker
Copy link
Copy Markdown
Member

/lgtm

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws

@hypershift-jira-solve-ci
Copy link
Copy Markdown

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2046296266266120192 | Cost: $1.6593806999999996 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@csrwng
Copy link
Copy Markdown
Contributor

csrwng commented Apr 21, 2026

/approve

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 21, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: apahim, csrwng, muraee

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cblecker
Copy link
Copy Markdown
Member

/retest-required

@hypershift-jira-solve-ci
Copy link
Copy Markdown

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2046419048152961024 | Cost: $2.1344776000000008 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@cblecker
Copy link
Copy Markdown
Member

/retest-required

@hypershift-jira-solve-ci
Copy link
Copy Markdown

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2046588062951018496 | Cost: $2.9581085 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@apahim
Copy link
Copy Markdown
Contributor Author

apahim commented Apr 22, 2026

/verified by @apahim

@openshift-ci-robot
Copy link
Copy Markdown

@apahim: This PR has been marked as verified by @apahim.

Details

In response to this:

/verified by @apahim

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD f2fd2ca and 2 for PR HEAD 2eef5fe in total

@hypershift-jira-solve-ci
Copy link
Copy Markdown

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2046943457359958016 | Cost: $4.5674927499999995 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 5b382b8 and 1 for PR HEAD 2eef5fe in total

@hypershift-jira-solve-ci
Copy link
Copy Markdown

hypershift-jira-solve-ci Bot commented Apr 22, 2026

Now I have a complete picture. Let me produce the final report:

Test Failure Analysis Complete

Job Information

  • Prow Job: pull-ci-openshift-hypershift-main-e2e-kubevirt-aws-ovn-reduced
  • Build ID: 2046943459855568896
  • Target: e2e-kubevirt-aws-ovn-reduced
  • PR: #7824GCP-431: feat: Add CNCC support for GCP WIF in HyperShift
  • Duration: ~2h49m (13:25 – 16:14 UTC)
  • Failed Tests: TestCreateCluster/ValidateHostedCluster, TestAutoscaling/ValidateHostedCluster (4 of 6 test cases failed)

Test Failure Analysis

Error

controlPlaneVersion state is Partial, expected Completed

Summary

Both TestCreateCluster and TestAutoscaling failed because the HostedCluster's controlPlaneVersion never transitioned from Partial to Completed. Despite the CVO (ClusterVersionOperator) successfully completing the rollout of all cluster operators, the HyperShift control-plane-operator's checkOperandsRolloutStatus function blocked the transition because it couldn't find the cloud-network-config-controller (CNCC) deployment. CNCC is a cloud-provider-specific controller that is not deployed on KubeVirt platforms, but the operand check is gated on NetworkType == OVNKubernetes rather than platform type, causing a false-negative on KubeVirt clusters that use OVN networking. This is a pre-existing bug on main — the checkOperandsRolloutStatus code in cno/component.go is identical between main and this PR branch.

Root Cause

The root cause is in control-plane-operator/controllers/hostedcontrolplane/v2/cno/component.go, function checkOperandsRolloutStatus() (line ~112). This function checks that CNO operand deployments have rolled out successfully before marking controlPlaneVersion as Completed.

The CNCC deployment (cloud-network-config-controller) is included in the operands check list whenever NetworkType == OVNKubernetes:

if cpContext.HCP.Spec.Networking.NetworkType == hyperv1.OVNKubernetes {
    operandsDeploymentsList = []operand{
        {DeploymentName: "ovnkube-control-plane", ...},
        {DeploymentName: "network-node-identity", ...},
        {DeploymentName: "cloud-network-config-controller", ...},  // ← Problem
    }
}

However, CNCC is only deployed on cloud platforms (AWS, Azure, GCP, OpenStack) that need cloud-specific network configuration. KubeVirt clusters do not deploy CNCC, but they DO use OVN networking (networkType: OVNKubernetes).

A previous fix (commit 1f4b7c931b, Nov 2025) attempted to solve this by skipping the check when NetworkType is Other, but KubeVirt uses OVNKubernetes, not Other, so the fix was insufficient.

The cluster-network-operator ControlPlaneComponent remains stuck with:

  • RolloutComplete: False
  • Reason: WaitingForOperands
  • Message: failed to get deployment cloud-network-config-controller: Deployment.apps "cloud-network-config-controller" not found

Since allComponentsAtVersion() requires every CPC to have RolloutComplete: True AND a populated version field, the CNO's stuck state prevents controlPlaneVersion from transitioning to Completed, causing the 10-minute validation timeout.

This is a pre-existing bug on main, not introduced by PR #7824. The code in cno/component.go is byte-for-byte identical between main (base SHA f2fd2ca7) and the PR head (2eef5fe2).

Recommendations
  1. Fix the CNCC operand check to be platform-aware — The checkOperandsRolloutStatus function should gate the CNCC check on platform type, not just network type. CNCC should only be checked on platforms that actually deploy it (AWS, Azure, GCP, OpenStack):

    if cpContext.HCP.Spec.Networking.NetworkType == hyperv1.OVNKubernetes {
        operandsDeploymentsList = []operand{
            {DeploymentName: "ovnkube-control-plane", ...},
            {DeploymentName: "network-node-identity", ...},
        }
        // Only check CNCC on cloud platforms that deploy it
        if cpContext.HCP.Spec.Platform.Type == hyperv1.AWSPlatform ||
           cpContext.HCP.Spec.Platform.Type == hyperv1.AzurePlatform ||
           cpContext.HCP.Spec.Platform.Type == hyperv1.GCPPlatform ||
           cpContext.HCP.Spec.Platform.Type == hyperv1.OpenStackPlatform {
            operandsDeploymentsList = append(operandsDeploymentsList, operand{
                DeploymentName: "cloud-network-config-controller", ...
            })
        }
    }
  2. File this as a separate bug — Since this is a pre-existing issue on main, file a bug against the HyperShift team to fix checkOperandsRolloutStatus independently of this PR.

  3. Re-run the CI job — This failure is not caused by the changes in PR GCP-431: feat: Add CNCC support for GCP WIF in HyperShift #7824. A /retest should produce the same failure since the bug is on main. The fix for the CNCC operand check needs to land on main before this test will pass for any PR.

Evidence
Evidence Detail
Failing step run-e2e-local (test phase) — exited with code 1 after 33m53s
Failed tests TestCreateCluster/ValidateHostedCluster (1532s), TestAutoscaling/ValidateHostedCluster (1540s)
Failure message controlPlaneVersion state is Partial, expected Completed — 10m timeout exceeded
Stuck CPC (TestCreateCluster) cluster-network-operatorRolloutComplete: False, reason: WaitingForOperands, message: failed to get deployment cloud-network-config-controller: Deployment.apps "cloud-network-config-controller" not found
Stuck CPC (TestAutoscaling) cluster-network-operator — identical: RolloutComplete: False, cloud-network-config-controller not found
CVO status Completed — CVO finished applying 5.0.0-0.ci-2026-04-22-134024 successfully at 15:18:28 UTC
All other CPCs 38/39 ControlPlaneComponents have version set to expected; only cluster-network-operator has no version
HostedCluster networkType OVNKubernetes (triggers the CNCC operand check)
HostedCluster platform KubeVirt (does not deploy CNCC)
Pre-existing on main cno/component.go is identical between main (f2fd2ca7) and PR HEAD (2eef5fe2) — diff shows 0 changes to this file
PR changes to CNO Only cno/deployment.go modified — adds GCP_CNCC_CREDENTIALS_FILE env var gated on Platform.Type == GCPPlatform
Previous fix attempt Commit 1f4b7c931b (Nov 2025) skips check for NetworkType: Other, but KubeVirt uses OVNKubernetes
Responsible code control-plane-operator/controllers/hostedcontrolplane/v2/cno/component.go:112-134checkOperandsRolloutStatus()

@hypershift-jira-solve-ci
Copy link
Copy Markdown

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2046984293258891264 | Cost: $4.305417650000001 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 22, 2026

@apahim: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/unit 58f6456 link true /test unit
ci/prow/docs-preview 5aa9c3c link false /test docs-preview
ci/prow/e2e-aks-4-21 7359f6b link true /test e2e-aks-4-21
ci/prow/e2e-v2-gke 2eef5fe link false /test e2e-v2-gke
ci/prow/e2e-gke 2eef5fe link false /test e2e-gke
ci/prow/e2e-azure-self-managed 2eef5fe link true /test e2e-azure-self-managed
ci/prow/e2e-kubevirt-aws-ovn-reduced 2eef5fe link true /test e2e-kubevirt-aws-ovn-reduced

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 0f75517 and 0 for PR HEAD 2eef5fe in total

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/api Indicates the PR includes changes for the API area/ci-tooling Indicates the PR includes changes for CI or tooling area/cli Indicates the PR includes changes for CLI area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/documentation Indicates the PR includes changes for documentation area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release area/platform/gcp PR/issue for GCP (GCPPlatform) platform area/testing Indicates the PR includes changes for e2e testing jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants