Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
109 changes: 109 additions & 0 deletions hips/hip-9999.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
hip: 9999
title: "Check Kubernetes feature gates with a new flag in Chart"
authors: [ "rysurd <thegogolplex@gmail.com>" ]
created: "2026-08-11"
type: "feature"
status: "draft"
requires: ["0020"]
---

## Abstract

This proposal aims to resolve issue [31432](https://github.com/helm/helm/issues/31432), offering a way to check Kubernetes feature gates within Chart.yaml (Chart v3 only). This is an extra layer of compatibility between a Chart and a cluster, on top of the `kubeVersion` field. When a Chart is installed or upgraded, it will poll the k8s cluster for its feature gates, either per components and per nodes or per a representative node. If a feature gate required isn't activated on the component as declared in Chart.yaml, the installation/upgrade will fail.

## Motivation

There is currently no existing mechanism in Helm that allows polling to Kubernetes before CRDs installation and during installation/upgrade. Moreover, there is no mechanism in place that checks feature gates.

## Rationale

The original issue mentioned a flat-map type for feature gates inputs, in Chart.yaml. Because FGs are declared per-components and components are declared per-nodes, it has been proposed to adopt a nested, per-component map instead.

There are multiple design options available, notably adding this feature as being part of the core, or as a plugin. Currently, Helm's core cannot support this feature as it lacks a polling mechanism. To make this feature fully integrated with the core, multiple enablers need to be done. Notably about [Component Flagz](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/4828-component-flagz) feature enablement in Kubernetes. This enhancement is being implemented as of now but is not currently in GA.

There is another issue raised to propose the creation of a controller that can scrape all FGs on all nodes, components, etc ([141162](https://github.com/kubernetes/kubernetes/issues/141162)) but it is currently being discussed with no fixed, defined plan.

The most realistic way to implement that feature in Helm, then, would be an incremental development:
1. First as a plugin.
2. Then as a feature in Helm's core, as soon as a proper upstream API feature is delivered (either via /flagz or any other custom endpoints).

The second blocker for this enabler (Core) is the general availability of Chart v3.

## Specification

Adding a new field in Chart.yaml similar to this format:
```yaml
kubeFeatureGatesPolling: representative #could also be a choice between representative, per nodes, node-name specific, etc
kubeFeatureGates:
kubelet:
SidecarContainers: true
apiserver:
ConstrainedImpersonation: true
scheduler:
ComponentFlagz: false
```

## Backwards compatibility

Not compatible with chart v2, would only work with chart v3. As this would cause a Chart contract change, this feature would be compatible only with v3. Feature would cause a hard validation error in Chart v2.

## Security implications

Since we allow Helm to poll Kubernetes components, we have to sanitize any input and output plus secure the channel. Also, we need to ensure that the Helm operator that is executing the k8s commands on the cluster has the rights to access required Kubernetes components.

If implementing this as a core feature, this would have an impact on any installation as the Helm operator would need to have the right RBAC setting. If user don't/can't have the proper RBAC setting for this feature, then the whole feature would be turned off, with a warning raised to user with the missing RBAC settings listed.

If implementing this as a plugin, RBAC can be circumvented by an opt-in setting by users installing this plugin.

## How to teach this

This new field is opt-in and not used by default. It will be added to the documentation if implemented, with a guide on how to use this new feature.

## Reference implementation

There is a pending PR on Helm repo ; [feat(action): add KubeFeatureGates field in Chart definition to check against k8s feature gates](https://github.com/helm/helm/pull/32497).

Besides, the implementation depends on the design choice (Core vs plugin).

Core:

- Compatible only with Chart v3.
- Only possible if an enhancement is done in upstream API, via /flagz endpoint or another one to be done.
- If using /flagz
- Poll /flagz endpoint before CRDs install step.
- Require system:monitoring group permission.

Plugin:

- Compatible only with Chart v3.
- Can poll /metrics endpoints for each component and node.
- Polling happens before CRDs installation.

There are common decisions and pitfalls for both implementations:

- Node selector option:
- Choose "representative" option to tell the plugin to pick a representative k8s node to check FGs status. Caveat ; picking this option means accepting the risk that the FGs values might differ across nodes.
- Choose "all" option to loop all available nodes and for each node check their FGs per component
- Choose "nodes" option to provide a fixed-list of nodes (per user input) so the plugin will only poll on these specific nodes.
- Polling failure:
- If polling fails for a technical reason, aka endpoints not reachable, RBAC issue, etc, then the process would be fail-open, making as if the FGs check isn't activated.
- If polling succeeds and confirms the gate is absent/disabled, then the process would be fail-closed.
- Subchart treatment:
- Check for all subchart FGs, if there are any. If there are FGs checks, they are taken into consideration.
- If there are conflicts between any subchart and the main chart, then the conflict is rejected and install/update is aborted with an error.

Pitfalls:
Some k8s components like scheduler/controllerManager/cloudControllerManager/kubeProxy are often unreachable externally. For a tier-0 implementation, we should consider only apiserver+kubelet (until upstream API change and proper controller is implemented to poll all components).

## Rejected ideas

-

## Open issues

I suggested that we implement a new feature in Kubernetes directly that would expose an endpoint, through the apiserver, that exposes all feature gates information across all components and nodes. Ref ; [[Feature request] Single API endpoint for feature gates values](https://github.com/kubernetes/kubernetes/issues/141162)

## References

-