Skip to content

[SPARK-56115] Support crds.install in Helm Chart #641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TQJADE wants to merge 4 commits into from
Closed

[SPARK-56115] Support crds.install in Helm Chart #641

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4e36717
Introduce crds.install to Helm Chart
TQJADE Apr 20, 2026
b2197df
Support crd.annotations
TQJADE Apr 23, 2026
4bfbb74
Second operator installation should not install crd
TQJADE Apr 25, 2026
3825979
Update operations doc for helm upgrade changes
TQJADE Apr 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions ...parkapplications.spark.apache.org-v1.yaml → ...parkapplications.spark.apache.org-v1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,15 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- if .Values.crds.install }}
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: sparkapplications.spark.apache.org
{{- with .Values.crds.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
group: spark.apache.org
names:
Expand Down Expand Up @@ -26511,3 +26516,4 @@ spec:
storage: true
subresources:
status: {}
{{- end }}
6 changes: 6 additions & 0 deletions ...ds/sparkclusters.spark.apache.org-v1.yaml → ...ds/sparkclusters.spark.apache.org-v1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,15 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- if .Values.crds.install }}
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: sparkclusters.spark.apache.org
{{- with .Values.crds.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
group: spark.apache.org
names:
Expand Down Expand Up @@ -22119,3 +22124,4 @@ spec:
storage: true
subresources:
status: {}
{{- end }}
18 changes: 18 additions & 0 deletions build-tools/helm/spark-kubernetes-operator/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,24 @@
"type": "string",
"description": "Override fully qualified app name"
},
"crds": {
"type": "object",
"description": "CRD installation configuration",
"required": ["install"],
"properties": {
"install": {
"type": "boolean",
"description": "Install CRDs with the chart. Set to false if CRDs are managed separately."
},
"annotations": {
"type": "object",
"description": "Annotations to add to CRD resources. Default includes helm.sh/resource-policy: keep to prevent CRD deletion on helm uninstall.",
"additionalProperties": {
"type": "string"
}
}
}
},
"image": {
"type": "object",
"description": "Operator image configuration",
Expand Down
5 changes: 5 additions & 0 deletions build-tools/helm/spark-kubernetes-operator/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ image:

imagePullSecrets: [ ]

crds:
install: true
annotations:
"helm.sh/resource-policy": keep

operatorDeployment:
# Replicas must be 1 unless leader election is enabled
replicas: 1
Expand Down
19 changes: 18 additions & 1 deletion docs/operations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ following table:

| Parameters | Description | Default value |
|------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
| crds.install | Whether to install CRDs with the chart. Set to `false` when CRDs are managed externally or when deploying additional operator instances. | true |
| crds.annotations | Annotations to add to CRD resources. The default prevents CRDs from being deleted on `helm uninstall`. | `"helm.sh/resource-policy": keep` |
| image.repository | The image repository of spark-kubernetes-operator. | apache/spark-kubernetes-operator |
| image.pullPolicy | The image pull policy of spark-kubernetes-operator. | IfNotPresent |
| image.tag | The image tag of spark-kubernetes-operator. | 0.9.0-SNAPSHOT |
Expand Down Expand Up @@ -156,6 +158,21 @@ metadata:
"spark.operator/sentinel": "true"
```

## Upgrading from chart versions that used the `crds/` directory

Older chart versions installed CRDs via Helm's special `crds/` directory, which does not
track ownership metadata. The current chart manages CRDs as regular templates. Before
running `helm upgrade`, you must label and annotate the existing CRDs so Helm can adopt
them. Replace `<release-name>` and `<release-namespace>` with your Helm release values:

```bash
kubectl label crd sparkapplications.spark.apache.org app.kubernetes.io/managed-by=Helm
kubectl annotate crd sparkapplications.spark.apache.org meta.helm.sh/release-name=<release-name> meta.helm.sh/release-namespace=<release-namespace>

kubectl label crd sparkclusters.spark.apache.org app.kubernetes.io/managed-by=Helm
kubectl annotate crd sparkclusters.spark.apache.org meta.helm.sh/release-name=<release-name> meta.helm.sh/release-namespace=<release-namespace>
```

## Example

Install HelmChart at `us-west-1` and `us-west-2` namespaces.
Expand All @@ -165,7 +182,7 @@ helm install us-west-1 spark/spark-kubernetes-operator --create-namespace --name
```

```bash
helm install us-west-2 spark/spark-kubernetes-operator --create-namespace --namespace us-west-2 --set operatorRbac.clusterRole.name=spark-operator-clusterrole-us-west-2 --set operatorRbac.clusterRoleBinding.name=spark-operator-clusterrolebinding-us-west-2 --set workloadResources.clusterRole.name=spark-workload-clusterrole-us-west-2
helm install us-west-2 spark/spark-kubernetes-operator --create-namespace --namespace us-west-2 --set crds.install=false --set operatorRbac.clusterRole.name=spark-operator-clusterrole-us-west-2 --set operatorRbac.clusterRoleBinding.name=spark-operator-clusterrolebinding-us-west-2 --set workloadResources.clusterRole.name=spark-workload-clusterrole-us-west-2
```

Check installation.
Expand Down
14 changes: 5 additions & 9 deletions spark-operator-api/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ tasks.register("assertGeneratedCRDMatchesHelmChart") {
def currentPath = projectDir.absolutePath
doLast {
def generatedCRDFileBase = "$currentPath/build/resources/main/"
def stagedCRDFileBase = "$currentPath/../build-tools/helm/spark-kubernetes-operator/crds/"
def stagedCRDFileBase = "$currentPath/../build-tools/helm/spark-kubernetes-operator/templates/crds/"
Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this safe?

def generatedAppCRD = [
"yq",
"e",
Expand All @@ -82,16 +82,12 @@ tasks.register("assertGeneratedCRDMatchesHelmChart") {
"${generatedCRDFileBase}sparkclusters.spark.apache.org-v1.yml"
].execute().text.trim()
def stagedAppCRD = [
"yq",
"e",
".spec.versions[-1]",
"${stagedCRDFileBase}sparkapplications.spark.apache.org-v1.yaml"
"bash", "-c",
"sed '/{{/d' ${stagedCRDFileBase}sparkapplications.spark.apache.org-v1.yaml | yq e '.spec.versions[-1]' -"
].execute().text.trim()
def stagedClusterCRD = [
"yq",
"e",
".spec.versions[-1]",
"${stagedCRDFileBase}sparkclusters.spark.apache.org-v1.yaml"
"bash", "-c",
"sed '/{{/d' ${stagedCRDFileBase}sparkclusters.spark.apache.org-v1.yaml | yq e '.spec.versions[-1]' -"
].execute().text.trim()
if (generatedAppCRD != stagedAppCRD || generatedClusterCRD != stagedClusterCRD) {
def errorMessage = new StringBuilder("Generated CRD yaml does not match the staged " +
Expand Down
3 changes: 3 additions & 0 deletions tests/e2e/helm/dynamic-config-values-2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.

crds:
install: false

workloadResources:
namespaces:
overrideWatchedNamespaces: false
Expand Down
Loading