Commit 94a043d
[SPARK-45670][CORE][3.3] SparkSubmit does not support
This is the cherry-pick of #43536 for branch-3.3
### What changes were proposed in this pull request?
Remove Kubernetes from the support list of `--total-executor-cores` in SparkSubmit
### Why are the changes needed?
`--total-executor-cores` does not take effect in Spark on K8s, [the
comments from original PR](#19717 (comment)) also proves that
### Does this PR introduce _any_ user-facing change?
The output of `spark-submit --help` changed
```patch
...
- Spark standalone, Mesos and Kubernetes only:
+ Spark standalone and Mesos only:
--total-executor-cores NUM Total cores for all executors.
...
```
### How was this patch tested?
Pass GA and review.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #43548 from pan3793/SPARK-45670-3.3.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>--total-executor-cores when deploying on K8s1 parent 1bb4143 commit 94a043d
File tree
2 files changed
+2
-2
lines changed- core/src/main/scala/org/apache/spark/deploy
2 files changed
+2
-2
lines changedLines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
647 | 647 | | |
648 | 648 | | |
649 | 649 | | |
650 | | - | |
| 650 | + | |
651 | 651 | | |
652 | 652 | | |
653 | 653 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
550 | 550 | | |
551 | 551 | | |
552 | 552 | | |
553 | | - | |
| 553 | + | |
554 | 554 | | |
555 | 555 | | |
556 | 556 | | |
| |||
0 commit comments