[Sandbox] KubeRay

[richardliaw]

Project summary

A toolkit and Kubernetes operator that simplifies the deployment, lifecycle management, and autoscaling of Ray applications on Kubernetes

Project description

KubeRay is a powerful, open source Kubernetes operator designed to run distributed AI, machine learning, and Python workloads powered by Ray seamlessly on cloud-native infrastructure. As distributed AI workloads (LLM training, batch inference, online serving, and fine-tuning) become a core cloud native construct, KubeRay bridges the gap between Python-first AI frameworks and Kubernetes-native infrastructure.

KubeRay manages the full lifecycle of Ray clusters through three core Custom Resource Definitions (CRDs):

1. RayCluster: Manages cluster creation, teardown, autoscaling, and fault tolerance across heterogeneous node pools (CPUs, GPUs, TPUs).
2. RayJob: Handles ephemeral job-based execution by dynamically provisioning a RayCluster, running a job to completion, and tearing down resources to optimize compute efficiency.
3. RayService: Enables zero-downtime upgrades, high availability, and multi-version traffic routing for Ray Serve applications.

In addition, KubeRay provides a kubectl-ray CLI plugin, an API server, and native integrations with cloud-native queuing and batch systems (such as Kueue, Volcano, and Apache YuniKorn) to ensure multi-tenant fair scheduling on Kubernetes.

Project vs Reference Architecture/Implementation

• This is a reusable open source project designed for broad adoption, not a reference architecture, reference implementation, or demonstration of patterns

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/ray-project/

Project repo URL in scope of application

https://github.com/ray-project/kuberay

Additional repos in scope of the application

N/A

Website URL

https://github.com/ray-project/kuberay

Roadmap

ray-project/kuberay#2501

Roadmap context

https://docs.google.com/document/d/1oz-onT3BTBruVGN0amgD0cfz8ELAiZatZHmjKcnx42Q/edit?tab=t.0#heading=h.8uojskj6olb9

Contributing guide

https://github.com/ray-project/kuberay/blob/master/CONTRIBUTING.md

Code of Conduct (CoC)

https://github.com/ray-project/kuberay/blob/master/CODE_OF_CONDUCT.md

Adopters

N/A

Maintainers file

https://github.com/ray-project/kuberay/blob/master/docs/community/PEOPLE.md

Security policy file

https://github.com/ray-project/kuberay/blob/master/SECURITY.md

Standard or specification?

N/A

Business product or service to project separation

KubeRay was originally co-developed and maintained by open-source contributors across multiple organizations (including Anyscale, Google, Microsoft, ByteDance, and AWS). While commercial vendors offer managed Ray or AI platform services that utilize KubeRay upstream, KubeRay operates strictly as a vendor-neutral, community-driven Kubernetes operator. Donating KubeRay to the CNCF will ensure long-term governance neutrality, encouraging broader cross-cloud and enterprise contribution without single-vendor control.

Why CNCF?

1. Neutral Infrastructure Governance: As Kubernetes becomes the defacto platform for AI/ML workloads, KubeRay serves as a critical control plane link. CNCF stewardship guarantees vendor-neutral governance and standard IP management (Apache 2.0).
2. Deep Ecosystem Collaboration: Bringing KubeRay into the CNCF facilitates tighter co-development with related cloud-native scheduling, observability, and storage projects (e.g., Kueue, Prometheus, Jaeger, Argo Workflows).
3. Enterprise Confidence: End-users deploying mission-critical AI/ML pipelines at scale benefit from CNCF's operational maturity, security audits, and community standards.

Benefit to the landscape

KubeRay fills a key capability gap in the CNCF Machine Learning & AI landscape by providing a standardized, production-grade Kubernetes-native pattern for managing Ray.
Unlike general-purpose job schedulers or batch operators, KubeRay is explicitly tailored to handle the nuances of Ray’s stateful head/worker architecture, actor-based dynamic placement, fine-grained GPU/TPU resource allocation, and zero-downtime model serving. It provides the cloud-native substrate needed to build AI platforms on top of vanilla Kubernetes.

Cloud native 'fit'

KubeRay is built natively on standard Kubernetes building blocks:

• Implemented as a custom controller using controller-runtime and CustomResourceDefinitions (RayCluster, RayJob, RayService).
• Fits directly into cloud-native GitOps workflows (ArgoCD, Flux) and Helm-based deployments.
• Adheres to Kubernetes operator best practices regarding status subresources, pod templates, autoscaling integration (Cluster Autoscaler / KEDA), and graceful termination.

Cloud native 'integration'

KubeRay integrates extensively with the CNCF ecosystem:

• Batch Scheduling & Queuing: Integrates with CNCF-hosted or affiliated batch schedulers like Kueue, Volcano, and Apache YuniKorn for gang scheduling, multi-tenant quota management, and fair sharing.
• Observability: Exposes Native Prometheus metrics for Ray cluster states, pod status, and resource usage, along with Grafana dashboard templates.
• Ingress & Networking: Leverages standard Kubernetes Ingress controllers (NGINX, Envoy Gateway) and Service Meshes (Istio) for exposing Ray Dashboard and Ray Serve endpoints.
• Tooling: Implements the kubectl plugin framework via kubectl-ray for direct developer interactions.

Cloud native overlap

Overlaps partially with Cluster API itself KubeAid CLI is a higher-level orchestrator built on top of CAPI, not a replacement for it; it doesn't compete with CAPI so much as remove the need to hand-write CAPI manifests and glue tooling directly.

Similar projects

Cluster API Operator, kOps, Kubespray, Rancher/RKE2 provisioning tooling, k0rdent, Sidero/Talos (bare-metal-focused). KubeAid CLI's differentiation is the GitOps-first design and the single-binary, no-long-lived-management-plane pivot model for cloud providers.

Landscape

https://l.cncf.io/?item=ai-native-infra--orchestration-and-scheduling--kuberay

Insights

https://insights.linuxfoundation.org/project/kuberay

Trademark and accounts

• If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

IP policy

• If the project is accepted, I agree the project will follow the CNCF IP Policy

Will the project require a license exception?

N/A project uses Apache 2.0.

Does the project utilize any dependencies with licenses not on the allowlist or an approved exception?

N/A project uses Apache 2.0.

Project "Domain Technical Review"

N/A

Application contact email(s)

rliaw@anyscale.com, rkn@anyscale.com, rueian@anyscale.com, kaihsun2019@gmail.com, andrewsy@google.com, eoakes@anyscale.com

Contributing or sponsoring entity signatory information

Name	Address	Type (e.g., Delaware corporation)	Signatory name and title	Email address
Anyscale Inc	600 Harrison St, San Francisco CA 94107	Delaware corporation	Richard Liaw, Product Manager	rliaw@anyscale.com
Anyscale Inc	600 Harrison St, San Francisco CA 94107	Delaware corporation	Robert Nishihara, Co-founder	rkn@anyscale.com

CNCF contacts

• Chris Aniszczyk

Additional information

• Vote: Donating KubeRay to CNCF ray-project/ray#65900

[github-actions]

🤖 Automated Sandbox Application Pre-Review

Note

This is an automated first-pass review generated by GitHub Copilot against the
Sandbox acceptance criteria
and the CNCF TOC Principles.
It is advisory only — the CNCF TOC makes all final decisions.

❌ Clear violations detected

• Invalid MAINTAINERS file (missing Company/Organization)
  • Evidence: Fetched https://github.com/ray-project/kuberay/blob/master/docs/community/PEOPLE.md lists Triagers, Reviewers, Committers, and TSC each in tables with only 'Name' and 'GitHub ID' columns — no Company/Organization column anywhere in the file.
  • Guideline: README.md 'MAINTAINERS File' — 'File must contain table with: Name, ID, Company/Organization columns'; application.yml Maintainers file field — 'File must contain a table with columns for: Name, GitHub ID, and Company/Organization'.
• Subproject separation not documented
  • Evidence: Application lists 'Project repo URL in scope of application: https://github.com/ray-project/kuberay' (under the ray-project org) and states KubeRay 'was originally co-developed and maintained by open-source contributors across multiple organizations,' but the 'Additional information' section only links a generic vote issue (Donating KubeRay to CNCF ray-project/ray#65900, in the ray repo, not documented as a maintainer-consensus vote to split) and provides no public issue in the parent ray-project org showing maintainer consensus to separate KubeRay into its own CNCF project.
  • Guideline: README.md 'Subproject Separation' — 'If your project is under another project's GitHub/GitLab organization, you must provide a link to a public issue in the parent project's repository showing maintainer consensus to split into separate CNCF project. Applies to ANY parent project'; application.yml lines 111-122 (formal vote requirement) and line 418 (pre-submission checklist).

The Postponed label has been applied automatically. Please review the
Postponed guidance for how to
address these items and return for review. If you believe this assessment
is incorrect, please leave a comment — CNCF staff and the TOC will take a look.

Assessment summary

• ✅ License: Apache-2.0 confirmed via GitHub API for ray-project/kuberay.
• ✅ Project age: repo created 2020-10-29, pushed as recently as today — well over 6 months, actively developed.
• ❌ MAINTAINERS file: linked PEOPLE.md exists but only has Name/GitHub ID columns, missing required Company/Organization affiliation.
• ✅ Project vs reference architecture: KubeRay is a genuine reusable Kubernetes operator/CRD-based project, not a reference architecture.
• ❌ Subproject separation: repo sits under the ray-project org (parent project); no public issue link showing parent maintainer consensus to split was provided as required.
• ⚠️ Docs: CoC, Contributing guide, Security policy links provided and appear to resolve; Adopters is 'N/A'; Roadmap points to an open GitHub issue plus an external Google Doc.
• ⚠️ Alignment with TOC Principles: application otherwise argues project-centric neutrality and ecosystem integration reasonably well, but internal inconsistency (unrelated 'Similar projects'/'overlap' text referencing a different tool) undermines confidence in the application's care/accuracy.

⚠️ Items for TOC verification (non-blocking)

• The linked 'Vote' issue (Donating KubeRay to CNCF ray-project/ray#65900) could not be fetched/rendered to confirm it documents parent-project maintainer consensus for separation — TOC should verify its content directly.
• The 'Cloud native overlap' and 'Similar projects' sections reference an unrelated project ('KubeAid CLI', 'Cluster API', bare-metal provisioning tools) that has nothing to do with KubeRay/Ray — this looks like copy-pasted boilerplate from a different application and raises quality/accuracy concerns about the rest of the submission.
• Adopters field is 'N/A' despite KubeRay being a widely-used project; TOC may want production adopter evidence.
• Roadmap context link is a Google Doc (non-public verification not attempted) rather than a GitHub-native roadmap artifact.

[richardliaw]

Addressing the above comments -

• MAINTAINERS file now has Company/Organization for all rows
• Donating KubeRay to CNCF ray-project/ray#65900 is the vote by the Ray TSC in support of the separation.

[angellk]

Thank you @richardliaw - moving forward to be reviewed!

[cncf-automation-bot]

The TOC recommends that the project proceed to a vote.
Please note that the CNCF TOC requires new projects to avoid using the word "kube" in their names, as the term has become overused and makes projects less distinct and noticeable.

[cncf-automation-bot]

/vote

[git-vote]

Vote created

@cncf-automation-bot has called for a vote on [Sandbox] KubeRay (#525).

The members of the following teams have binding votes:

Team
@cncf/cncf-toc-voters

Non-binding votes are also appreciated as a sign of support!

How to vote

You can cast your vote by reacting to this comment. The following reactions are supported:

In favor	Against	Abstain
👍	👎	👀

Please note that voting for multiple options is not allowed and those votes won't be counted.

The vote will be open for 2months 30days 2h 52m 48s. It will pass if at least 66% of the users with binding votes vote In favor 👍. Once it's closed, results will be published here as a new comment.