Allow prefixes in dask dashboard links#7408
Open
katrogan wants to merge 4 commits into
Open
Conversation
katrogan
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds optional support for running the Dask scheduler with a --dashboard-prefix so Bokeh dashboard tab/WebSocket links can route correctly through the dataproxy when a per-cluster identity is configured (similar in spirit to existing Spark behavior).
Changes:
- Added
Config.ClusterNameto gate whether a dashboard prefix is injected for the Dask scheduler. - Implemented
buildDashboardURLPrefix(...)and wired it into scheduler container args/env (POD_NAMESPACEvia downward API) to enable correct URL generation. - Exposed the new config via pflags and updated the generated config flags test.
Reviewed changes
Copilot reviewed 2 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
flyteplugins/go/tasks/plugins/k8s/dask/dask.go |
Builds a long-form dataproxy dashboard prefix and conditionally injects it into the scheduler container args/env. |
flyteplugins/go/tasks/plugins/k8s/dask/config.go |
Adds ClusterName configuration field controlling whether the dashboard prefix is enabled. |
flyteplugins/go/tasks/plugins/k8s/dask/config_flags.go |
Adds a generated pflag for clusterName. |
flyteplugins/go/tasks/plugins/k8s/dask/config_flags_test.go |
Updates generated flag tests to cover clusterName flag decoding. |
Files not reviewed (2)
- flyteplugins/go/tasks/plugins/k8s/dask/config_flags.go: Language not supported
- flyteplugins/go/tasks/plugins/k8s/dask/config_flags_test.go: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
254
to
+263
| primaryContainer.Args = []string{"dask-scheduler"} | ||
| if prefix := buildDashboardURLPrefix(clusterName, teMetadata); prefix != "" { | ||
| primaryContainer.Env = append(primaryContainer.Env, v1.EnvVar{ | ||
| Name: "POD_NAMESPACE", | ||
| ValueFrom: &v1.EnvVarSource{ | ||
| FieldRef: &v1.ObjectFieldSelector{FieldPath: "metadata.namespace"}, | ||
| }, | ||
| }) | ||
| primaryContainer.Args = append(primaryContainer.Args, "--dashboard-prefix", prefix) | ||
| } |
Comment on lines
+247
to
+255
| // Override args. When a cluster name is configured, point the scheduler's | ||
| // Bokeh dashboard at the LONG-form union dataproxy URL prefix so its internal | ||
| // tab/WebSocket links resolve through the proxy chain. The LONG form works | ||
| // regardless of whether the cluster is on the legacy CP-routed path or the | ||
| // zero-trust DP-direct path (the dataproxy on both sides parses it | ||
| // identically). Without this, Bokeh emits root-relative links like | ||
| // "/individual-task-stream" that 404 outside the dashboard mount point. | ||
| primaryContainer.Args = []string{"dask-scheduler"} | ||
| if prefix := buildDashboardURLPrefix(clusterName, teMetadata); prefix != "" { |
Comment on lines
+23
to
+26
| // that the leaseworker is running in. When set, the dask plugin starts the | ||
| // scheduler with `--dashboard-prefix`, baked from this name + the task's | ||
| // execution identity, so the Bokeh dashboard's internal links resolve through | ||
| // the union dataproxy reverse-proxy chain. Without it, dashboard tabs render |
Comment on lines
+71
to
+76
| func dashboardPrefixFromLogConfig(taskTemplate *core.TaskTemplate, teMetadata pluginsCore.TaskExecutionMetadata) string { | ||
| logPlugin, err := logs.InitializeLogPlugins(&GetConfig().Logs, taskTemplate) | ||
| if err != nil || logPlugin == nil { | ||
| return "" | ||
| } | ||
| out, err := logPlugin.GetTaskLogs(tasklog.Input{ |
Comment on lines
+67
to
+70
| // Returns "" when no dashboard template is configured, in which case the | ||
| // scheduler runs without `--dashboard-prefix` and Bokeh falls back to emitting | ||
| // root-relative URLs (the historical pre-fix behavior — page loads but tab | ||
| // navigation 404s outside the proxy mount point). |
…ly-through-dataplane
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Customize dash daskboard links with a custom cluster name field to conditionally prepend a dashboard prefix to dask dashboard links, like we do for spark
What changes were proposed in this pull request?
How was this patch tested?
Tested on a local cluster deployment.
Labels
Please add one or more of the following labels to categorize your PR:
This is important to improve the readability of release notes.
Setup process
Screenshots
Check all the applicable boxes
Related PRs
Stack
If you do use
git townto manage PR Stacks, the stack relevant to this PRwill show below. Otherwise, you can ignore this section.
Docs link