Modernize the Apache Airflow mixin #1540

schmikei · 2025-11-11T19:27:01Z

Modernizes the Apache Airflow mixin to use g.libsonnet/signals/common-lib.

As of submission most validation was mostly via the git diff.

schmikei · 2025-11-17T14:10:54Z

Working on getting some improved loadgen for this but think the Mixin is probably good to review for general guidance

Dasomeone

Generally layout looks fine, but hard to review given current sample-app and lack of screenshots. Are we missing any load-gen for the sample-app?

Happy to give this another look this week

Dasomeone · 2025-12-16T13:09:36Z

apache-airflow-mixin/config.libsonnet

-    dashboardTimezone: 'default',
-    dashboardRefresh: '1m',
+  local this = self,
+  filteringSelector: 'job="integrations/apache-airflow"',


We should remove the job label

Dasomeone · 2025-12-16T13:12:11Z

apache-airflow-mixin/config.libsonnet

-    alertsCriticalDAGScheduleDelayLevel: 60,  //s
-    alertsCriticalFailedDAGs: 0,
+  // additional params
+  dashboardPeriod: 'now-6h',


6 hour period may be a bit long

Dasomeone · 2025-12-16T13:16:27Z

apache-airflow-mixin/links.libsonnet

+  new(this): {
+               apacheAirflowOverview:
+                 link.link.new(this.config.dashboardNamePrefix + ' overview', '/d/' + this.grafana.dashboards['apache-airflow-overview.json'].uid)
+                 + link.link.options.withKeepTime(true),


Suggested change

+ link.link.options.withKeepTime(true),

+ link.link.options.withKeepTime(true),

+ link.dashboards.options.withIncludeVars(true)

Dasomeone · 2025-12-16T13:38:24Z

apache-airflow-mixin/dashboards.libsonnet

Struggled to review as the mixin didn't seem to use metrics provided by the sample app all that much

Dasomeone · 2025-12-16T13:41:14Z

apache-airflow-mixin/dashboards.libsonnet

+            g.dashboard.variable.query.new('dag_id')
+            + g.dashboard.variable.query.withDatasourceFromVariable(vars.datasources.prometheus)
+            + g.dashboard.variable.query.queryTypes.withLabelValues('dag_id', 'airflow_dagrun_duration_success_sum{job=~"$job", instance=~"$instance"}')
+            + g.dashboard.variable.query.generalOptions.withLabel('DAG ID')
+            + g.dashboard.variable.query.selectionOptions.withMulti(true)
+            + g.dashboard.variable.query.selectionOptions.withIncludeAll(true, '.+')
+            + g.dashboard.variable.query.refresh.onLoad()
+            + g.dashboard.variable.query.refresh.onTime(),
+
+            g.dashboard.variable.query.new('task_id')
+            + g.dashboard.variable.query.withDatasourceFromVariable(vars.datasources.prometheus)
+            + g.dashboard.variable.query.queryTypes.withLabelValues('task_id', 'airflow_task_finish_total{job=~"$job", instance=~"$instance"}')
+            + g.dashboard.variable.query.generalOptions.withLabel('Task')
+            + g.dashboard.variable.query.selectionOptions.withMulti(true)
+            + g.dashboard.variable.query.selectionOptions.withIncludeAll(true, '.+')
+            + g.dashboard.variable.query.refresh.onLoad()
+            + g.dashboard.variable.query.refresh.onTime(),
+
+            g.dashboard.variable.query.new('state')
+            + g.dashboard.variable.query.withDatasourceFromVariable(vars.datasources.prometheus)
+            + g.dashboard.variable.query.queryTypes.withLabelValues('state', 'airflow_task_finish_total{job=~"$job", instance=~"$instance"}')
+            + g.dashboard.variable.query.generalOptions.withLabel('State')
+            + g.dashboard.variable.query.selectionOptions.withMulti(true)
+            + g.dashboard.variable.query.selectionOptions.withIncludeAll(true, '.+')
+            + g.dashboard.variable.query.refresh.onLoad()
+            + g.dashboard.variable.query.refresh.onTime(),


These three template variables aren't showing any values in the sample-app. Missing load-gen perhaps?

Specifically, the metrics don't seem to be present

Dasomeone · 2025-12-16T13:59:43Z

apache-airflow-mixin/panels.libsonnet

+      dagFileParsingErrorsPanel:
+        commonlib.panels.generic.timeSeries.base.new('DAG file parsing errors', targets=[signals.overview.dagParsingErrors.asTarget()])
+        + g.panel.timeSeries.panelOptions.withDescription('The number of errors from trying to parse DAG files in an Apache Airflow system.')
+        + g.panel.timeSeries.standardOptions.withUnit('none'),


instead of none, use short where possible

Dasomeone · 2025-12-16T14:03:03Z

apache-airflow-mixin/signals/overview.libsonnet

+    groupLabels: this.groupLabels,
+    instanceLabels: this.instanceLabels,
+    enableLokiLogs: this.enableLokiLogs,
+    legendCustomTemplate: legendCustmoTemplate,


Typo on legendCustmoTemplate

aalhour

A few important notes, in addition to Emily's comments.

aalhour · 2025-12-16T16:28:40Z

apache-airflow-mixin/signals/overview.libsonnet

Can we use 'short' instead of 'none' for count metrics? Some examples:

dagParsingErrors

dagSuccessDurationCount

dagFailedDurationCount

taskFailures

taskSlaMisses

aalhour · 2025-12-16T16:28:56Z

apache-airflow-mixin/signals/overview.libsonnet

+        },
+      },
+
+      dagSuccessDurationSum: {


Is this used anywhere?

aalhour · 2025-12-16T16:29:09Z

apache-airflow-mixin/signals/overview.libsonnet

+        },
+      },
+
+      dagSuccessDurationCount: {


Is this used anywhere?

aalhour · 2025-12-16T16:29:18Z

apache-airflow-mixin/signals/overview.libsonnet

+        },
+      },
+
+      dagFailedDurationSum: {


Is this used anywhere?

aalhour · 2025-12-16T16:29:29Z

apache-airflow-mixin/signals/overview.libsonnet

+        },
+      },
+
+      dagFailedDurationCount: {


Is this used anywhere?

aalhour · 2025-12-16T16:32:54Z

apache-airflow-mixin/alerts.libsonnet

+            expr: |||
+              increase(airflow_dagrun_schedule_delay_sum{%(filteringSelector)s}[5m]) / clamp_min(increase(airflow_dagrun_schedule_delay_count{%(filteringSelector)s}[5m]),1) > %(alertsCriticalDAGScheduleDelayLevel)s
+            ||| % this.config,
+            'for': '1m',


Same here: this could cause a flapping alert, can we make it 5m?

aalhour · 2025-12-16T16:32:58Z

apache-airflow-mixin/alerts.libsonnet

+            expr: |||
+              increase(airflow_dagrun_duration_failed_count{%(filteringSelector)s}[5m]) > %(alertsCriticalFailedDAGs)s
+            ||| % this.config,
+            'for': '1m',


Also this: this could cause a flapping alert, can we make it 5m?

aalhour · 2025-12-16T16:33:30Z

apache-airflow-mixin/panels.libsonnet

+
+      slaIssuesPanel:
+        commonlib.panels.generic.timeSeries.base.new('SLA misses', targets=[
+          signals.overview.taskSlaMisses.asTarget() { interval: '2m' },


Why are we hardcoding the 2m interval here?

aalhour · 2025-12-16T16:33:39Z

apache-airflow-mixin/panels.libsonnet

+
+      taskFailuresPanel:
+        commonlib.panels.generic.timeSeries.base.new('Task failures', targets=[
+          signals.overview.taskFailures.asTarget() { interval: '2m' },


Also here: why are we hardcoding the 2m interval?

aalhour · 2025-12-16T16:33:44Z

apache-airflow-mixin/panels.libsonnet

+
+      dagSuccessDurationPanel:
+        commonlib.panels.generic.timeSeries.base.new('DAG success duration', targets=[
+          signals.overview.dagAvgSuccessDuration.asTarget() { interval: '2m' },


And here: why are we hardcoding the 2m interval?

schmikei added 2 commits November 11, 2025 14:26

modernize the airflow mixin

e06fbe9

newline at EOF

ead3805

schmikei mentioned this pull request Nov 11, 2025

chore: Modernize apache-airflow-mixin #1534

Closed

schmikei added the monitoring-mixins label Nov 14, 2025

schmikei marked this pull request as ready for review November 25, 2025 21:34

schmikei requested a review from a team as a code owner November 25, 2025 21:34

Dasomeone requested changes Dec 16, 2025

View reviewed changes

Dasomeone self-assigned this Dec 16, 2025

aalhour requested changes Dec 16, 2025

View reviewed changes

Dasomeone assigned mariaalons Dec 17, 2025

aalhour self-assigned this Dec 17, 2025

	+ link.link.options.withKeepTime(true),
	+ link.link.options.withKeepTime(true),
	+ link.dashboards.options.withIncludeVars(true)

Modernize the Apache Airflow mixin #1540

Are you sure you want to change the base?

Modernize the Apache Airflow mixin #1540

Uh oh!

Conversation

schmikei commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

schmikei commented Nov 17, 2025

Uh oh!

Dasomeone left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aalhour left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

schmikei commented Nov 11, 2025 •

edited

Loading