chore: Modernize the Apache CouchDB mixin #1522

schmikei · 2025-10-29T21:13:24Z

This modernizes the CouchDB Mixin to use newer libraries.

Overview

Nodes:

Logs:

Dasomeone

Going to head out in a second so I'll publish my comments so-far! Going to pick it back up in the morning
@aalhour would also appreciate a second pair of eyes on this one :)

apache-couchdb-mixin/config.libsonnet

apache-couchdb-mixin/mixin.libsonnet

apache-couchdb-mixin/signals/overview.libsonnet

…ouchdb-modernization

…nnet-libs into chore/couchdb-modernization

aalhour

I caught some issues, thanks for pushing this.

aalhour · 2025-11-18T13:52:06Z

apache-couchdb-mixin/alerts.libsonnet

            expr: |||
-              sum by(job, instance) (increase(couchdb_httpd_status_codes{code=~"4.*"}[5m])) > %(alertsWarning4xxResponseCodes5m)s
-            ||| % $._config,
+              sum by(job, instance) (increase(couchdb_httpd_status_codes{code=~"4.."}[5m])) > %(alertsWarning4xxResponseCodes5m)s


Isn't the 4.* regex more general than the 4..?

https://github.com/grafana/jsonnet-libs/actions/runs/19470908585/job/55717683328?pr=1522

Unfortunately its considered a messy selector. For some reason the lint passes locally for myself but within Ci it fails

The reason for this failing is due to the Pint linter rules that takes a bit more setup to install locally. I've no strong feelings about this either way. On one hand it's good to be specific to the three digit setup, on the other it could be an issue if mixed with text, e.g. 404NotFound as a label.

I doubt we will run into the latter, so for now I think 4.. is perfectly fine

aalhour · 2025-11-18T13:52:35Z

apache-couchdb-mixin/mixin.libsonnet

+  cluster+: {
+    allValue: '.*',
+  },
+  couchb_cluster+: {


couchb --> couchdb, missing d character.

Ah yep thank you for the catch!

aalhour · 2025-11-18T13:53:43Z

apache-couchdb-mixin/alerts.libsonnet

          {
            alert: 'CouchDBUnhealthyCluster',
            expr: |||
              min by(job, couchdb_cluster) (couchdb_couch_replicator_cluster_is_stable) < %(alertsCriticalClusterIsUnstable5m)s


Should the alerts use the filteringSelector from the config? Just wondering.

expr: ||| min by(job, couchdb_cluster) (couchdb_couch_replicator_cluster_is_stable{%(filteringSelector)s}) < %(alertsCriticalClusterIsUnstable5m)s ||| % this.config,

Went ahead and implemented these!

aalhour · 2025-11-18T13:55:23Z

apache-couchdb-mixin/config.libsonnet

+  alertsCriticalClusterIsUnstable5m: 1,  //1 is stable
+  alertsWarning4xxResponseCodes5m: 5,
+  alertsCritical5xxResponseCodes5m: 0,
+  alertsWarningRequestLatency5m: 500,  //ms


This is in milliseconds, which means that the alert threshold is broken.

aalhour · 2025-11-18T13:57:29Z

apache-couchdb-mixin/alerts.libsonnet

          {
            alert: 'CouchDBHighRequestLatency',
            expr: |||
              sum by(job, instance) (couchdb_request_time_seconds_sum / couchdb_request_time_seconds_count) > %(alertsCriticalRequestLatency5m)s


I know that you didn't change this but I think there is a bug in the calculation, sum / count is in seconds but the config declares the threshold in milliseconds (500), we should multiply the value before the compare operator by 1000. I think this is the second time I catch this bug, which is not on you but it's there in the original implementations.

expr: ||| sum by(job, instance) (couchdb_request_time_seconds_sum / couchdb_request_time_seconds_count) * 1000 > %(alertsWarningRequestLatency5m)s ||| % this.config,

Alternatively, we can change the alert threshold in the config. I am fine with either solution.

Ah yep didn't really audit the content of the queries too much. Will go ahead and take a stab at a fix

apache-couchdb-mixin/alerts.libsonnet

aalhour · 2025-11-18T14:00:47Z

apache-couchdb-mixin/signals/overview.libsonnet

+      unit: 'requests',
+      sources: {
+        prometheus: {
+          expr: 'sum by(' + groupLabelAggTerm + ') (couchdb_httpd_status_codes{%(queriesSelector)s, code=~"[45].*"})',


Why are we not using increase(...) like the goodResponseStatuses signal above it? Also neither interval nor offset. At least when I read them, I get the feeling that they are complementary but their expressions are not really consistent.

aalhour · 2025-11-18T14:01:49Z

apache-couchdb-mixin/panels.libsonnet

+          + g.query.prometheus.withInstant(true)
+          + g.query.prometheus.withFormat('timeseries'),
+        ])
+        + g.panel.gauge.queryOptions.withDatasource('prometheus', '${' + this.grafana.variables.datasources.prometheus.name + '}')


Do we really want to specify the datasource here? I think it should be derived from the board config and selectors.

aalhour · 2025-11-18T14:03:27Z

apache-couchdb-mixin/signals/nodes.libsonnet

+          expr: 'couchdb_httpd_view_reads_total{%(queriesSelector)s}',
+        },
+      },
+    },


I see that these signals don't specify a rate(...)[$__rate_interval] like other counterparts in overview.libsonnet, does the function also break the panels for you? At least for me with the databricks data, it can be a gauge type and it might break it. But here i see that these are counters 🤔

These are counters so the rate gets auto-added from my experience!

Dasomeone

I gave this another pass while checking with the sample-app and I realised the node dashboard is pretty cluttered and the panels hard to read.
I've left a couple comments but I think it's a more overarching issue for the node views so happy to set up a call to discuss between us :)

apache-couchdb-mixin/README.md

apache-couchdb-mixin/panels.libsonnet

Dasomeone · 2025-11-19T12:49:16Z

apache-couchdb-mixin/alerts.libsonnet

            expr: |||
-              sum by(job, instance) (increase(couchdb_httpd_status_codes{code=~"4.*"}[5m])) > %(alertsWarning4xxResponseCodes5m)s
-            ||| % $._config,
+              sum by(job, instance) (increase(couchdb_httpd_status_codes{code=~"4.."}[5m])) > %(alertsWarning4xxResponseCodes5m)s


The reason for this failing is due to the Pint linter rules that takes a bit more setup to install locally. I've no strong feelings about this either way. On one hand it's good to be specific to the three digit setup, on the other it could be an issue if mixed with text, e.g. 404NotFound as a label.

I doubt we will run into the latter, so for now I think 4.. is perfectly fine

apache-couchdb-mixin/panels.libsonnet

Dasomeone

Couple more minor things, but it's almost there from my perspective!

apache-couchdb-mixin/main.libsonnet

apache-couchdb-mixin/panels.libsonnet

apache-couchdb-mixin/rows.libsonnet

apache-couchdb-mixin/links.libsonnet

…; fix log link in dashboards

…ouchdb-modernization

Dasomeone · 2025-12-16T10:19:16Z

apache-couchdb-mixin/alerts.libsonnet

-              sum by(job, instance) (increase(couchdb_httpd_status_codes{%(filteringSelector)s,code=~"4.."}[5m])) > %(alertsWarning4xxResponseCodes5m)s
-            ||| % this.config,
+              sum by(job, instance) (increase(couchdb_httpd_status_codes{%(filteringSelector)s}[5m])) > %(alertsWarning4xxResponseCodes5m)s
+            ||| % (this.config { filteringSelector: if this.config.filteringSelector != '' then this.config.filteringSelector + ',code=~"4.."' else 'code=~"4.."' }),


It's only here twice so I don't feel strongly about it, but if we need to do similar in other mixins then let's extract that to a variable

Dasomeone

Looks good to me, thanks for all the iteration on this one Keith!

aalhour

LGTM, thanks a lot!

schmikei · 2025-12-16T21:30:13Z

apache-couchdb-mixin/dashboards_out/couchdb-logs.json

+               "label": "Level",
+               "multi": true,
+               "name": "level",
+               "query": "label_values({,job=~\"$job\",couchdb_cluster=~\"$couchdb_cluster\",cluster=~\"$cluster\"}, level)",


@Dasomeone @aalhour I'm noticing that with the empty filteringSelector we get this invalid label. Is this something we're okay with expecting that people are going to probably overwrite the filteringSelector? Otherwise just setting it to something like job=~".+" could also be appropriate maybe?

This only affects logs dashboards as far as I can tell.

schmikei added 2 commits October 29, 2025 17:08

modernize the apache couchdb mixin

8cad84a

fix links and try to fix lint

c3f6bb4

schmikei marked this pull request as ready for review October 29, 2025 21:19

schmikei requested a review from a team as a code owner October 29, 2025 21:19

Merge branch 'master' into chore/couchdb-modernization

a969714

Dasomeone reviewed Nov 11, 2025

View reviewed changes

schmikei added 5 commits November 13, 2025 10:05

Merge branch 'master' of github.com:grafana/jsonnet-libs into chore/c…

2c6abca

…ouchdb-modernization

pr feedback

d9e918d

Merge branch 'chore/couchdb-modernization' of github.com:schmikei/jso…

17e7fa2

…nnet-libs into chore/couchdb-modernization

make fmt

7006703

fix units on histogram

092776c

aalhour requested changes Nov 18, 2025

View reviewed changes

schmikei added 5 commits November 18, 2025 10:08

fix a couple of issues caught in PR review

9186490

fix lint with selector

f492f27

fix lint with selector

467d456

fix some issues due to recent commits; interval/legends

e6a5654

make fmt

7f424aa

Dasomeone reviewed Nov 19, 2025

View reviewed changes

address PR feedback minus the description implementation

58ca718

Dasomeone assigned Dasomeone and aalhour Dec 4, 2025

Dasomeone added the monitoring-mixins label Dec 4, 2025

Dasomeone requested changes Dec 15, 2025

View reviewed changes

schmikei added 5 commits December 15, 2025 13:09

use couchdb_database_reads_total as varMetric; filter out zero values…

cadc1cd

…; fix log link in dashboards

make fmt

a81eabc

Merge branch 'master' of github.com:grafana/jsonnet-libs into chore/c…

d0ee50a

…ouchdb-modernization

withIncludeVars(true)

0da8fe8

remove public mixin setting of the filteringSelector

bb2bd36

Dasomeone reviewed Dec 16, 2025

View reviewed changes

Dasomeone approved these changes Dec 16, 2025

View reviewed changes

aalhour approved these changes Dec 16, 2025

View reviewed changes

schmikei commented Dec 16, 2025

View reviewed changes

chore: Modernize the Apache CouchDB mixin #1522

Are you sure you want to change the base?

chore: Modernize the Apache CouchDB mixin #1522

Uh oh!

Conversation

schmikei commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dasomeone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aalhour left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dasomeone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Dasomeone left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dasomeone left a comment

Choose a reason for hiding this comment

Uh oh!

aalhour left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

schmikei commented Oct 29, 2025 •

edited

Loading