OAuth client registration: CIMD is enabled by default. DCR is deprecated by MCP 2026-07-28 and remains a compatibility fallback. See registration, consent, egress and TLS configuration.
An MCP (Model Context Protocol) server for Prometheus and Mimir, written in Go. Deployed in-cluster at Giant Swarm to give AI assistants authenticated, multi-tenant access to metrics infrastructure.
MCP Prometheus exposes 18 read-only MCP tools that wrap the Prometheus HTTP API: instant and range PromQL queries, metric/label/series discovery, target and runtime information, TSDB stats, alerting rules, and exemplars.
When deployed with OAuth enabled it acts as a full OAuth 2.1 Authorization Server (backed by Dex/OIDC or Google), so MCP clients authenticate with the server before any tool call. The server then resolves the authenticated user's Mimir tenant IDs and enforces them on every query.
- Architecture
- Installation
- Configuration reference
- Transport modes
- OAuth 2.1 authentication
- Multi-tenancy
- Available tools
- Kubernetes deployment (Helm)
- Development
MCP Client (Claude, muster, …)
│ OAuth 2.1 + MCP over HTTP
▼
┌──────────────────────────────────┐
│ mcp-prometheus │
│ │
│ ┌────────────┐ ┌─────────────┐ │
│ │ OAuth 2.1 │ │ MCP Tools │ │
│ │ server │ │ (PromQL, │ │
│ │ (mcp-oauth)│ │ labels, …)│ │
│ └─────┬──────┘ └──────┬──────┘ │
│ │ │ │
│ ┌─────▼──────┐ ┌──────▼──────┐ │
│ │ Dex OIDC │ │ Tenancy │ │
│ │ provider │ │ resolver │ │
│ └────────────┘ └──────┬──────┘ │
└─────────────────────────┼────────┘
│
┌───────────────┴──────────────┐
│ │
┌──────▼───────┐ ┌───────▼──────┐
│ GrafanaOrg │ │ Prometheus │
│ CRDs (k8s) │ │ / Mimir │
└──────────────┘ └──────────────┘
The server listens on two ports:
:8080— MCP + OAuth endpoints (served to clients):9091— observability:/metrics,/healthz,/readyz(internal only)
Download the latest release from the releases page.
git clone https://github.com/giantswarm/mcp-prometheus.git
cd mcp-prometheus
go build -o mcp-prometheus ./...See Kubernetes deployment (Helm).
All configuration is via environment variables.
| Variable | Default | Description |
|---|---|---|
PROMETHEUS_URL |
— | Prometheus/Mimir base URL |
MCP_ALLOW_BACKEND_URL_OVERRIDES |
true |
Allow per-call backend destinations; alternate destinations receive no configured credentials or default tenant |
PROMETHEUS_USERNAME |
— | Basic auth username |
PROMETHEUS_PASSWORD |
— | Basic auth password |
PROMETHEUS_TOKEN |
— | Bearer token |
PROMETHEUS_ORGID |
— | Default Mimir org/tenant ID |
PROMETHEUS_TLS_SKIP_VERIFY |
false |
Skip TLS verification (dev only) |
PROMETHEUS_TLS_CA_CERT |
— | Path to PEM CA certificate |
ALERTMANAGER_URL |
— | Alertmanager base URL for get_alertmanager_alerts (Mimir: http://mimir-gateway.mimir/alertmanager); uses the authentication, TLS and org ID settings above |
| Variable | Default | Description |
|---|---|---|
MCP_OAUTH_ISSUER |
required | Public base URL of this server (e.g. https://mcp.example.com) |
MCP_OAUTH_ENCRYPTION_KEY |
— | 32-byte AES-256-GCM key for token encryption, base64 (openssl rand -base64 32) or hex (openssl rand -hex 32) |
MCP_OAUTH_PROVIDER |
dex |
Identity provider: dex or google (see the provider tables below) |
OAUTH_REDIRECT_URL |
required | Callback URL registered at the provider (e.g. https://mcp.example.com/oauth/callback). The legacy DEX_REDIRECT_URL is still honoured when this is unset |
MCP_OAUTH_ALLOW_PUBLIC_REGISTRATION |
false |
Allow unauthenticated dynamic client registration (dev/MCP Inspector only) |
MCP_OAUTH_ALLOW_PRIVATE_IP_JWKS_HOSTS |
— | Comma-separated hosts, other than the issuer's own, whose JWKS may resolve to a private address (see below) |
MCP_OAUTH_ALLOW_PRIVATE_IP_JWKS |
false |
Deprecated, no effect (see below); logs a warning when true |
OAUTH_TRUSTED_AUDIENCES |
— | Comma-separated client IDs trusted for SSO token forwarding |
OAUTH_STORAGE |
memory |
Token storage: memory or valkey |
VALKEY_URL |
— | Valkey/Redis address (required when OAUTH_STORAGE=valkey) |
VALKEY_PASSWORD |
— | Valkey auth password |
VALKEY_TLS_ENABLED |
false |
Enable TLS for Valkey |
VALKEY_KEY_PREFIX |
mcp: |
Key namespace prefix |
| Variable | Default | Description |
|---|---|---|
DEX_ISSUER_URL |
required | Dex issuer URL (e.g. https://dex.mc.example.io) |
DEX_CLIENT_ID |
required | OAuth client ID registered in Dex |
DEX_CLIENT_SECRET |
required | OAuth client secret |
DEX_REDIRECT_URL |
— | Legacy alias of OAUTH_REDIRECT_URL; used when the latter is unset |
DEX_CA_FILE |
— | PEM CA file verifying TLS for Dex and JWKS endpoints (private-CA installations). Added on top of the system trust store |
| Variable | Default | Description |
|---|---|---|
GOOGLE_CLIENT_ID |
required | OAuth client ID of the Google Cloud OAuth client (….apps.googleusercontent.com) |
GOOGLE_CLIENT_SECRET |
required | OAuth client secret of that client |
Google's discovery, token and JWKS endpoints are public, so DEX_CA_FILE does not
apply (it is ignored with a warning).
Google ID tokens carry no groups claim: pair this provider with
tenancy mode none or a static all-users tenant list.
| Variable | Default | Description |
|---|---|---|
TENANCY_STATIC_GROUP_MAP |
— | JSON map of Dex group → list of Mimir tenant IDs (static mode) |
Tenancy mode is a flag: --tenancy-mode grafana-organization|static|none (see Multi-tenancy).
| Variable | Default | Description |
|---|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT |
— | OTLP HTTP endpoint for tracing (no-op if unset) |
OTEL_SERVICE_NAME |
mcp-prometheus |
Service name in traces |
Start the server with serve --transport <mode>:
| Mode | Flag | Use case |
|---|---|---|
stdio |
--transport stdio |
Local desktop clients (Claude Desktop, MCP Inspector) |
sse |
--transport sse |
Legacy SSE clients |
streamable-http |
--transport streamable-http |
Production in-cluster (default) |
OAuth requires sse or streamable-http.
# Local stdio (no OAuth)
./mcp-prometheus serve
# In-cluster HTTP with OAuth
./mcp-prometheus serve --transport streamable-http --http-addr :8080 --enable-oauthMCP Prometheus implements OAuth 2.1 (RFC 9700) using mcp-oauth with the platform's
identity provider upstream: Dex (default) or Google, selected with MCP_OAUTH_PROVIDER.
The flow, endpoints and token handling are identical for both; only the upstream login and the
provider-specific environment variables differ.
| Path | Method | Purpose |
|---|---|---|
/.well-known/oauth-authorization-server |
GET | OAuth server metadata (RFC 8414) |
/.well-known/protected-resources |
GET | Protected resource metadata |
/oauth/authorize |
GET | Authorization endpoint (redirects to the identity provider) |
/oauth/callback |
GET | Identity provider callback (OAUTH_REDIRECT_URL) |
/oauth/token |
POST | Token exchange |
/oauth/register |
POST | Dynamic client registration (RFC 7591) |
/oauth/revoke |
POST | Token revocation |
MCP Client mcp-prometheus Dex OIDC
│ │ │
│ 1. GET /.well-known/… │ │
│──────────────────────────>│ │
│ server metadata │ │
│<──────────────────────────│ │
│ │ │
│ 2. POST /oauth/register │ │
│──────────────────────────>│ │
│ client_id + secret │ │
│<──────────────────────────│ │
│ │ │
│ 3. GET /oauth/authorize │ │
│──────────────────────────>│ │
│ │ 4. redirect to Dex │
│ │───────────────────────>│
│<──────────────────────────────────────────────────│
│ browser: Dex login UI │ │
│ (user authenticates) │ │
│ │ │
│ │ 5. callback + code │
│ │<───────────────────────│
│ │ │
│ 6. POST /oauth/token │ │
│──────────────────────────>│ │
│ access_token + refresh │ │
│<──────────────────────────│ │
│ │ │
│ 7. MCP tool call │ │
│ Authorization: Bearer … │ │
│──────────────────────────>│ │
│ │ 8. validate token │
│ │ resolve tenants │
│ │ forward to Mimir │
The access token is a short-lived JWT signed by mcp-prometheus and validated on every request. Refresh token rotation is enabled — every refresh issues a new refresh token.
memory(default): in-process, lost on restart. Suitable for single-replica deployments and development.valkey: production-grade Redis/Valkey backend. Required for multi-replica deployments.
The Dex client (OIDC discovery, token and userinfo calls) reaches a Dex whose hostname
resolves to a private IP (RFC-1918 range) without any setting, and it follows HTTP_PROXY,
HTTPS_PROXY and NO_PROXY. On a proxied installation, list the Dex host in NO_PROXY when
Dex must be reached directly.
The JWKS fetch that validates forwarded ID tokens (OAUTH_TRUSTED_AUDIENCES, e.g. from muster)
keeps its SSRF protection, except that the configured Dex issuer host may resolve to a private
address. The JWKS URL comes from the configured Dex issuer, never from the token. TLS
verification is still enforced.
When the issuer's discovery document puts jwks_uri on another host that resolves to an
RFC 1918 or RFC 4193 address, list that host in MCP_OAUTH_ALLOW_PRIVATE_IP_JWKS_HOSTS
(comma-separated; Helm app.oauth.allowPrivateIPJWKSHosts). Entries are exact hostnames,
without port. The issuer host needs no entry. Loopback and link-local addresses stay refused
for a listed host.
MCP_OAUTH_ALLOW_PRIVATE_IP_JWKS and Helm app.oauth.allowPrivateIPJWKS are deprecated and
have no effect; setting them to true logs a warning. Use
MCP_OAUTH_ALLOW_PRIVATE_IP_JWKS_HOSTS for a JWKS host other than the issuer's.
See migration instructions for obsolete private-URL settings.
When Dex is served with a certificate from a private/internal CA (e.g. private management
clusters), TLS verification fails with x509: certificate signed by unknown authority.
Point DEX_CA_FILE at a PEM CA file to add that CA on top of the system trust store.
The pool verifies the Dex provider connection (OIDC discovery, code flow, userinfo)
and the forwarded-ID-token JWKS endpoint.
DEX_CA_FILE=/etc/ssl/certs/dex-ca/ca.crtIn Helm, reference a Secret holding the CA certificate:
app:
oauth:
dexCASecret:
name: mcp-prometheus-dex-ca
key: ca.crtWhen users connect through an upstream MCP aggregator (e.g. muster) that has already authenticated them, the aggregator can forward the user's ID token from the platform IdP directly instead of starting a new OAuth flow.
Configure OAUTH_TRUSTED_AUDIENCES with a comma-separated list of the aggregator's OAuth client IDs:
OAUTH_TRUSTED_AUDIENCES=muster-client,my-aggregatormcp-prometheus will:
- Detect that the incoming token's audience matches a trusted client ID
- Verify the token signature against the provider's JWKS endpoint (Dex or Google)
- Accept the token and proceed with tenant resolution
Tokens must still originate from the configured provider's issuer. With the Google provider, list the platform's Google OAuth client ID (the one muster logs users in with) here.
When OAuth is enabled, every tool call is scoped to the authenticated user's allowed Mimir tenant IDs.
The user can pass an explicit org_id parameter; the server validates it against their allowed tenants.
If no org_id is given, the query tools inject all allowed tenants as a Mimir pipe-separated
multi-tenant selector, which the querier federates.
Mimir's ruler and Alertmanager serve one tenant per request, so get_alerts, get_rules and
get_alertmanager_alerts address only the tenants of type alerting (a tenant of type data only
has neither rules nor an Alertmanager and never joins). get_alertmanager_alerts reads every alerting
tenant and merges the alerts, each marked with its tenant; org_id, pipe-separated for several,
narrows the read. get_alerts and get_rules read a single alerting tenant by itself and, with
several, answer an error naming them until org_id picks one. In static mode every configured tenant
counts as alerting.
Three resolution modes are available, selected with --tenancy-mode (or app.tenancy.mode in Helm):
grafana-organization (default), static, and none.
--tenancy-mode grafana-organization
Reads GrafanaOrganization custom resources from the Kubernetes API.
Each CR declares which Dex groups have access (spec.rbac) and which Mimir tenants map to it
(spec.tenants), each with the types it carries: data (the default) and alerting.
apiVersion: observability.giantswarm.io/v1alpha2
kind: GrafanaOrganization
metadata:
name: team-platform
spec:
rbac:
admins:
- github-org:team-platform # Dex group from LDAP/GitHub
tenants:
- name: prod_eu_west
types: [data, alerting]
- name: prod_us_east # data onlyWhen a user authenticates:
- The server reads the
groupsclaim from their Dex token - It looks up all
GrafanaOrganizationCRs where any group inspec.rbacmatches - It collects all tenants from
spec.tenantsacross matching CRs, with theirtypes - Results are cached per group-set for 60 seconds
The Helm chart creates a ClusterRole + ClusterRoleBinding granting get, list, watch on grafanaorganizations.observability.giantswarm.io when this mode is active.
--tenancy-mode static
No Kubernetes API access required. Tenants are configured statically.
# All authenticated users get these tenant IDs
--static-tenants=prod-eu,prod-usHelm:
app:
tenancy:
mode: static
static:
tenants: "prod-eu,prod-us"When TENANCY_STATIC_GROUP_MAP is set (or app.tenancy.static.groups in Helm), tenant IDs are resolved per group:
TENANCY_STATIC_GROUP_MAP='{"team-ops":["prod-eu","prod-us"],"team-dev":["staging"]}'Helm:
app:
tenancy:
mode: static
static:
groups:
team-ops:
- prod-eu
- prod-us
team-dev:
- stagingThe user's allowed tenants are the union of all tenants from their Dex groups.
--tenancy-mode none
For a plain, single-tenant Prometheus (no Mimir, no X-Scope-OrgID). OAuth still
authenticates every caller — unauthenticated requests are rejected, forwarded tokens are
validated and the identity is available for auditing — but no tenant is derived from the
identity and no tenant header is injected. An explicit org_id tool parameter or
PROMETHEUS_ORGID passes through verbatim. The chart creates no ClusterRole in this mode.
This is the mode to use with the Google provider (Google tokens have no groups claim)
and whenever mcp-prometheus sits behind muster in front of a single Prometheus.
Helm:
app:
tenancy:
mode: noneAll tools accept optional prometheus_url and org_id parameters for per-call overrides.
Backend URL overrides default to enabled; set MCP_ALLOW_BACKEND_URL_OVERRIDES=false to restrict calls to configured endpoints. See backend destinations for credential and tenant handling.
Every description and input schema, exactly as a model reads them, is in docs/tools.md, generated by a test.
| Tool | Description |
|---|---|
mcp_prometheus_execute_query |
PromQL instant query |
mcp_prometheus_execute_range_query |
PromQL range query with start, end, step |
Query tools accept: timeout, limit, stats, lookback_delta, unlimited.
| Tool | Description |
|---|---|
mcp_prometheus_get_metric_metadata |
Metadata for a specific metric |
mcp_prometheus_list_label_names |
All label names |
mcp_prometheus_list_label_values |
Values for a specific label |
mcp_prometheus_find_series |
Find series by label matchers |
| Tool | Description |
|---|---|
mcp_prometheus_get_targets |
Scrape target list and health |
mcp_prometheus_get_build_info |
Build/version information |
mcp_prometheus_get_runtime_info |
Runtime information |
mcp_prometheus_get_flags |
Runtime flags |
mcp_prometheus_get_config |
Prometheus configuration |
mcp_prometheus_get_tsdb_stats |
TSDB cardinality statistics |
mcp_prometheus_check_ready |
Readiness check (/-/ready), works with Mimir |
| Tool | Description |
|---|---|
mcp_prometheus_get_alerts |
Active alerts |
mcp_prometheus_get_alertmanagers |
AlertManager discovery |
mcp_prometheus_get_rules |
Recording and alerting rules |
mcp_prometheus_get_alertmanager_alerts |
Alerts that notify, from Alertmanager: active, not silenced, not inhibited |
get_rules accepts the filters of GET /api/v1/rules, applied server-side by Prometheus or the
Mimir ruler: type (alert or record), rule_name, rule_group, file (the Mimir rule
namespace) and exclude_alerts. Label matchers (match[]) are not exposed because the Mimir
ruler ignores them. On Mimir the ruler API serves one tenant per request: under OAuth the caller's
alerting tenant is read by itself and, with several, org_id names one (see Multi-tenancy);
without OAuth pass org_id when no default tenant is configured. The same holds for get_alerts.
get_alertmanager_alerts reads GET /api/v2/alerts of the Alertmanager at alertmanager_url or
ALERTMANAGER_URL and returns the alerts that notify, oldest first, each with its fingerprint,
alertname, severity, start time, labels, annotations and receivers. filter takes label matchers
(team="bumblebee", severity=~"page|notify") and receiver a receiver regex; both are applied
by the Alertmanager. On Mimir the Alertmanager serves one tenant per request: under OAuth every
alerting tenant of the caller is read and the alerts merged, each with its tenant, the result
listing the tenants read; org_id, pipe-separated for several, narrows the read. Without OAuth
pass org_id when no default tenant is configured.
| Tool | Description |
|---|---|
mcp_prometheus_query_exemplars |
Exemplar queries for trace correlation |
mcp_prometheus_get_targets_metadata |
Per-target metric metadata |
Large query results are automatically truncated with guidance for the AI to refine its query.
The chart's CiliumNetworkPolicy admits only these callers; a direct call from any other pod is dropped, so it cannot skip muster's sign-in, audit trail and per-user authorization. Dropping one cuts that path off, so each has a render test (helm/mcp-prometheus/tests/ciliumnetworkpolicy_test.yaml).
| Caller | Port | Admitted by | Why |
|---|---|---|---|
muster (agent-platform) |
server | ciliumNetworkPolicy.ingress.muster |
The client of this server: it signs users in and forwards their calls. |
teleport-kube-agent (kube-system) |
server | ciliumNetworkPolicy.ingress.teleportPeers |
A muster on another cluster reaches this server through Teleport application access, and its calls arrive from the agent. |
the Gateway's proxies (envoy-gateway-system) |
server | ciliumNetworkPolicy.ingress.gatewayPeers, while gatewayAPI.enabled |
They carry the HTTPRoute's traffic to the server. |
alloy-metrics (kube-system) |
metrics | ciliumNetworkPolicy.ingress.metricsScrapers, while monitoring.enabled |
It scrapes the metrics endpoint. |
| the kubelet (host entity) | all | always | Liveness and readiness probes. |
Further clients on the server port go into ciliumNetworkPolicy.ingress.additionalPeers.
Every releasable merge to main cuts a release candidate (vX.Y.Z-rc.N, a pre-release); a candidate is proven on a test installation, and the stable vX.Y.Z is cut by running the Auto Release workflow with release-type: stable, which promotes the latest candidate.
app:
env:
- name: PROMETHEUS_URL
value: "http://mimir-gateway.monitoring:8080/prometheus"
- name: PROMETHEUS_ORGID
value: "my-tenant"app:
server:
transport: streamable-http
oauth:
enabled: true
dexClientSecret: "..." # stored in K8s Secret
encryptionKey: "..." # openssl rand -base64 32
storage:
type: valkey
valkey:
url: "valkey:6379"
trustedAudiences:
- muster-client
tenancy:
mode: grafana-organization
env:
- name: MCP_OAUTH_ISSUER
value: "https://mcp-prometheus.mc.example.io"
- name: DEX_ISSUER_URL
value: "https://dex.mc.example.io"
- name: DEX_CLIENT_ID
value: "mcp-prometheus"
- name: DEX_REDIRECT_URL
value: "https://mcp-prometheus.mc.example.io/oauth/callback"
- name: PROMETHEUS_URL
value: "http://mimir-gateway.monitoring:8080/prometheus"The platform IdP is Google, muster forwards the user's Google ID token
(auth.forwardToken: true), and there are no Mimir tenants to resolve.
app:
oauth:
enabled: true
provider: google
redirectURL: "https://mcp-prometheus.example.io/oauth/callback"
google:
clientID: "1234567890-abc.apps.googleusercontent.com" # this server's OAuth client
googleClientSecret: "..." # or existingSecret with key GOOGLE_CLIENT_SECRET
encryptionKey: "..." # openssl rand -base64 32
trustedAudiences:
- "1234567890-platform.apps.googleusercontent.com" # the platform client muster logs in with
tenancy:
mode: none
env:
- name: MCP_OAUTH_ISSUER
value: "https://mcp-prometheus.example.io"
- name: PROMETHEUS_URL
value: "http://prometheus-operated.monitoring:9090"app:
oauth:
enabled: true
dexClientSecret: "..."
encryptionKey: "..."
tenancy:
mode: static
static:
groups:
team-ops:
- prod-eu
- prod-us
team-dev:
- stagingWhen DEX_ISSUER_URL resolves to a private IP:
app:
oauth:
enabled: true
dexClientSecret: "..."
encryptionKey: "..."app:
oauth:
storage:
type: valkey
valkey:
url: "valkey.default:6379"
password: ""
tlsEnabled: false
keyPrefix: "mcp-prometheus:"mcp-prometheus/
├── cmd/ # CLI (serve, version)
├── internal/
│ ├── oauth/ # OAuth 2.1 setup (Config, NewHandler)
│ ├── server/ # ServerContext, PrometheusConfig
│ ├── tenancy/ # TenancyResolver, GrafanaOrg + static modes
│ ├── tools/prometheus/ # 18 MCP tool registrations
│ └── observability/ # /metrics, /healthz, /readyz, OTel
├── helm/mcp-prometheus/ # Helm chart
├── go.mod
└── README.md
go build -o mcp-prometheus ./...
go test ./...- Every package has a
doc.go - 80%+ unit test coverage on new code
- Run
goimports -w . && go fmt ./...before committing - Files kept under 500 lines; GoDoc on all exported members