fix: Add NetworkPolicy to allow Prometheus metrics scraping on port 8383

[redhat-chai-bot]

Summary

ocm-agent-operator creates a ServiceMonitor for Prometheus to scrape port 8383, but its NetworkPolicies only allow Alertmanager and MUO ingress. This causes TargetDown alerts because Prometheus scrapes are silently dropped (context deadline exceeded on :8383/metrics).

This PR adds a NetworkPolicy allowing prometheus-k8s pods from openshift-monitoring to reach the OCM Agent on port 8383 for metrics scraping.

Root Cause

In ensureAllNetworkPolicies(), the only openshift-monitoring NetworkPolicy targets Alertmanager pods (alertmanager: main). No policy exists for Prometheus pods, so the ServiceMonitor creates a scrape target that the NetworkPolicy blocks.

PR #326 added granular pod selectors (OBO, RHOBS, MUO) but did not address the Prometheus gap.

Changes

pkg/consts/ocmagenthandler/ocmagenthandler.go

• Added OCMAgentPrometheusNetworkPolicySuffix for the policy name
• Added NamespacePrometheus dispatch key (not a literal namespace — follows the same pattern as NamespaceRHOBS)
• Added PrometheusPodLabelKey/PrometheusPodLabelValue for pod selection (app.kubernetes.io/name: prometheus)

pkg/ocmagenthandler/ocmagenthandler_networkpolicy.go

• Added NamespacePrometheus case to buildNetworkPolicyName()
• Added NamespacePrometheus case to callerPodSelector() returning Prometheus pod labels
• Added NamespacePrometheus case to callerNamespace() mapping to NamespaceMonitorng (Prometheus runs in openshift-monitoring)
• Added NamespacePrometheus to both fleet and non-fleet namespace lists in ensureAllNetworkPolicies() and ensureAllNetworkPoliciesDeleted()

pkg/ocmagenthandler/ocmagenthandler_networkpolicy_test.go

• Added test context "for the Prometheus dispatch key" with pod selector and namespace scoping specs
• Updated non-fleet policy count (2 → 3) and fleet policy count (3 → 4)

Validation

• go build ./... — passes
• go test ./pkg/ocmagenthandler/ — 78/78 specs passed

References

• Incident: OHSS-58308 (3 prior resolved incidents with the same symptom)
• OCPSTRAT-819 mandates explicit ingress-for-metrics NetworkPolicies for all operators

---

AI-generated. Review for accuracy.

@joshbranham requested in Slack thread

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Team

Run ID: 52ff5925-2966-4fc6-9f04-3ffaebc0a906

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci]

@redhat-chai-bot: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 87.50000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.60%. Comparing base (1c770c2) to head (4742074).

Files with missing lines	Patch %	Lines
...g/ocmagenthandler/ocmagenthandler_networkpolicy.go	87.50%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #366      +/-   ##
==========================================
+ Coverage   65.39%   65.60%   +0.21%     
==========================================
  Files          23       23              
  Lines        1598     1608      +10     
==========================================
+ Hits         1045     1055      +10     
  Misses        473      473              
  Partials       80       80              

Files with missing lines	Coverage Δ
pkg/consts/ocmagenthandler/ocmagenthandler.go	100.00% <ø> (ø)
...g/ocmagenthandler/ocmagenthandler_networkpolicy.go	78.57% <87.50%> (+1.64%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tkong-redhat]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: redhat-chai-bot, tkong-redhat

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [tkong-redhat]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 12:26 AM UTC · Completed 12:38 AM UTC

Commit: 4742074 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $4.80

[fullsend-ai-retro]

Retro: PR #366 — Add NetworkPolicy for Prometheus metrics scraping

Timeline

1. 2026-09-07 17:12 UTC — redhat-chai-bot (co-authored by Claude Opus 4.6) opened PR fix: Add NetworkPolicy to allow Prometheus metrics scraping on port 8383 #366 on branch fix/add-prometheus-networkpolicy. The fix adds a NetworkPolicy allowing prometheus-k8s pods from openshift-monitoring to reach the OCM Agent on port 8383, resolving incident OHSS-58308 (recurring TargetDown alerts).
2. 2026-09-07 17:12 UTC — Fullsend shim workflow run 34146638278 fired on pull_request_target / opened. Route job: "No stage matched — skipping dispatch." The review agent was not dispatched.
3. 2026-09-08 00:21 UTC — Two more workflow runs (34173032617, 34173064245) fired on labeled events (lgtm, approved). Both also skipped dispatch.
4. 2026-09-08 00:21 UTC — tkong-redhat posted /lgtm. No inline comments. No formal GitHub review.
5. 2026-09-08 00:25 UTC — PR merged by openshift-merge-bot. Retro agent dispatched (ungated closed event).

Why the review agent did not run

The Route job's pull_request_target / opened path requires either [bot]$ in the PR author's login OR at least triage permission on the repo. redhat-chai-bot fails both checks:

• Its login does not end with [bot] (it's a service account, not a GitHub App)
• It has only read permission on the repository

This meant a security-relevant change (NetworkPolicy modifications) received no automated review.

Existing upstream coverage

This gap is well-tracked in fullsend-ai/fullsend:

• #6642 — Review dispatch should accommodate fullsend bot-authored PRs in the authorization check. This retro provides additional evidence: redhat-chai-bot on openshift/ocm-agent-operator PR fix: Add NetworkPolicy to allow Prometheus metrics scraping on port 8383 #366 hit this exact gap.
• #5185 — Extend bot detection to use user.type instead of hardcoded [bot]$ regex.
• #5480 — is_org_bot() can't recognize custom bot identities.
• #7054 — Fork pull_request_target events may face additional silent blocking (PR fix: Add NetworkPolicy to allow Prometheus metrics scraping on port 8383 #366 was a cross-repo fork from redhat-chai-bot/openshift_ocm-agent-operator).

Code quality assessment

The PR itself was well-executed: clean single-commit diff (+53/-12 across 3 files), follows existing dispatch-key patterns, includes Ginkgo BDD tests for the new Prometheus dispatch key, updates policy counts in both fleet and non-fleet test contexts, and provides a thorough PR description with root cause analysis and external references. Codecov reported 87.5% patch coverage. All Prow CI tests passed.

No new proposals

The primary improvement opportunity (ensuring review dispatch fires for trusted service-account PRs) is already tracked by the upstream issues listed above. No repo-level proposals are warranted — the code quality was high and the workflow gap is a platform-level routing issue.