STAC-24773 — Omnibus → Bazel migration handoff

Handoff for humans and AI pair-programming on the StackState Agent fork.
Ticket: STAC-24773 · MR: !426 · Branch: STAC-24773-bazel-migration

Status (June 2026)

Milestone	State
Phase A (initial Bazel deps: libpcap, snmp-traps, jmxfetch, systemd, …)	Done (earlier commits on branch)
Phase B (openscap chain, curl/nghttp2)	Done
Phase C (orphan `.rb` sweep + integrations-chain deps)	Done (C1–C3, C5, C6, C-E)
Phase D1 (`python3.rb` → `@cpython`, pip 26, pip entrypoints)	Done — pipeline 2628270529 + Beest green
Phase D2 (drop python-build `.rb` orphans)	Done — pipeline 2628732118 + Beest green
Phase D3 (`libffi`/`libtool` omnibus orphans)	Done — pipeline 2629204999 + Beest green
`zlib.rb` / `openssl.rb` (1.x)	Removed — STAC-25143 wave 2 (openssl3 → Bazel, openssl 1.x orphan swept)
Non-Bazel cleanup MR	Not started — separate ticket (not Bazel migration)

Commit history on the branch may be squashed; use git log --grep=STAC-24773 and file contents (grep STAC-24773 in omnibus/) rather than assuming one commit per phase.

Why this exists

Datadog Agent 7.78 moved most native dependencies from omnibus/config/software/*.rb to Bazel (bazelisk run @dep//:install, //packages/agent/dependencies:install). STS stayed hybrid after the 7.71.2 → 7.78.2 merge, maintaining dozens of extra Ruby recipes. STAC-24773 closes that gap in waves so each wave is CI-gated.

Driver: upstream alignment and lower merge cost — not housekeeping alone.

Resume protocol (AI or human)

Branch: git checkout STAC-24773-bazel-migration (not STAC-25035 CVE work unless explicitly switching tickets).
Read: this file + omnibus/config/software/datadog-agent-dependencies.rb (Bazel install blocks).
Do not use: ~/where-do-I-go-from-here.txt (archived May 2026 merge handoff).
Claude session name: bazel-migration-phase-b-c (session 792c5837-…) — optional; repo docs are canonical.
Plan file (historical): ~/.claude/plans/reactive-enchanting-hamming.md.

CI gating cadence

Push one logical commit → wait for build_deb x86 + arm green → push next.
Beest extra gate: after C2 (libxml2/libxslt) and before relying on integrations ODBC paths; not required between every phase.

Required Bazel flag on STS runners

All bazelisk run invocations from omnibus must include:

--downloader_config=/dev/null

Also pass flavor_flag (--//packages/agent:flavor=fips or heroku) where upstream does. Without /dev/null, fetches go through DD's internal mirror and time out on GitLab STS runners.

Core patterns

Cache ownership

Keep always_build true in datadog-agent-dependencies.rb. Omnibus fingerprints its Ruby recipes, but does not track the Bazel dependency manifests and overlays. The stage must invoke Bazel on every package build so Bazel can validate its own cache against those inputs. After a native dependency update, verify the version inside the resulting DEB or image, including when Omnibus restores a warm cache. Image builds using mutable base tags must pull those tags before building so cached base images do not hide distribution security updates.

Shared libraries → `install` + `replace_prefix`

Do not ship Bazel-built .so files only via //packages/agent/dependencies:install / pkg_filegroup all_files if they need to live under /opt/stackstate-agent/embedded/lib with embedded RPATH. That failed omnibus health check (C2 pipeline 2619669055): DT_NEEDED pointed at system /lib for libz / liblzma.

Do mirror curl/nghttp2/libxml2:

command_on_repo_root "bazelisk run #{flavor_flag} --downloader_config=/dev/null -- @dep//:install --destdir='#{install_dir}'"
command_on_repo_root "bazelisk run #{flavor_flag} --downloader_config=/dev/null -- //bazel/rules:replace_prefix --prefix '#{install_dir}/embedded' #{install_dir}/embedded/lib/lib….so"

Canonical templates: git show origin/base-7.78.2:omnibus/config/software/datadog-agent-dependencies.rb (Linux block).

Pre-delete audit (Pitfall #12)

Before deleting omnibus/config/software/foo.rb, grep both quote styles:

rg "dependency ['\"]foo['\"]" omnibus/

Remove every consumer dependency line in the same commit, or omnibus parse fails.

STS shebang branding

deps/gstatus/BUILD.bazel and deps/nfsiostat/BUILD.bazel hardcode /opt/datadog-agent/embedded/bin/python. After Bazel install, re-stamp in omnibus:

command "sed -i '1s|.*|#!#{install_dir}/embedded/bin/python|' #{install_dir}/embedded/sbin/gstatus"

@cpython console scripts (pip3.*) have the same problem: Bazel bakes //:install_dir (/opt/datadog-agent) while fix_branding.sh sets omnibus install_dir to /opt/stackstate-agent. Re-stamp in python3.rb after pip upgrade; integrations invoke pip via python -m pip (upstream 7.78.2).

Local probe: scripts/dev/test-python3-pip-entrypoint.sh (run inside build container via local.sh cmd).

(Long-term: ABLD-302-style config_flag in Bazel; until then, omnibus sed is intentional.)

Where Bazel installs live (post B+C)

Deps	Omnibus hook
jmxfetch, libpcap, snmp-traps, systemd (top-level)	`datadog-agent-dependencies.rb` → `//packages/agent/dependencies:install`
nghttp2, curl, libxml2, libxslt	`datadog-agent-dependencies.rb` Linux `build do`
secret-generic-connector	Not packaged by STS; use a custom `secret_backend_command` such as `/readsecret.py` instead
unixodbc, freetds, msodbcsql18	`datadog-agent-integrations-py3-dependencies.rb`
gstatus, nfsiostat	same (plus shebang `sed`)

stackstate-agent-integrations-py3.rb depends on datadog-agent-integrations-py3-dependencies (after datadog-agent).

Phase map (what each wave dropped)

Phase	Omnibus recipes removed / notes
B1	openscap → Bazel; attr, libacl, libgcrypt, libgpg-error, libyaml, popt, rpm, xmlsec, …
B2	curl.rb, nghttp2.rb
C1	dbus, systemd, libselinux, libsepol, pcre2, util-linux, expat (+ iot-agent.rb systemd dep)
C2	libxml2, libxslt only (libffi, unixodbc, nfsiostat deferred — see below)
C3	file, elfutils, m4, libxcrypt (+ drop libxcrypt from python3.rb)
C5	freetds, msodbcsql18, unixodbc, secret-generic-connector.rb; STAC-25211 later removed the connector from STS packaging because `secret_backend_type` is not a supported STS chart surface
C6	gstatus, nfsiostat, lua, sysstat
C-E	mac-app, cf-finalize, buildpack-finalize, cacerts_py{2,3}_local (+ agent-binaries buildpack dep)
D1	`python3.rb` → Bazel `@bzip2`/`@xz`/`@sqlite3`/`@cpython`; drop `pip3.rb`; pip 26.0.1; `python -m pip` in integrations; re-stamp `pip3.*` shebangs
D2	`bzip2.rb`, `liblzma.rb`, `libsqlite3.rb`, `libdb.rb`, `libiconv.rb` (orphaned after D1 Bazel installs)
D3	`libffi.rb`, `libtool.rb`; drop arm `dependency 'libffi'` from STS integrations recipes (upstream 7.78.2 has none)

Still kept: none — zlib.rb was removed post-STAC-24773 by STAC-25143 wave 2 (openssl3 → Bazel).

Phase D (same ticket — STAC-24773)

Goal: Replace omnibus python3.rb source build with upstream pattern: @bzip2, @xz, @sqlite3, @cpython//:install, pip 26.0.1 bump; drop orphaned python-build recipes.

D1 complete (June 2026): build_deb pipeline 2628270529 + Beest green after pip entrypoint fixes (fix-ups 5–6).

D2 complete (June 2026): pipeline 2628732118 + Beest green.

D3: Drop libffi.rb / libtool.rb and STS-only arm libffi omnibus deps; @cpython links @libffi//:ffi statically (upstream 7.78.2 pattern).

Parallel workstreams (do not confuse)

Ticket	Branch / focus
STAC-24773	`STAC-24773-bazel-migration` — this doc
STAC-25035	CVE remediation, `stackstate-deps.json` integrations pin `7.78.2-2`, prometheus/OTel
STAC-25069	Cutover docs, beest/helm-charts branches
Sandbox soak	`github.com/StackVista/argocd-apps` — `cluster_definitions/sandbox-main/apps/suse-observability-agent/values.yaml`; pins auto-drop 00:07 UTC

Integrations coupling

Integrations git: github.com/StackVista/stackstate-agent-integrations
Pin: STACKSTATE_INTEGRATIONS_VERSION in stackstate-deps.json (e.g. tag 7.78.2-2)
Keep python_version = "3.13" in stackstate-agent-integrations-py3.rb in sync with python3.rb default_version
Arm builds: libxml2/libxslt via Bazel in datadog-agent-dependencies.rb (C2); no omnibus libffi dep (D3, matches upstream)

After upstream merge

Do not run the old “restore all .rb from stackstate-<prev>” script without review.
For each DD-deleted software def, check whether STAC-24773 already Bazel-installs it (grep bazelisk + this doc).
Prefer taking DD's Bazel wiring from origin/base-7.78.2 over restoring pre-7.78 STS recipes.

References

UPSTREAM_MERGE.md — merge workflow, sandbox soak, cutover
.claude/skills/omnibus-to-bazel/SKILL.md — single-dep migration procedure
Claude memory: omnibus-bazel-migration-gap.md (short index, points here)

MR !426 description (keep in sync with branch tip)

Copy into MR !426 when status changes.

## STAC-24773 — Omnibus → Bazel migration (7.78.2)

Migrates StackState Agent native/python dependencies from omnibus Ruby recipes to Bazel (`bazelisk run @dep//:install`), aligned with Datadog Agent 7.78.2. All `bazelisk` invocations use `--downloader_config=/dev/null` for STS runner egress.

### Phases complete

| Phase | Summary |
|-------|---------|
| A | libpcap, snmp-traps, jmxfetch, systemd, … |
| B | openscap chain, curl, nghttp2 |
| C | Orphan `.rb` sweep + integrations-chain deps (C1–C3, C5, C6, C-E) |
| D1 | `python3.rb` → `@cpython`; pip 26.0.1; `python -m pip` in integrations; pip3 shebang re-stamp |
| D2 | Drop `bzip2`, `liblzma`, `libsqlite3`, `libdb`, `libiconv` (Bazel replaces them) |
| D3 | Drop `libffi`, `libtool`; remove STS arm `libffi` dep from integrations recipes |

### CI / validation

- D1: `build_deb` + Beest — [pipeline 2628270529](https://gitlab.com/stackvista/agent/stackstate-agent/-/pipelines/2628270529) ✅
- D2: `build_deb` + Beest — [pipeline 2628732118](https://gitlab.com/stackvista/agent/stackstate-agent/-/pipelines/2628732118) ✅
- D3: `build_deb` + Beest + unit tests — [pipeline 2629204999](https://gitlab.com/stackvista/agent/stackstate-agent/-/pipelines/2629204999) ✅

### Deferred (not Phase D)

- `zlib.rb` — removed post-STAC-24773 by STAC-25143 wave 2 (openssl3 → Bazel).

### Test plan

- [x] `build_deb` green (x86 + arm) — D1–D3
- [x] Beest on branch tip — D1–D3
- [x] Unbranded unit tests — [pipeline 2629204999](https://gitlab.com/stackvista/agent/stackstate-agent/-/pipelines/2629204999)
- [ ] Sandbox soak (optional before merge to `stackstate-7.78.2`)
- [ ] Reviewer smoke: metrics/topology/checks in StackState UI

### Docs

- Handoff: `docs/dev/stac-24773-bazel-migration.md`
- Local pip probe: `scripts/dev/test-python3-pip-entrypoint.sh`

Navigation Menu

Repository navigation

stac-24773-bazel-migration.md

stac-24773-bazel-migration.md

STAC-24773 — Omnibus → Bazel migration handoff

Status (June 2026)

Why this exists

Resume protocol (AI or human)

CI gating cadence

Required Bazel flag on STS runners

Core patterns

Cache ownership

Shared libraries → `install` + `replace_prefix`

Pre-delete audit (Pitfall #12)

STS shebang branding

Where Bazel installs live (post B+C)

Phase map (what each wave dropped)

Phase D (same ticket — STAC-24773)

Parallel workstreams (do not confuse)

Integrations coupling

After upstream merge

References

MR !426 description (keep in sync with branch tip)

FilesExpand file tree

stac-24773-bazel-migration.md

Latest commit

History

stac-24773-bazel-migration.md

File metadata and controls

STAC-24773 — Omnibus → Bazel migration handoff

Status (June 2026)

Why this exists

Resume protocol (AI or human)

CI gating cadence

Required Bazel flag on STS runners

Core patterns

Cache ownership

Shared libraries → install + replace_prefix

Pre-delete audit (Pitfall #12)

STS shebang branding

Where Bazel installs live (post B+C)

Phase map (what each wave dropped)

Phase D (same ticket — STAC-24773)

Parallel workstreams (do not confuse)

Integrations coupling

After upstream merge

References

MR !426 description (keep in sync with branch tip)

Shared libraries → `install` + `replace_prefix`