[rollout] feat: add native Harbor backend

[JoyboyBrian]

What

• Add NativeHarborBackend: run one native Harbor trial per rollout with local, package, or Git task resolution, bounded concurrency, controller endpoint injection, and Docker, Daytona, and SkyPilot support. No AgentWorkflow or Grader to write.
• Persist Harbor-authored ATIF directly, preserve Harbor-collected artifacts, redact credential-bearing metadata, and overlay controller-reported metrics.
• Admit agents on training parity: a Harbor built-in is registered only when the training path can consume it, so eval never gets a wider agent set than training. Registered bindings are terminus-2 and oracle.
• Support bring-your-own agent loops through two custom bindings, split by how the agent actually accepts configuration:
  • custom-chat-completions — in-process agent taking api_base / llm_kwargs
  • custom-installed-chat-completions — container-side agent reading OPENAI_BASE_URL / OPENAI_API_KEY
• Add rollout admission controls, submit preflight validation, structured diagnostics, task prewarming, retry safeguards, docs, and unit coverage.

Why

Native Harbor backend development paused before the SDK gained the current URL-routed protocol, ATIF persistence, file artifact, output download, and Harbor runtime contracts. This brings the integration forward against the current SDK architecture without losing Harbor's richer native trajectory data.

Why agent admission is training-parity

Earlier revisions of this branch also shipped an SDK-side translation gateway so codex, opencode, and claude-code could be offered as eval-only agents. That has been removed, on evidence gathered against the pinned harbor==0.20.0 wheel and the real controllers:

• The protocol is not reachable. Both controllers expose exactly one model-facing route per rollout, POST /sessions/{id}/v1/chat/completions (traingate callback_server.py, monolith eval run_eval.py). Codex and OpenCode speak OpenAI Responses; Claude Code speaks Anthropic Messages.
• Translation only ever bought eval. Those agents' training blocker is opaque context management (compaction, subagents, session rewriting), which forks the token trajectory. Protocol translation cannot fix that, so they could never graduate to training.
• The translation did not actually work. Real end-to-end runs on Daytona found each of the three still failing on translation fidelity — Codex on a namespace tool type, Claude Code on an unmapped reasoning_effort, OpenCode on inconsistent Responses SSE part IDs.
• It carried a deployment hazard. The gateway's per-rollout route table was in-process state, so a multi-worker or multi-replica rollout server could answer an agent's callback from an instance that never registered the route.

Dropping the three bindings and their gateway removes ~1.7k lines and that hazard, and collapses the support model from "eval set ⊃ train set" to a single admission rule.

An import_path agent cannot reintroduce a de-registered built-in: the guard resolves the class against Harbor's own agent registry rather than the SDK's binding table.

This PR supersedes #267 with the same backend rebuilt directly on the latest main. The old brian/harbor branch inherited pre-squash commits from #235, which made its comparison against main unnecessarily noisy.

Tracks OSM-1653.

How to Test

uv run ruff check .
uv run ruff format --check .
uv run pyright osmosis_ai/
uv run pytest

1,911 passed locally.

Beyond unit tests, terminus-2 and oracle were exercised against the real harbor==0.20.0 wheel on Daytona sandboxes, serially and at concurrency 16 (16/16 passing in both cases, with valid ATIF trajectories and non-zero token metrics). Note that unit tests monkeypatch TrialQueue.submit, so they do not by themselves prove the real Harbor path.

Known follow-up

The controller API key reaches terminus-2 through AgentConfig.kwargs["llm_kwargs"]["api_key"], and Harbor serializes that config, so the key is written into each trial's config.json, lock.json, result.json, and agent/trajectory.json. In traingate that key is minted once per training run and shared by every rollout. This predates this PR's final shape and is tracked separately rather than fixed here.

Checklist

• PR title follows [module] type: description format
• Appropriate labels added
• Ruff lint and format checks pass
• Pyright passes
• Pytest passes
• Public API changes are documented
• No secrets or credentials included

[codecov]

Codecov Report

❌ Patch coverage is 90.51491% with 70 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...smosis_ai/rollout/backend/native_harbor/backend.py	89.92%	36 Missing and 28 partials ⚠️
osmosis_ai/rollout/server/app.py	90.56%	3 Missing and 2 partials ⚠️
osmosis_ai/rollout/trajectory/save.py	97.91%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!