Case Study: Autonomous Multi-Agent PR Workflow — 288 Commits, 59 PRs Merged, 10-Minute Cycle Times

[kai-linux]

TL;DR

Agent OS is an autonomous multi-agent system that turns GitHub Issues into merged PRs — dispatch, code, test, review, merge, and self-improvement, all without human intervention. In 24 days managing its own repository, it shipped 288 commits, merged 59+ PRs, and closed 79+ issues with a median PR cycle time of ~8 minutes. This post walks through a real failure-recovery workflow with links to every commit and issue.

---

The System in One Paragraph

Agent OS watches a GitHub project board. Every 60 seconds, a dispatcher picks the highest-priority ready issue, formats a task prompt, and hands it to an agent (Claude, Codex, Gemini, or DeepSeek) running in an isolated git worktree. The agent writes code, runs tests, and produces a structured result file. If tests pass, a PR is opened automatically. A PR monitor checks CI every 5 minutes — green CI triggers squash-merge; red CI creates a debug issue. A weekly log analyzer and backlog groomer file improvement tickets from operational metrics. The loop is recursive: the system improves itself through the same pipeline it runs.

Concrete Metrics (24 days, self-managed repo)

Metric	Value	Source
Total commits	288	git log
PRs merged	59+	Merged PRs
Issues closed	79+	Closed issues
Agent tasks executed	122+	runtime/metrics/agent_stats.jsonl
First-attempt success rate	55.7%	Same metrics file
Median PR cycle time	~8 min (create → merge)	Computed from gh pr list --json createdAt,mergedAt
Agents in fallback pool	4 (Claude, Codex, Gemini, DeepSeek)	config.yaml
Issue closure rate	92%	79 of 86 created
PR merge rate	91%	59 of 65 created

Deep Dive: The PR-98 Cascading Failure (and How the System Recovered)

This is the most interesting workflow because it shows both failure and recovery — the part most demos hide.

What happened

1. PR #98 merged on 2026-03-31 with a subtle bug: the CI completion verification gate extracted failed job names from markdown prose in issue bodies.

2. The cascade began. When follow-up debug tasks reformatted the issue body, the job names disappeared. The gate couldn't find them, so it downgraded successful fixes to partial, spawning new follow-up tasks. This created 8+ cascading issues: #99, #102–#106, #109, #111, #117.

3. The log analyzer detected the pattern. After accumulating evidence from the cascading failures, the system filed #148: "RCA and fix for PR-98 cascading CI failure pattern."

4. An agent performed the RCA and shipped the fix as PR #153 — 81 lines added, 2 deleted, 4 files changed. The fix: persist failed_checks as structured frontmatter metadata at dispatch time instead of parsing it from markdown prose. Created at 13:15, merged at 13:25. 10 minutes from issue to production.

Before vs. After

Aspect	Before (manual)	After (Agent OS)
Detecting the pattern	Human reads logs, notices repeated failures	Log analyzer correlates cascading task chains automatically
Root cause analysis	Human reads code, traces data flow	Agent reads the failing code path, identifies the parsing fragility
Writing the fix	Human codes, tests, opens PR	Agent writes fix + tests in isolated worktree, PR auto-opened
Review & merge	Human reviews, approves, merges	CI runs, PR monitor auto-merges on green
Total time	Hours to days (context switching)	10 minutes end-to-end
Follow-up	Hope it doesn't happen again	Structured failed_checks metadata prevents recurrence

What failed (honestly)

• 55.7% first-attempt success rate means 44% of tasks needed retries or fallback agents. Not great — but the system handles it automatically.
• The PR-98 cascade burned 8 task executions before the pattern was caught. That's real wasted compute.
• DeepSeek authentication failures caused multiple task failures before the system learned to preflight credentials (PR #101).
• Codex hit rate limits on at least one task, requiring fallback to another agent.

These aren't hidden — they're in the closed issues and CODEBASE.md.

Sample PR Cycle Times

PR	Task	Files	+/-	Cycle time
#155	Add adoption metrics to production feedback	2	+221/-0	< 1 min
#153	RCA fix for PR-98 cascade	4	+81/-2	10 min
#144	Skip signal persistence for planner	6	+499/-4	10 min
#143	PR review signal extraction	4	+554/-49	5 min
#132	Domain evaluation rubrics	9	+249/-51	15 min
#122	CI failure clustering	6	+405/-0	42 min

What the System Built (Autonomously)

Every feature below went through the full pipeline — issue → dispatch → agent codes → tests pass → PR → CI green → auto-merge:

• Parallel queue workers with per-repo locking
• Priority-aware dispatch with age-based scoring
• Task dependency resolution (Depends on #N)
• CI failure clustering by error signature
• Automatic escalation with Telegram decision cards
• Post-merge outcome attribution
• Production feedback substrate for evidence-driven planning
• Structured blocker codes and unblock notes
• Domain evaluation rubrics
• PR review signal extraction for follow-up generation
• Adaptive agent health checks (skip agents below 25% success rate)
• Sprint plan skip signal persistence

How to Verify

Every claim links to public GitHub data:

• Commits: 288 commits on main
• Merged PRs: All merged PRs
• Closed issues: All closed issues
• Self-improvement loop: Issues labeled bot-generated were filed by the system's own log analyzer and backlog groomer
• Sprint history: STRATEGY.md shows evidence-driven sprint plans
• Failure recovery: CODEBASE.md Known Issues documents the PR-98 cascade RCA

Try It

Agent OS is MIT-licensed. The README has a 5-minute quickstart, and the full case study has more detail.

If you're a solo builder or technical founder tired of coordination overhead, this might be useful. Questions welcome in this thread.