Skip to content

Commit 4658aa6

Browse files
authored
feat: MVP closed loop — observer, scope, hard stops, ledger (PR #2)
Squash-merge of hitome0123's MVP implementation. Includes complete ledger on all paths (including scope-violation), strict scope matching with boundary tests, Observer, YAML config, hard stops, and stub role examples. Selected over PR #3 due to stricter audit coverage and adversarial self-review.
1 parent f1e6487 commit 4658aa6

22 files changed

Lines changed: 1511 additions & 39 deletions

File tree

‎.github/workflows/tests.yml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: tests
2+
3+
on:
4+
push:
5+
branches: [main, "feat/**", "fix/**"]
6+
pull_request:
7+
branches: [main]
8+
9+
jobs:
10+
test:
11+
runs-on: ubuntu-latest
12+
strategy:
13+
fail-fast: false
14+
matrix:
15+
python-version: ["3.10", "3.12"]
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Set up Python ${{ matrix.python-version }}
20+
uses: actions/setup-python@v5
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
24+
- name: Configure git identity
25+
# `_commit_candidate` injects per-commit identity, but a couple of
26+
# tests still call `git commit` against a fresh tempdir directly.
27+
run: |
28+
git config --global user.email "ci@evolution-kernel.local"
29+
git config --global user.name "evolution-kernel-ci"
30+
31+
- name: Install package (PyYAML is the only runtime dep)
32+
run: python -m pip install -e .
33+
34+
- name: Run unit + acceptance + scope tests
35+
run: python -m unittest discover -s tests -v
36+
37+
- name: Run token-ignition golden cases
38+
run: python adapters/token_ignition/evaluate_golden_cases.py

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2026 Cara Lai
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
</p>
1414

1515
<p align="center">
16+
<a href="https://github.com/hitome0123/evolution-kernel/actions/workflows/tests.yml"><img src="https://github.com/hitome0123/evolution-kernel/actions/workflows/tests.yml/badge.svg?branch=feat/mvp-observer-scope-hardstops" alt="tests"></a>
1617
<img src="https://img.shields.io/badge/status-v0%20prototype-orange" alt="Status: v0 prototype">
1718
<img src="https://img.shields.io/badge/python-%3E%3D3.10-blue" alt="Python >= 3.10">
1819
<img src="https://img.shields.io/badge/license-MIT-green" alt="MIT License">
@@ -101,6 +102,17 @@ python3 adapters/token_ignition/evaluate_golden_cases.py
101102

102103
## CLI Shape
103104

105+
YAML-config mode (the primary MVP entry point — observer + scope + hard stops):
106+
107+
```bash
108+
python3 -m evolution_kernel.cli \
109+
--config /path/to/evolution.yml \
110+
--repo /path/to/target-repo \
111+
--ledger /path/to/evolution-ledger
112+
```
113+
114+
Legacy direct-flags mode (still supported for the original golden-case tests):
115+
104116
```bash
105117
python3 -m evolution_kernel.cli \
106118
--repo /path/to/target-repo \
@@ -111,10 +123,120 @@ python3 -m evolution_kernel.cli \
111123
--evaluator python3 /path/to/evaluator.py
112124
```
113125

126+
Reset the persistent hard-stop state (after a halt) without running a loop:
127+
128+
```bash
129+
python3 -m evolution_kernel.cli --reset --ledger /path/to/evolution-ledger
130+
```
131+
114132
Each role command receives:
115133

116134
```text
117135
--input <json>
118136
--output <json>
119137
--worktree <sandbox path>
120138
```
139+
140+
## MVP Usage (closed loop with observer, scope, hard stops)
141+
142+
This MVP wires the full closed loop described in the protocol:
143+
`config -> observe -> plan/execute -> evaluate -> accept/reject -> ledger`.
144+
145+
### 1. Author an `evolution.yml`
146+
147+
```yaml
148+
mission: "Add a minimal in-scope mutation so the evaluator accepts."
149+
150+
evidence_sources:
151+
- type: file
152+
path: metrics.json
153+
- type: shell
154+
command: "bash scripts/status.sh"
155+
156+
mutation_scope:
157+
allowed_paths:
158+
- "src/"
159+
160+
hard_stops:
161+
max_iterations: 3
162+
max_consecutive_failures: 2
163+
164+
roles:
165+
planner: ["python3", "bots/planner.py"]
166+
executor: ["python3", "bots/executor.py"]
167+
evaluator: ["python3", "bots/evaluator.py"]
168+
```
169+
170+
`evidence_sources` are read into `observation.json` before the planner runs.
171+
`mutation_scope.allowed_paths` are enforced after the executor commits — anything
172+
outside the scope is auto-rejected with `decision.reason = "scope_violation: ..."`.
173+
`hard_stops` persist across runs in `<ledger>/.evolution_state.json` so a stuck
174+
loop halts even across CLI invocations.
175+
176+
### 2. Run a single iteration
177+
178+
```bash
179+
# one-time: install the package (pulls PyYAML, the only runtime dep)
180+
python3 -m pip install -e .
181+
182+
# one-time: prepare a target repo
183+
bash examples/demo_target/setup.sh
184+
185+
python3 -m evolution_kernel.cli \
186+
--config examples/evolution.yml \
187+
--repo examples/demo_target \
188+
--ledger /tmp/ek-ledger
189+
```
190+
191+
> The `pip install -e .` step is only needed once per environment — it pulls
192+
> `PyYAML>=6.0` (declared in `pyproject.toml`). After that the three-line
193+
> command above is reproducible from a clean checkout.
194+
195+
Reset the persistent hard-stop counters when you want to start fresh:
196+
197+
```bash
198+
python3 -m evolution_kernel.cli --reset --ledger /tmp/ek-ledger
199+
```
200+
201+
### 3. Inspect the ledger
202+
203+
Every run produces a directory under `<ledger>/runs/<run_id>/` containing the
204+
full evidence trail:
205+
206+
```text
207+
goal.json # legacy mode only
208+
config.json # full snapshot of the YAML config (full mode)
209+
observation.json # what the observer collected before planning
210+
plan.json # planner output
211+
patch.diff # diff between baseline and candidate commit
212+
candidate_commit.txt # the candidate commit hash inside the sandbox
213+
evaluation.json # evaluator output (synthesised on scope_violation)
214+
decision.json # accept / reject + reason
215+
reflection.json # post-decision summary
216+
```
217+
218+
### 4. Acceptance criteria -> tests
219+
220+
The six acceptance bullets from issue #1 each map to a test in
221+
`tests/test_acceptance.py`:
222+
223+
| # | Acceptance bullet | Test |
224+
| - | --- | --- |
225+
| 1 | Accept advances `evolution/accepted` | `test_accept_advances_accepted_branch` |
226+
| 2 | Reject does not advance it | `test_reject_does_not_advance_accepted_branch` |
227+
| 3 | Mutation scope enforced + violation logged | `test_scope_violation_is_rejected_and_logged` |
228+
| 4 | Observer writes `observation.json` (file + shell) | `test_observer_writes_observation_with_file_and_shell` |
229+
| 5 | Hard stops halt then `reset` re-enables | `test_hard_stop_blocks_then_reset_allows_via_cli` |
230+
| 6 | Ledger contains all required artifacts | `test_ledger_contains_all_required_artifacts` |
231+
232+
### What this MVP intentionally does **not** do
233+
234+
In line with the issue's "out of scope" list:
235+
236+
- No LLM / agent-swarm / dashboard.
237+
- No PR router and no auto-merge to upstream `main`.
238+
- No multi-target adapter framework — the only example target is
239+
`examples/demo_target/`.
240+
- No container/process sandbox beyond git worktrees.
241+
242+
These are the natural next steps once the kernel itself is trusted.

‎README.zh.md‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
</p>
1414

1515
<p align="center">
16+
<a href="https://github.com/hitome0123/evolution-kernel/actions/workflows/tests.yml"><img src="https://github.com/hitome0123/evolution-kernel/actions/workflows/tests.yml/badge.svg?branch=feat/mvp-observer-scope-hardstops" alt="tests"></a>
1617
<img src="https://img.shields.io/badge/status-v0%20prototype-orange" alt="状态:v0 原型">
1718
<img src="https://img.shields.io/badge/python-%3E%3D3.10-blue" alt="Python >= 3.10">
1819
<img src="https://img.shields.io/badge/license-MIT-green" alt="MIT License">
@@ -101,6 +102,17 @@ python3 adapters/token_ignition/evaluate_golden_cases.py
101102

102103
## CLI 形状
103104

105+
YAML 配置模式(MVP 主入口 — 包含 observer + scope + hard stops):
106+
107+
```bash
108+
python3 -m evolution_kernel.cli \
109+
--config /path/to/evolution.yml \
110+
--repo /path/to/target-repo \
111+
--ledger /path/to/evolution-ledger
112+
```
113+
114+
旧版直接传参模式(保留以兼容原始的 golden-case 测试):
115+
104116
```bash
105117
python3 -m evolution_kernel.cli \
106118
--repo /path/to/target-repo \
@@ -111,10 +123,119 @@ python3 -m evolution_kernel.cli \
111123
--evaluator python3 /path/to/evaluator.py
112124
```
113125

126+
熔断后清空持久化的 hard-stop 状态(不会触发一次 run):
127+
128+
```bash
129+
python3 -m evolution_kernel.cli --reset --ledger /path/to/evolution-ledger
130+
```
131+
114132
每个角色命令都会收到:
115133

116134
```text
117135
--input <json>
118136
--output <json>
119137
--worktree <sandbox path>
120138
```
139+
140+
## MVP 使用方式(observer + scope + hard stops 闭环)
141+
142+
本 MVP 串起协议描述的完整闭环:
143+
`config -> observe -> plan/execute -> evaluate -> accept/reject -> ledger`。
144+
145+
### 1. 编写 `evolution.yml`
146+
147+
```yaml
148+
mission: "Add a minimal in-scope mutation so the evaluator accepts."
149+
150+
evidence_sources:
151+
- type: file
152+
path: metrics.json
153+
- type: shell
154+
command: "bash scripts/status.sh"
155+
156+
mutation_scope:
157+
allowed_paths:
158+
- "src/"
159+
160+
hard_stops:
161+
max_iterations: 3
162+
max_consecutive_failures: 2
163+
164+
roles:
165+
planner: ["python3", "bots/planner.py"]
166+
executor: ["python3", "bots/executor.py"]
167+
evaluator: ["python3", "bots/evaluator.py"]
168+
```
169+
170+
`evidence_sources` 在 planner 运行前被读入 `observation.json`。
171+
`mutation_scope.allowed_paths` 在 executor 提交后被强制校验 —— 范围之外
172+
的任何改动都会被自动 reject,`decision.reason` 写为 `scope_violation: ...`。
173+
`hard_stops` 通过 `<ledger>/.evolution_state.json` 跨 run 持久化,循环卡死
174+
时即使重启 CLI 也会被拦截。
175+
176+
### 2. 跑一次实验
177+
178+
```bash
179+
# 一次性:安装包(PyYAML 是唯一运行时依赖,已在 pyproject.toml 声明)
180+
python3 -m pip install -e .
181+
182+
# 一次性:准备目标仓库
183+
bash examples/demo_target/setup.sh
184+
185+
python3 -m evolution_kernel.cli \
186+
--config examples/evolution.yml \
187+
--repo examples/demo_target \
188+
--ledger /tmp/ek-ledger
189+
```
190+
191+
> 上面 `pip install -e .` 每个环境只需要做一次。之后那三行 CLI 命令是
192+
> 干净 checkout 下可复现的。
193+
194+
需要重置熔断器从头来过:
195+
196+
```bash
197+
python3 -m evolution_kernel.cli --reset --ledger /tmp/ek-ledger
198+
```
199+
200+
### 3. 检查 ledger
201+
202+
每一次 run 都会在 `<ledger>/runs/<run_id>/` 下产出完整的证据链:
203+
204+
```text
205+
goal.json # 仅 legacy 模式
206+
config.json # 完整 YAML 配置快照(full 模式)
207+
observation.json # planning 之前 observer 收集到的证据
208+
plan.json # planner 输出
209+
patch.diff # baseline 与 candidate commit 之间的 diff
210+
candidate_commit.txt # sandbox 中 candidate commit 的 hash
211+
evaluation.json # evaluator 输出(scope_violation 时由 Governor 合成)
212+
decision.json # accept / reject + 原因
213+
reflection.json # 决策后的总结
214+
```
215+
216+
### 4. 验收标准 → 测试映射
217+
218+
issue #1 中六条验收标准在 `tests/test_acceptance.py` 中各对应一个测试:
219+
220+
| # | 验收要求 | 测试 |
221+
| - | --- | --- |
222+
| 1 | accept 推进 `evolution/accepted` | `test_accept_advances_accepted_branch` |
223+
| 2 | reject 不推进 | `test_reject_does_not_advance_accepted_branch` |
224+
| 3 | 强制 mutation scope + 记录违规 | `test_scope_violation_is_rejected_and_logged` |
225+
| 4 | observer 写出 `observation.json`(file + shell) | `test_observer_writes_observation_with_file_and_shell` |
226+
| 5 | hard stops 触发熔断后 `--reset` 恢复 | `test_hard_stop_blocks_then_reset_allows_via_cli` |
227+
| 6 | ledger 包含全部必需 artifact | `test_ledger_contains_all_required_artifacts` |
228+
229+
此外 `tests/test_scope.py` 单独钉死了 `allowed_paths` matcher 的边界语义
230+
(递归 / 精确匹配 / 兄弟名碰撞 / `..` 逃逸 / 空作用域 等)。
231+
232+
### 本 MVP 有意**不做**的内容
233+
234+
按照 issue 的“不要做”清单:
235+
236+
- 不做 LLM / agent-swarm / dashboard。
237+
- 不做 PR router,不做自动 merge 到上游 `main`。
238+
- 不做多目标适配器框架 —— 唯一示例目标是 `examples/demo_target/`。
239+
- 不做超出 git worktree 的容器/进程级沙箱。
240+
241+
这些都是在内核本身被信任之后才适合做的下一步。

0 commit comments

Comments
 (0)