Skip to content

test(native): verify policy reads through CLI fixtures - #8228

Open
cocolord wants to merge 1 commit into
mainfrom
codex/small-kernel-native-readers-20261107
Open

cocolord wants to merge 1 commit into
mainfrom
codex/small-kernel-native-readers-20261107

Conversation

@cocolord

@cocolord cocolord commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

The two real CLI test fixtures intentionally return raw transport output. Add native/on and inline/off policy observations that first verify navigation reaches the caller, then use the existing current-capture oracle to check the authority decision and original Goal/Agent/Turn binding. The progress case selects its original Todo; the successor case observes the pure replan binding. Both run against real File and SQLite authority.

Current target main already passes the 84 original tests across all three consumer suites. This proposal adds 8 semantic policy-read cases; every original function body, CLI argument and assertion remains unchanged, including the staged-read suite. Python remains test adaptation of the existing policy and transport owners.

Validation:

  • Baseline at ac62962255322cd80a5b701a4187c6f47bc17d8c: 84 passed. The candidate runs those 84, the 8 new cases and 85 existing reader/reference/runtime-cleanup negative cases: 177 passed.
  • AST parity: all 34 existing functions are unchanged. The new cases cover selected-Todo and autonomous-replan identities in native/on and inline/off modes.
  • Targeted native/on and inline/off rounds pass 4/4 per module.
  • Original/new native test runs retire their owned runtimes. The combined negative suite retains 3 deliberately invalid/copied locator fixtures; the existing runtime reader rejects all 3. They are not valid runtime records.
  • Ruff, compilation, diff hygiene, public/private scan and the bilingual RFC diff check pass. No supported new semantic vocabulary carrier was detected by the scoped advisory.
  • Standard scoped premerge passes 5 direct checks and 6/7 canaries. Its dashboard-status budget check fails at 25,646 characters against the unchanged 25,000 limit. The producer, fixture and budget contract are unchanged from baseline. Matched base/head controls both emit 24,895 characters in the short namespace and 25,478 in the long namespace; both fail the long-namespace check. The original failed aggregate and budget limit are retained. This is a validation hold, not a green premerge result.
  • Remote CI and exact-head review remain open; this author proposal grants no merge readiness.

Bounded refactoring review found the existing raw callbacks and metadata oracle correct; reuse their reader at the new semantic observation sites. No production behavior or budget was changed. This qualifies source test coverage for the native decision-read boundary; installation, live-model retention, adoption and whole-kernel benefit still require their own evidence.

Signed-off-by: lusendong.6789 <lusendong.6789@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent; gpt-6.1-sol; OpenAI; runtime_reported; reasoning_effort=max.

动机 / Motivation

维护重规划 CLI 回归的开发者,需要同时确认原始传输导航可用和当前权威决策仍绑定原 Goal、Agent、Turn 与 Todo。 以前这两个真实 CLI 夹具主要在 off 模式检查策略,native 导航通过并不证明原任务身份可读;现在同样的 File/SQLite 输入在 on/off 两侧均检查原决策与身份。 当前 head 的六模块完整运行 217 项通过,新增 8 项检查覆盖两种绑定;实际主干原有三模块 84 项通过,原函数 AST 和 staged-read 源码保持不变。 本 PR 只交付源码测试覆盖,不修改生产准入、调度、结算、预算或默认值,也不认证实际模型阅读、安装或长期收益。

本次固定 8228@70d0242b2c1f0c0caf17b62ba82fd29a31901cac,实际 base/merge-base 为 ac62962。这是补足实际消费边界的测试任务;原有测试已经通过,无需制造生产缺陷来说明新增覆盖的价值。

改动思路 / Approach

保留 raw transport 返回值,先确认 native 导航确实抵达调用方,再复用 read_cli_policy_payload 读取同次完整恢复。progress 检查原 Todo,successor 检查 autonomous_replan 绑定;两种身份不同,不能只验证导航形状。既有 reader 负责摘要、权限和失败处理,测试不新增策略 owner,也不把读取磁盘说成模型已读。

两条新增分支用 prose 已足够说明,没有新增状态机需要图示。与同作者 #8222 的 first-connect 修复比较后,这里保护的是另两类重规划调用方,未重复造一套 smoke 或解码器。

具体改动 / Changes

独立查看完整两文件差异:+72/-0,只有两个 import 和两个参数化测试。15 个原有函数 AST 全同,第三个 staged-read 模块源码未改;生产代码、CLI 参数和原断言均未变。

规格为 docs/development/good-smokes.md,spec_revision ac62962。Start With The Contract:真实 CLI 与 File/SQLite 覆盖稳定身份不变量;Prefer Semantic Pressure:8 个新用例配合现有 reader/引用/清理负例;Consolidate Without Losing Coverage:复用既有 reader,保持原夹具与断言。三个标准均在当前头满足,测试数量只是证据范围,不是产品交付量。

对主干的风险 / Risks

未发现当前改动的阻塞性缺陷。独立六模块完整运行 217 PASS,实际主干原有三模块 84 PASS;配置内17入口 Mypy PASS。standard premerge 的5个直接检查、7个选择检查机械 PASS,但总体 FAIL,原因是严格 CQR 保留必检 Ruff 失败。

同一 Ruff 命令在不可变 base/head 均产生 test_goal_create_completion.py:184 的同一 F841;仅归一化工作树前缀后,完整诊断、因果文件及配置一致。它是既有无关失败,不能要求本 PR 改无关文件;也不能将必检或严格 CQR 记为通过。本轮未查询、轮询或等待 CI。作者此前 namespace 预算失败仍是原观测;本次7项全部通过不解释其历史原因,也不重写旧结果。

我的整体评价 / Verdict

源码评审 APPROVE;long_horizon=preserved,user_experience=not_applicable。新增覆盖不会引入用户准备、确认或执行轮次,保留原始任务身份与现有失败边界。检查两处测试及其未改动的 CLI 消费路径后,确认没有渲染页面、布局或产品控件变化,视觉评审不适用。 未验证安装态 App 或真实模型采用;本 PR 不改变这些渲染表面。

这不是 merge readiness、合并授权或整个 small-kernel 目标完成。严格质量门仍 FAIL;实际安装、模型采用和完整成本不由这组源码测试认证。

English verdict: APPROVE

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants