Skip to content

simplify: coarser mention gates, tested poll scripts, anchor-based run window - #965

Merged
max-sixty merged 4 commits into
mainfrom
radical-simplify
Aug 13, 2026
Merged

simplify: coarser mention gates, tested poll scripts, anchor-based run window#965
max-sixty merged 4 commits into
mainfrom
radical-simplify

Conversation

@max-sixty

@max-sixty max-sixty commented Aug 13, 2026

Copy link
Copy Markdown
Owner

Cuts the four complexity hotspots from the recent bot-PR audit back to the #960 vision — mechanism only where it prevents wrong outward actions — and moves the surviving logic under real tests. Everything loaded or executed in production shrinks (~420 lines across the template, generated workflows, and skill prose); the growth is tests for scripts that previously had none.

  • mention.yaml.j2 verify gates: 4 → 2. The stacked review-path skip gates collapse into two structural rules: a bot-authored review summons a session only as the reviewer→author handoff (fresh content on a PR the bot authored), and a contentless approval is terminal whoever submitted it. GitHub rejects self-approvals, so the empty-body-APPROVED gate was already subsumed by an author-keyed rule. Outward behavior is unchanged — verified case-by-case twice (by hand, and independently by the review sweep's verifier); each rendered workflow drops ~52 lines.
  • running-in-ci: 869 → 677 lines. The two markdown poll recipes become tested scripts. poll-pr-checks.sh polls by commit OID, never the PR head, fixing the false green a concurrent push caused (closes running-in-ci: CI-monitor poll follows the PR head, so a concurrent push makes a run report another commit's green as its own #829) and carrying the edge cases fix(running-in-ci): pin the CI-monitor poll to the SHA the run pushed #837's five review rounds surfaced — null merge-ref rollups, superseded check runs, STARTUP_FAILURE/ACTION_REQUIRED bucketing — so it supersedes fix(running-in-ci): pin the CI-monitor poll to the SHA the run pushed #837. rerun-failed-jobs.sh owns the rerun and finds the new attempt's jobs by run_attempt, so a fast rerun no longer reads as "nothing re-queued" and an unregistered one can't report stale conclusions as fresh. The rarely-taken skill-PR mechanics move to references/skill-pr-workflow.md.
  • list-recent-runs.sh: 207 → 113 lines. The completion window becomes "floor = the last successful run's start, clamped at 6h, else 6h with a coverage-gap warning". Any successful run's window opens at or before its own start, so consecutive windows overlap and never gap — which makes the cron parsing, tick tiling, dropped-tick recovery, and the schedule-event restriction all unnecessary, at any cadence. The retry wrapper goes too: a transient failure now fails loud, and the next tick's floor reaches back past the lost window. This supersedes the window-anchoring half of perf(review-reviewers): cut cadence to 3-hourly, and anchor the window to the cron period #845; its cadence-cut half stands alone and needs no script support under anchor-based windows.
  • review-reviewers Non-issues: 7 carve-outs → 3 structural rules (designed no-ops at whatever layer catches them, designed silence, reviewer-role independence), with the closed menu folded into Gate 3's cost classification.

Also: pre-commit's shellcheck now covers plugins/**/*.sh (it covered none of them), and the fake-gh test scaffold consolidates into generator/tests/__init__.py (five copies → one).

Testing

The scripts are exercised end-to-end under pytest with fake gh/date/sleep binaries, with the scripts' own jq filters doing the reductions: 21 cases for the poll scripts (false-green guards: >100-context page, null rollup, superseded runs, moved head), 11 for the window logic (anchoring, clamping, self-exclusion, loud failure), and the mention gate consolidation is pinned by the reworked test_generate.py assertions plus regenerated snapshots. Full suite green. The verify gate itself is still string-shape-tested rather than executed — extracting it to a .sh template like review-gate.sh is the natural follow-up, left out here to keep this diff reviewable.

Review-hardening notes (later commits)

An 8-angle verified code-review of the first commit found the substantive gaps in the new scripts themselves, all fixed and pinned by tests: the GraphQL query's one 100-node page could hide a red check past the cap (now routes to UNVERIFIED); a nullable startedAt could let a stale settled check run outrank its QUEUED replacement (a group with any non-terminal entry now reads pending); a transient API blip on a late poll iteration discarded the pending list the cap report names (the last usable rollup is kept); rerun discovery by job status misread fast reruns (now keyed on run_attempt advancing); and overlapping workflow prefixes double-counted runs (unique_by). The known remaining limit: latest-wins can't distinguish two independent runs of a group-less workflow on one SHA — documented in the script header and the skill, with the instruction to read the specific run's conclusion where that distinction must be exact.

A further tend-review round on the PR added: explicit --repo/owner arguments on the three gh calls that leaned on cwd remote detection (with test assertions pinning the flags), a Step 1 sentence in review-reviewers covering a failing list-recent-runs.sh call (retry once, then record a coverage gap — the run still concludes green, so the next tick anchors past the span), and the zero-checks commit named in poll-pr-checks.sh's exit-2 docs.

This was written by Claude Code on behalf of max-sixty

max-sixty and others added 2 commits August 12, 2026 16:36
…n window

Four complexity hotspots cut back to the #960 vision (simplicity outranks
efficiency; complexity only where it prevents wrong outward actions):

- mention.yaml.j2: the four review-path skip gates collapse into two
  structural rules — a bot review summons a session only as the
  reviewer→author handoff (fresh content on the bot's own PR), and a
  contentless approval is terminal for everyone. Outward behavior is
  unchanged case-by-case; GitHub's self-approval rejection is what makes
  the empty-body-APPROVED gate redundant.
- running-in-ci: the two CI-poll recipes move to tested scripts.
  poll-pr-checks.sh pins to a given SHA (fixing #837's false-green defect:
  the old recipe followed the PR's moving head) and carries that PR's
  edge cases — null-rollup guard, superseded-check-run reduction,
  STARTUP_FAILURE/ACTION_REQUIRED bucketing. The Learning-from-Feedback
  mechanics move to references/; the skill drops 869 → ~660 lines.
- list-recent-runs.sh: the completion window anchors on the last
  successful run's start (clamped at 6h), which subsumes the cron-tick
  arithmetic, dropped-tick recovery, and the schedule-event restriction;
  the retry wrapper goes too — a blip now fails loud and a later tick
  retries. 207 → 113 lines, and the first tests for the script.
- review-reviewers: the seven Non-issues carve-outs compress into three
  structural rules (designed no-ops, designed silence, reviewer-role
  independence).

shellcheck now covers plugins/**/*.sh.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Applies the code-review findings on the previous commit:

- poll-pr-checks: more than 100 rollup contexts routes to UNVERIFIED (a
  node dropped past the query's one page could be the failing check); a
  group with any non-terminal check run reads pending, so a nullable
  startedAt can't let a stale settled entry outrank a QUEUED replacement;
  a late API blip no longer discards the pending list the cap report
  names; the header keeps the why-not-mergeStateStatus / --required
  rationale the deleted inline recipe carried.
- poll-rerun-jobs → rerun-failed-jobs: the script owns the rerun and
  selects the new attempt's jobs by run_attempt, so a fast rerun no
  longer reads as "nothing re-queued" and an unregistered one can't
  report the prior attempt's stale conclusions as fresh.
- list-recent-runs: overlapping prefixes no longer double-count a
  workflow's runs.
- review-reviewers: the designed-no-ops bullet distinguishes pre-boot
  gate skips from sessions that boot and exit silently.
- tests: fake_bin and the fake-gh preamble move to tests/__init__.py
  (five copies → one); new cases pin each fix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the mention-gate consolidation case by case against the four gates it replaces (bot review on own PR with a body / with fresh inline comments / reply-only / empty-body approved; bot review on another author's PR; human bare approval) — outward behaviour matches on every one, and the self-approval argument that collapses the #747 gate holds. Ran the suite locally: 458 passed, and shellcheck -S warning is clean on all of plugins/**/*.sh now that the hook covers them.

Three things, one of which I think is a real hole opened by deleting gh_retry — it's a caller-contract gap rather than a script bug, and the fix is one sentence in review-reviewers Step 1 (outside this diff), so I've left it inline rather than as a suggestion. Happy to push it if you'd like it in this PR.

Comment thread plugins/tend-ci-runner/scripts/list-recent-runs.sh
Comment thread plugins/tend-ci-runner/scripts/rerun-failed-jobs.sh Outdated
Comment thread plugins/tend-ci-runner/scripts/poll-pr-checks.sh Outdated
- rerun-failed-jobs and poll-pr-checks address the repo explicitly on the
  two calls that fell back to cwd remote detection, so a non-checkout cwd
  can't masquerade as "the rerun did not take" or drop the moved-head note.
- review-reviewers Step 1 covers a failing list-recent-runs call: retry
  once, then record the window as a coverage gap — the run still concludes
  green, so the next tick anchors past the span (the contract gh_retry's
  deletion left to the caller).
- poll-pr-checks' exit-2 doc names the zero-checks commit (a paths-filtered
  push), whose rollup is also null.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reviewer's two approval residuals: the anchor query was the third call
leaning on cwd remote detection, and none of the --repo additions were
pinned — the substring assertions passed with or without the flag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@max-sixty
max-sixty merged commit 0712d15 into main Aug 13, 2026
14 checks passed
@max-sixty
max-sixty deleted the radical-simplify branch August 13, 2026 10:51
max-sixty added a commit that referenced this pull request Aug 13, 2026
#965 replaced the cron-payload anchoring while this branch was open. On the
post-merge script the only branch is GITHUB_WORKFLOW set vs unset, so a
dispatched run anchors on the previous successful run and clamps 6h back,
warnings included — the now-anchored hour is the fallback for invoking the
script outside Actions. Verified by running the merged tree both ways.

The skill's note shrinks to the part the existing WARNING paragraph doesn't
already cover: scope claims to the window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
max-sixty added a commit that referenced this pull request Aug 13, 2026
…eck (#966)

Drops the `47 * * * *` cron from `review-reviewers`, leaving
`workflow_dispatch`. Its recent output had converged on maintaining its
own evidence machinery, at the fleet's highest per-run cost and a
seven-day quota ceiling that twice took the user-facing workflows down
with it; each adopter's daily `tend-review-runs` already reviews that
repo's runs from the inside and routes bundled-skill defects upstream,
so this removes the direct loop rather than the only one.

The skill is scoped to match: it described itself as hourly and told the
session to analyze "the past hour". It now states the window
`list-recent-runs.sh` actually hands it — floor at the previous
successful run of the workflow, clamped 6h back — which with no cron
makes the clamp the normal case rather than an outage signal, and scopes
its claims to that window. (An earlier revision of this PR described a
cron-payload branch; #965 replaced that logic while this branch was
open, and the text was rebased onto what merges.)

<details><summary>What the sweep caught, and when</summary>

Its unique catches — the ones no inside-out session would have found —
cluster in April–June: rubber-stamped APPROVEs over visibly failing
checks (#667), backslash-backtick corruption in already-published
comment bodies across repos (#359, #405, #424, #562), stale PRs silently
dropping mentions (#391), and the stale/re-anchored approval family
(#798, #821, #884).

The last two weeks are dominated by fixes to its own machinery —
evidence-gist plumbing, survey-window arithmetic, matrix branch naming,
run→output attribution — while per-run cost climbed from $5.31 to
$10.49. That is the shape the cost gate in #960 was written for.

</details>

Re-enable by dispatching it after a change that could move bot behavior
fleet-wide: a release, a harness switch, a model bump — within the hour
of that change, per the window note in the overlay.

> _This was written by Claude Code on behalf of max-sixty_

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
max-sixty added a commit that referenced this pull request Aug 13, 2026
tend-mention's engagement gate decides whether a comment or review
summons a session — a false negative leaves someone un-answered, a false
positive burns a session that can only exit silently. It lived as ~200
lines of bash inlined in `mention.yaml.j2`, where nothing shellchecked
it and the tests could only assert that certain substrings appeared in
the rendered YAML in a certain order. This is the follow-up #965 named
and deferred to keep that diff reviewable.

The gate moves to `generator/src/tend/templates/mention-verify.sh`, the
shape `review-gate.sh` and `notifications-check.sh` already use: env in,
`GITHUB_OUTPUT` out, inlined verbatim into the generated workflow. The
bot name arrives as `$BOT_NAME` instead of a Jinja substitution, which
is what lets the script run outside the renderer.

**The logic is unchanged.** Normalizing comments, quoting and
`$BOT_NAME`, the old inlined body and the new script are the same 119
lines with zero differences.

## Testing

38 tests in `generator/tests/test_mention_verify.py` run the script
against a fake `gh` whose `--jq` goes through real jq, so the gate's own
filters do the reductions: dispatch-payload forgery (non-numeric ids, an
unknown kind, a review or comment the API doesn't have, a comment bound
to another PR), the mention paths, the bot-review handoff gate,
contentless approvals, and the engagement heuristics including a
paginated lookup.

Seven mutations confirm they bite — dropping the fresh-inline narrowing,
un-keying the handoff gate from the review author, dropping
`ascii_downcase`, dropping the self-comment guard, ungating the inline
fetch from the review kind, unguarding the comment fetch, and dropping
an env var from the workflow each fail 1–3 tests.

`test_generate.py` keeps what generation owns and sheds seven shape
tests: the `env:` block is pinned by exact value, plus a scan proving
nothing the script reads is left unset — an unwired name is simply empty
at runtime, so the gate would answer on a blank (a missing
`COMMENT_AUTHOR` reads as "not the bot") on a green job.

Also checked: `shellcheck` covers the new file through the existing
`templates/*.sh` glob, `actionlint` (which shellchecks inlined `run:`
blocks) passes on the generated workflow, and the script ships in the
built wheel.

<details><summary>One thing to watch</summary>

The three engagement `--jq` filters moved from single to double quotes
so `$BOT_NAME` interpolates. A future filter containing `$` or a
backtick would now be shell-expanded rather than passed through to jq.
None does today.

</details>

> _This was written by Claude Code on behalf of max-sixty_
@max-sixty max-sixty mentioned this pull request Aug 13, 2026
max-sixty added a commit that referenced this pull request Aug 13, 2026
Release 0.1.16: the version bump, the lockfile sync, and the
`CHANGELOG.md` section that `pypi-release.yaml` publishes verbatim as
the release notes.

11 commits since 0.1.15. On the shipped surface: a cost gate on
self-initiated fixes (#960), tested CI-poll scripts and an anchor-based
run window (#965), one review-anchor resolver plus the head-resolution
fix past 100 commits (#971), the mention engagement gate as a tested
script (#970), `review-reviewers` paused to manual dispatch (#966), the
stale re-anchored approval on rebased dependency PRs (#890), and a
line-anchored evidence-log selector (#963).

After merge: tag `0.1.16` to publish, then regenerate tend's own
workflows with `uvx tend@latest init` and restamp
`review-reviewers.yaml`, whose action pin `init` never rewrites.

> _This was written by Claude Code on behalf of max-sixty_
max-sixty added a commit that referenced this pull request Aug 13, 2026
Post-release regeneration for 0.1.16. The eight generated workflows move
to the `max-sixty/tend/claude@0.1.16` pin, and `review-reviewers.yaml`
is restamped by hand, since `init` never rewrites it.

`tend-mention` carries real content beyond the pin: the engagement gate
is now inlined from `mention-verify.sh` (#970) rather than rendered from
the template, and the four review-path skip gates collapse into the two
structural rules from #965.

> _This was written by Claude Code on behalf of max-sixty_
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

running-in-ci: CI-monitor poll follows the PR head, so a concurrent push makes a run report another commit's green as its own

2 participants