Unchanged ClawHub issue is re-reviewed hourly and re-rated each pass: rating and needs-live-repro labels flip with identical target SHA and verdict

Summary

An unchanged ClawHub issue is re-reviewed roughly hourly, and every pass re-rates it. Over 14 hours, the durable review comment on openclaw/clawhub#3921 was rewritten 12 times. Every revision reviewed the same target SHA and reached the same verdict, yet the issue rating and clawsweeper:* labels changed back and forth between passes. That includes re-adding clawsweeper:needs-live-repro after an earlier pass had removed it because of the reporter's live reproduction.

There was no reporter or maintainer activity on the item after 2026-10-06 19:49 UTC (last human comment). Every later change came from ClawSweeper itself.

Evidence

Durable review comment: openclaw/clawhub#3921 (comment)

Data per revision comes from the comment's userContentEdits (GraphQL), and labels come from the issue timeline. All times are UTC.

Revision published	Reviewed against	First line	Rating label after the pass	Other label changes
10-06 19:42	`a18bc7496a10`	needs some work	🐚 platinum hermit	+ `needs-live-repro`, `needs-maintainer-review`, `no-new-fix-pr`
10-06 20:56	`a18bc7496a10`	needs some work	🦪 silver shellfish	− `needs-live-repro`
10-06 22:36	`a18bc7496a10`	needs some work	🦪 silver shellfish	+ `not-repro-on-main`, − `no-new-fix-pr`, − `needs-maintainer-review`
10-06 22:40	`a18bc7496a10`	needs some work	🦪 silver shellfish	− `not-repro-on-main`
10-07 00:00	`a18bc7496a10`	needs some work	🦪 silver shellfish	—
10-07 00:06	`a18bc7496a10`	needs some work	🦪 silver shellfish	+ `not-repro-on-main`
10-07 01:15	`a18bc7496a10`	needs some work	🦞 diamond lobster	− `not-repro-on-main`, + `source-repro`
10-07 02:52	`a18bc7496a10`	needs some work	🦞 diamond lobster	—
10-07 04:43	`a18bc7496a10`	needs some work	🦞 diamond lobster	—
10-07 06:53	`a18bc7496a10`	needs some work	🦞 diamond lobster	—
10-07 08:46	`a18bc7496a10`	needs some work	🦪 silver shellfish	− `source-repro`
10-07 09:53	`a18bc7496a10`	needs some work	🐚 platinum hermit	+ `needs-live-repro`

Observations:

All 12 revisions name the same reviewed commit (a18bc7496a10), the same model line (model internal, reasoning medium) and the same verdict. In each one, the "Do we have a high-confidence way to reproduce the issue?" answer starts with "Yes".
Despite that, the rating moved platinum → silver → diamond → silver → platinum, not-repro-on-main was added, removed and added again within 90 minutes, and needs-live-repro returned at 09:53. The live reproduction that led to its removal at 20:56 is still in the issue: Official steipete/peekaboo skill teaches Peekaboo 3 commands removed in 4.0 and shadows the newer bundled skill when installed clawhub#3921 (comment)
Two pairs of passes ran only minutes apart (22:36/22:40 and 00:00/00:06).
For subscribers, each pass also produces a GitHub notification email, apparently from the temporary status comment posted shortly before publication. Email timestamps from the issue author's inbox are 23:58, 00:04, 01:13 and 02:50 UTC, each about 2 minutes before the matching revision. Reading the issue afterwards shows no new comment, only the edited durable one.

Expected

The README Scheduler section says hot/new items are checked hourly, so frequent passes on a new item are expected. However, #1052 (closed 2026-08-08) describes this exact effect as a defect: "unchanged items are re-reviewed and re-rated every pass". Its fix was meant to let scheduled passes reuse the structural review cache when nothing changed. For an item with an unchanged body, comments, reviewed target SHA and verdict, I would expect one of these:

The pass reuses the prior review and leaves labels untouched, or
if a fresh model review runs anyway, rating and reproduction labels are stable for identical evidence, and a removed needs-live-repro is not re-added without new contrary evidence.

Actual

Each pass publishes a fresh review and rewrites rating and reproduction labels non-deterministically. Followers of the item get an email per pass and a label change on many passes, but nothing on the item changed.

Notes

Repository: openclaw/clawhub (the issue is from there; I have not checked whether openclaw/openclaw items show the same churn).
Related closed issues: Scheduled review deliveries cannot reach the structural review cache, so unchanged items are re-reviewed and re-rated every pass #1052 , Scheduled hot intake re-reviews unchanged PRs after owned comment timestamps outrun sync metadata #1006 , ClawSweeper's own review writes expire the review content cache, so unchanged heads are re-reviewed in a loop #967 , ClawSweeper command status comment edits can re-enter activity handling and churn review labels #266
I have not traced which cache decision (activity_changed, target_changed or another) misses here. The reviewed target SHA is identical across all revisions.

Thanks for the context here. I did a careful shell check against current main, and this is already implemented.

The merged repair restores saved-verdict reuse on current main, resolving the missing-record cause of repeated model reviews and label changes.

So I’m closing this as already implemented rather than keeping a duplicate issue open.

Review details

Best possible solution:

Retain pre-generation canonical-record hydration and the existing exact-input reuse safeguards.

Do we have a high-confidence way to reproduce the issue?

The historical missing-record failure is established by the corrected diagnosis and source contract; current main restores the prerequisite, and no live reproduction or post-merge production replay was performed in this review.

Is this the best way to solve the issue?

The merged repair uses the existing focused hydrator at the missing workflow boundary, preserving changed-evidence invalidation and full-review fallback without changing scheduler policy.

Security review:

AGENTS.md: found and applied where relevant.

What I checked:

Merged repair explicitly addresses this issue: GitHub confirms fix(review): unchanged items are re-reviewed and re-rated every scheduled pass #1753 merged at 2026-10-11T23:46:55Z as 69e22d8; its body and commit message explicitly relate the repair to this issue. (.github/workflows/sweep.yml:807, 69e22d843857)
Current main loads the record before generation: The item-scoped setup-state step precedes Review exact event item, uses the same condition, disables git-state and blob hydration, and retains a bounded full-review fallback when hydration fails. (.github/workflows/sweep.yml:804, 796958c8d866)
Canonical record reaches the cache consumer: The focused hydrator reads the selected Worker record, verifies its content digest, and installs records/<slug>/items/<number>.md; generation reads the existing report before probing caches. (scripts/worker-records.ts:459, 796958c8d866)
Reuse preserves the prior verdict: The content-cache branch carries priorReview.markdown and skips the model; structural reuse still rejects changed source, policy, model, target, or uncovered activity. (src/clawsweeper-review-command-workflow.ts:1358, 796958c8d866)
Regression coverage and documentation match the repair: Coverage requires pre-generation item-scoped hydration; a timestamp fixture distinguishes missing_review, reservation-covered reuse, and changed human evidence. docs/review-cache.md describes the restored prerequisite. (test/review-reliability-workflow.test.ts:243, 796958c8d866)
Fix provenance: Merged history identifies the hydration repair in commit 69e22d8; authenticated commit metadata confirms its date and changed workflow, tests, docs, and changelog fragment. Missing historical blobs prevented local git show, so GitHub commit metadata supplied that inspection. (.github/workflows/sweep.yml:807, 69e22d843857)

Codex review notes: model internal, reasoning medium; reviewed against 796958c8d866; fix evidence: commit 69e22d843857, main fix timestamp 2026-10-11T23:46:55Z.

The public Actions logs give the miss diagnostics the review asked for. Two consecutive scheduled passes on openclaw/clawhub#3921, job "Review exact event item":

Run	`CLAIM_DECISION.sourceAction`	`sourceUpdatedAt`	Review counters
https://github.com/openclaw/clawsweeper/actions/runs/37595639676 (08:43 UTC, "rev 19")	`scheduled_normal_backfill`	`2026-10-07T06:53:32Z`	`structural_cache_checks=1 structural_cache_hits=0 structural_cache_revalidations=0 content_cache_hits=0 hydrations=1`
https://github.com/openclaw/clawsweeper/actions/runs/37603325532 (09:50 UTC)	`scheduled_normal_backfill`	`2026-10-07T08:46:46Z`	`structural_cache_checks=1 structural_cache_hits=0 structural_cache_revalidations=0 content_cache_hits=0 hydrations=1`

In both passes, sourceUpdatedAt falls within seconds of ClawSweeper's own write in the previous pass: the durable comment was edited at 06:53:26 and 08:46:40, and labels were written at 08:46:32. No human activity on the item after 2026-10-06 19:49 UTC. Each pass reports zero structural and zero content cache hits and a full hydration.

So for this item, the item's "last activity" is ClawSweeper's own previous publication. That matches the activity_changed condition described in #1052. I have not traced the code path, so the link between that timestamp and the cache miss or hot cadence is an inference from the logs.

The run titles suggest this is not limited to one item. Scheduled-normal runs for other ClawHub items in the same 08:40–08:47 UTC window show high revision counts:

openclaw/clawhub#3888 rev 178: issue opened 2026-10-01, 1 comment
openclaw/clawhub#3548 rev 63: PR opened 2026-08-30
openclaw/clawhub#3856 rev 55, #3863 rev 44, #3892 rev 41, #3868 rev 38

I have not checked whether those revisions also changed labels.

Follow-up: I checked the points left open above, at 34cc1aa014a1 (current main).

1. Other ClawHub items show the same churn

Counted from GraphQL userContentEdits on the durable review comment and from issue timeline label events by clawsweeper[bot]. A "flip" is a label removed after being added, or added again after being removed.

Item	Opened	Durable comment rewrites	Label events / flips	Rating sequence
openclaw/clawhub#3888	10-01	65 (10–14 per day since 10-02)	90 / 81	24 rating changes between silver, gold and platinum
openclaw/clawhub#3548	08-30	100 (6–9 per day since 10-03)	35 / 25	platinum → gold → silver → gold → platinum → gold → platinum
openclaw/clawhub#3868	09-30	39	14 / 8	platinum → silver → platinum
openclaw/clawhub#3892	10-01	48	7 / 3	stable
openclaw/clawhub#3856	09-30	52	6 / 1	stable
openclaw/clawhub#3863	09-30	47	4 / 0	stable

PR 3548 is 38 days old. Per docs/scheduler.md ("pull requests outside the hot window: daily"), it should be reviewed about once a day, but it is rewritten 6–9 times a day.

For comparison, six of our items in openclaw/openclaw (opened 10-02 to 10-06) have 0–7 rewrites each. Their rating changes coincide with new contributor evidence. So the churn looks specific to the ClawHub lane.

2. Likely cause: the lease-comment release moves `updated_at` after the recorded owned syncs

For the two 3921 passes in the previous comment, sourceUpdatedAt is 6 seconds after the durable comment edit of the prior pass (06:53:26 → 06:53:32, 08:46:40 → 08:46:46). The labels were synced earlier (08:46:32). docs/scheduler.md (paragraph "Exact PR review marks …") says the producer syncs the durable comment, edits labels, and then "releases (deletes) its review lease comment". Deleting that comment is a GitHub write that advances updated_at. I did not verify the deletion time directly, because deleted comments are not in the timeline. The constant 6-second offset is consistent with it.

Both owned-activity guards compare against markers that are older than that final write:

Cadence: docs/scheduler.md ("Review cadence") ignores owned writes only "if GitHub updated_at is at or before" review_comment_synced_at or labels_synced_at. An updated_at from the later lease deletion is after both, so the item counts as having "target-side activity since the last real review: hourly". The next pass repeats the same writes, so the item never leaves the hourly bucket. That explains 3548 staying near hourly at 38 days old.
Structural cache: activityCoveredByReview (src/review-structural-cache.ts:1173) accepts the newer activity only if currentActivity <= max(reviewCommentSyncedAt, labelsSyncedAt, ownedReservationUpdatedAt). ownedReservationUpdatedAt is the reservation, not its release, so the release timestamp fails the check, and the pass misses with activity_changed. This is consistent with structural_cache_hits=0 in both runs.
Scheduled no-op: classifyScheduledReviewNoop (src/scheduled-review-noop.ts:114) does recognize "trusted owned activity only" within a settling window, but it returns early for any sourceAction !== "scheduled_hot_intake". Both 3921 runs were scheduled_normal_backfill, so this guard never applies to them.

The "Apply freshness" rule in the same doc already accepts a release-shaped update ("the review generation's own lease comment is gone … the latest update follows the generation's last recorded ClawSweeper write by at most five minutes"). The cadence planner and the structural cache don't seem to use an equivalent rule.

This is a reading of the code and docs plus the run logs. I have not run ClawSweeper locally. I also have not found why the openclaw/openclaw lane is less affected.

Correction to my previous comment: the lease-comment release is not the cause. I traced it locally against 34cc1aa014a1.

Actual cause. The review step reads the item's prior canonical record from records/<slug>/items/<number>.md (src/clawsweeper-review-command-workflow.ts:696). In the event-review-apply job, that record is hydrated only after "Review exact event item", by direct-setup-state for publication. Every scheduled pass therefore starts without a prior review. The structural cache stops at missing_review, and the content cache has no verdict to compare.

In https://github.com/openclaw/clawsweeper/actions/runs/37603325532 the review ran 09:50:52–09:53:09 with structural_cache_hits=0 structural_cache_revalidations=0 content_cache_hits=0 hydrations=1, and the record was hydrated afterwards at 09:53:15 (focused tuples hydrated repo=openclaw-clawhub items=1 records=1). The same counters appear in all 40 scheduled-normal runs sampled on 2026-10-07 10:28–11:02 UTC, 32 of which were openclaw/openclaw items. So the cache miss is not ClawHub-specific.

What I got wrong about cadence. docs/scheduler.md still describes the sync-marker exemption, but since #1007 hasActivitySinceReview (src/scheduler-policy.ts:85) treats any update after reviewed_at as activity and relies on the structural cache to suppress the review. The lease release only adds one more such update; it doesn't change the outcome. In activityCoveredByReview, the next pass's own reservation covers the release timestamp, so it would not cause activity_changed once a record is present.

Why ClawHub items show it so much more (inference, not measured in the planner): with 99 open items, capacity is enough to re-review every "active" item hourly. With roughly 9,400 open items in openclaw/openclaw, a single item comes up far less often.

A fix that hydrates the selected item's record before generation is in #1753 . The hourly cadence and the per-pass lease comment are unchanged there.

Update after 48 hours, two observations that go beyond the original report. All times UTC.

1. The churn continues after the item should have left the hot window. The README Scheduler section says hot/new items are checked hourly and that items younger than 30 days are checked daily once they leave the hot window. On openclaw/clawhub#3921 the durable review comment now has 30 revisions between 10-06 19:42 and 10-08 19:35, 13 of them in the last 24 hours (gaps between 52 and 282 minutes, median about 109). There has still been no reporter or maintainer activity since 10-06 19:49.

	At report time (10-07 09:53)	Now (10-08 19:35)
Revisions of the durable comment	12	30
Revisions with the same reviewed target SHA and the same `source_revision` as the previous one	10	24
Label events on the issue	24	46

The other six are the first review, the pass after the reporter's reproduction comment, and four passes that coincide with new main commits (target SHA a18bc7496a10 → c23e34ad353d → 5218bd885833 → 5ae1941bb1a7 → d044664a7636). All 30 reach the same verdict ("this still needs some work", reproduction answer "Yes"). The rating label kept moving after the report: 🐚 platinum hermit → 🦪 silver shellfish (10-07 11:34) → 🦞 diamond lobster (10-07 20:32) → 🦪 silver shellfish (10-08 11:38) → 🦞 diamond lobster (10-08 17:36), and clawsweeper:not-repro-on-main was added and removed three more times.

I have not found the exact length of the hot window in docs/scheduler.md, so this is an observation against the README wording, not a confirmed scheduler defect.

2. It also happens in this repository. The durable review comment on this issue (#1752 (comment)) has 9 revisions. Seven of them came after the last comment here (10-07 12:26), the latest three at 10-08 04:38, 10:35 and 15:51. Here the labels have stayed stable since 10-07 13:12, so the effect is repeated review work rather than label churn.

Data source: userContentEdits of the durable comments (GraphQL) and the issue timelines.

improve(review): count only completed issues out of the bulk-filer th…

improve(review): leave completed issues out of the bulk-filer count (#…

fix(review): unchanged items are re-reviewed and re-rated every sched…

Navigation Menu

Repository navigation

Unchanged ClawHub issue is re-reviewed hourly and re-rated each pass: rating and needs-live-repro labels flip with identical target SHA and verdict #1752

Summary

Evidence

Expected

Actual

Notes

1. Other ClawHub items show the same churn

2. Likely cause: the lease-comment release moves `updated_at` after the recorded owned syncs

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Unchanged ClawHub issue is re-reviewed hourly and re-rated each pass: rating and needs-live-repro labels flip with identical target SHA and verdict #1752

Description

Summary

Evidence

Expected

Actual

Notes

Activity

clawsweeper commented on Oct 7, 2026

holgergruenhagen commented on Oct 7, 2026

holgergruenhagen commented on Oct 7, 2026

1. Other ClawHub items show the same churn

2. Likely cause: the lease-comment release moves updated_at after the recorded owned syncs

holgergruenhagen commented on Oct 7, 2026

holgergruenhagen commented on Oct 8, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions

2. Likely cause: the lease-comment release moves `updated_at` after the recorded owned syncs