docs(skills): daily-data-sweep — monthly complete + exhaustive data sweep - #1
Draft
maxchampoux wants to merge 25 commits into
Draft
docs(skills): daily-data-sweep — monthly complete + exhaustive data sweep#1maxchampoux wants to merge 25 commits into
maxchampoux wants to merge 25 commits into
Conversation
Defines a monthly data-quality sweep over Well's MCP financial graph, separating two failure classes that route to different fixes: - COMPLETE (depth): the records that exist are whole, verified, fresh, reconciled. Fails as "right scope, wrong number". - EXHAUSTIVE (breadth): nothing is missing — no unconnected source, no gap period, no uncovered entity. Fails as "right number, wrong scope". Structure: - PART A: the sweep unit is (workspace x month), entered through the existing month-grid picker re-labelled for sweep semantics, with its own four-state vocabulary and findings-as-dots. Close-readiness is a cross-check, not the driver. - PART B: an iteration protocol so a control point can prove it examined every object rather than a sample — page-to-exhaustion, fan-out-per- object, and converge loops, with four-valued verdicts (pass / fail / inconclusive / sampled) and a per-iteration ledger. - PART C: a ten-gate execution order where a red re-labels later gates rather than halting them. Control families: BANK, BOOK, IPAY, DOC, EINV, ING, GRAPH, RECON, plus CMP/EXH and an ASSUME table of preconditions the 12 published skills rely on but never verify. Every count is scope-tagged and every ungrounded field is recorded under Known limits rather than asserted. Sections that need a schema change to become buildable say so instead of implying coverage. Draft: distribution artifacts (docs/, dist/, plugin manifests, symlinks) are deliberately omitted until the control-point list is final. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Splits a 1,554-line single file into a 377-line executable SKILL.md plus 14 progressive-disclosure references, and closes six validation failures against the skill-forge master standard. Body size — the master mandates progressive disclosure and "understood by a human in under two minutes". SKILL.md keeps only what an agent needs to execute: the COMPLETE/EXHAUSTIVE definitions, the (workspace x month) sweep unit and its four states, the loop rules, the ten-gate order, and a File layout table that says when to open each reference. Every control point table, the latent-assumption audit, the MCP surface limits, the tolerances and the dated measurements move to references/. Placeholders — all `[TO COMPLETE]` slots are resolved rather than deferred, so the skill no longer behaves as a draft reporting its own open questions. The category-confidence threshold is wired to the value tolerance 10 already specifies (0.70). The open-ended "add more control points" slots become an extension contract. The pending per-line VAT/FX work becomes a named not-covered entry: those checks report INCONCLUSIVE with the reason "no field exists", which is the sweep's own rule that an unexpressible check is named, never silently dropped. Well MCP standard — adds the four required elements that were absent: the OAuth/DCR branch with retry-in-the-same-turn, the connector sufficiency gate (filtering the set of non-enabled statuses, not one value), transient-failure retry-once, and the optional compliance line. The workspace-link fallback carries no unverified query parameter. Discovery — the description now leads with the job rather than the schedule, and drops trigger phrases that collided with missing-receipts and the close flow. Examples — three worked examples replace the placeholder: a clean sweep that still states its scope, a mixed sweep where a breadth failure scopes the depth findings, and a sweep that could not run and must not read as clean. Losslessness verified mechanically: 189 distinct control-point ids and 204 table rows before and after, none lost. Session-authoring narrative and the team backlog are removed; the durable schema facts they carried survive in references/schema-facts.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
maxchampoux
changed the title
First live execution and three domain reviews contradicted assumptions the skill was written on. Corrections, in order of how badly each would have corrupted a real run. The iteration protocol was unimplementable. The MCP has no pagination: nextCursor is always null, and there is no offset. Measured on transactions where category_key IS NULL — totalCount 793, returned 545, nextCursor null; retested at limit 5, returned 10, nextCursor null. So examined == total is unreachable for any root larger than one response, and the page-to-exhaustion loop can never terminate. The binding rule is now: returned < totalCount yields SAMPLED, never pass. Three further measured contradictions of the documented contract: limit is applied per workspace rather than globally (limit 5 returned 10 rows across two workspaces), totalCount counts across all workspaces while rows are capped per workspace (two scopes in one response — the mixing error this sweep exists to catch), and field selection does not control payload because composite columns inject automatically at roughly 1.2 KB per row (a 500-row request exceeded the response limit outright). The working idiom is to count with limit 1 and read totalCount, fetching rows only for example ids. Connector status filtering was wrong in both directions. status is a stored, filterable enum with eight members, not seven — degraded was added 2026-07-08 and is deployed; a checkout forked before that date shows seven. Filtering on "not enabled" cries wolf on to_configure and disabled while treating processing as broken. Replaced with the same three-way bucket the close flow uses, so the sweep and the close cannot disagree. Also: degraded needs no user action and clears itself on the next successful sync, so its remediation is now worded as investigate, never reconnect. Recorded the two connector-status vocabularies that must not be compared: well_query_records returns the raw stored value, while well_list_connectors returns a display value resolved server-side to enabled, processing or error. Removed session-authoring narrative from the reference set (21 rewrites across 8 files) so an executing agent reads facts rather than the history of a previous draft, and corrected a control-point row that still claimed a schema read was pending after the field was confirmed to exist. Losslessness verified: 189 distinct control-point ids before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A dry-run executability audit walked the skill gate 0-9 as the executing agent and found its honesty rules were written in prose but never wired into the path that computes a verdict. Four fixes close that path; the remaining nineteen findings are recorded rather than left implicit. The L1 primitive returned pass on an empty population. It did not distinguish an empty hit set from an empty population, so a workspace with zero cards passed all five card checks and a month with no transactions passed every reconciliation check. That is the exact failure this skill exists to prevent, encoded in its own primitive. total == 0 now yields INCONCLUSIVE unless a control point explicitly declares empty_is_pass, which is correct only where absence is the passing state. The worked example taught the opposite of the rule. It printed a bare COMPLETE: pass while three control points were never evaluated, listing them below the verdict where a reader takes the headline and not the footnote. An agent copies the example, not the prose. Corrected to pass (partial - 3 not evaluated). How red and amber counts aggregate into a bucket verdict was never stated, leaving the common amber-only case undefined. The rule is now explicit: any red fails the bucket, any amber yields pass (with findings), and any INCONCLUSIVE or SAMPLED appends (partial - N not evaluated). A bare pass is legal only when every control point in the bucket passed over a fully examined population. The severity floor is a reporting filter and never changes a verdict. PAGE_BUDGET had no value anywhere while being the sole determinant of pass versus SAMPLED, so two agents would produce different verdicts on identical data. Set to 20 pages per control point. The nineteen open findings are recorded in references/known-issues.md and linked from the File layout table with the instruction to read it before trusting any verdict. Gates 1, 6, 7 and 8 are not fully runnable as written; the file names why for each. Recording them follows the skill's own rule that a check it cannot make is named, never silently dropped. Losslessness verified: 189 distinct control-point ids before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four domain reviews plus one live probe. The control-point set is now 216 ids across 10 families, every row declares its bucket, and no row claims a severity it has no ground to claim. Gates 7 and 8 were unrunnable and now run. IPAY- had zero enumerable control points and EINV- had two; both are now six-column tables with a per-row bucket — IPAY-01..16 and EINV-01..36, of which 13 are marked blocked with fail signal "INCONCLUSIVE, no field exists". Naming an unbuildable check rather than omitting it is the skill's own rule; an absent row reads as checked and clean. The CMP-/EXH- layer is deleted. All 32 of its ids map to a family equivalent or were moved, three of them to ING- because the family tables could not see what they saw: cross-connector account duplicates, a connector with zero sync logs of any status as distinct from zero successful ones, and a recommended-but-unconnected source. Eight duplicate control points removed, each with a named canonical. The largest bundled five predicates into a single red over one population and was the main driver of a headline red count inflated roughly fourfold. Contradicting thresholds are resolved rather than averaged: sync staleness at 26h red, hung sync aligned to the tolerance that already governed it, one canonical uncategorized ladder, and the confidence floor at 0.70 everywhere. Every run is cold. The MCP is read-only with no output root, so no prior-sweep store exists or can exist. The diff requirement, SPINE-05 and BOOK-proof-lost-trend are deleted and the deletions recorded; two checks were salvaged to their single-run arm rather than dropped. A permanent INCONCLUSIVE was rejected: it is indistinguishable from a disabled check and it inflates the not-evaluated count that stamps a bucket verdict partial, making a fully-checked month read as degraded. Dated measurements no longer masquerade as current. The duplicated audit tables are gone from the family files and every measured figure is out of every fail-signal cell; a fail signal is a condition, never an observation. The one figure that existed only in a family file was moved to the baseline rather than dropped. Five rows carried a severity while being unevaluable and are now blocked, including a twin of the reported case that the review found on its own. Five further specs were wrong in ways that would have produced silent wrong answers, the sharpest being a check specified as a join to a root that has no such column: an agent following it emits an empty join and reports zero defects. Relation predicates work. A live probe returned 213 filtering invoices on document.size, exactly the invoices that have a document, so the "no cross-root join" limit was over-conservative and the DOC- family, including the DOC-07 cross-workspace security control, is executable. The real limit is no aggregation over a related set. Window capped at six months, gate 0 no longer tests a condition only knowable at gate 9, L3 restricted to the month chain, and the tolerance table is normative with literal defaults rather than proposals an agent could not apply without committing a documented violation. Four findings remain open and are recorded in references/known-issues.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The counting primitive was stated in two files. iteration-protocol.md carries the measured contract — no pagination, per-workspace limit, mixed-scope totalCount — so the copy in the ingestion family file is replaced by a pointer. Two statements of the same rule drift; one does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Closes #8 (SPINE-02/03 assumed a nonexistent CloseReadinessStatus MCP root; SPINE-02 is now a self-consistency check, SPINE-03 is blocked/INCONCLUSIVE), #18 (schema pass now runs before the first query instead of after), #14 (SPINE-/ASSUME- tables were missing the bucket column), #19 (soft-delete verdict contradiction between mcp-surface-limits.md and graph-recon.md reconciled against the file's own later live-probe correction), and #22 (records_url now has an explicit never-fabricate construction rule). Also reconciles known-issues.md against the actual file state: several findings (#9, #11, #12, #16, #17, #20, #21) were already fixed by earlier commits but never struck through. Only #4 (cross-file BOOK-/GRAPH-/DOC- duplication) remains open.
The skill existed only as skills/daily-data-sweep/SKILL.md, invisible
to any host that discovers skills via .agents/skills/, undocumented,
and not installable from the README or Claude Desktop.
- Add .agents/skills/daily-data-sweep symlink (Codex/OpenCode discovery)
- Add docs/daily-data-sweep.md, matching the existing per-skill doc shape
- Build dist/daily-data-sweep.{zip,skill}
- List it in all four README.md install surfaces: the Available skills
table, the AI-assisted fetch list, the Claude Desktop download table,
and the npx skills add list
- Add data-quality/audit keywords to both plugin manifests
Every other skill has a symlink under both .claude/skills/ and .agents/skills/. daily-data-sweep had only the .agents/ one, so the Claude Code plugin path could not discover it. Verified against the sibling convention: a relative symlink to ../../skills/<name>, not a copy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An independent pass re-verified every claimed closure instead of trusting the strikethroughs, and found two that would have made a live run lie. empty_is_pass was declared by no control point. The rule that an empty population is never a pass landed in the L1 primitive and never landed in the roughly thirty rows whose passing state IS an empty result set: every duplicate, collision, dangling, orphan and tenancy check. A clean workspace therefore returned INCONCLUSIVE on all of them, and the bucket rule stamped partial on every run forever, making a bare pass unreachable. A verdict that is always partial carries the same zero information as the always-pass it replaced. The class is now declared, together with the inverse class that must not declare it: a check asserting something should exist is INCONCLUSIVE on an empty population, never a pass. The own-company fact contradicted itself across four files. schema-facts, which the skill designates authoritative, says own_company_pk exists and is stripped by the API formatter, and that a company row carrying company_origin counterparty is not a defect. Two other files said the field does not exist and the data is wrong. An agent firing ASSUME-own-company-correct would have emitted a red against the skill's own reference file. Both are reframed to the authoritative wording. Nine further defects are recorded rather than fixed, three of them threshold twins where a value was changed in one file and not its duplicate in another. The re-audit's verdict stands: do not merge until those are closed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Eight ids removed across the graph, reconciliation, bookkeeping and document families, each against a named canonical. Edge properties — currency, dangling side, full-allocation amount, over-allocation, and match confidence — consolidate into the proof chain, on the rule that an edge property is owned by the chain that proves the payment. The null-document_pk gap consolidates into the invoice-side check. The dangling-document_pk case was a three-way, not a pair, and consolidates into DOC-02, which alone carries the live relation predicate and the B term of the two-direction bound formula. Every deletion leaves a cross-reference naming the canonical and any threshold the deleted row carried that the survivor did not: the expensive-paging marker on over-allocation, the 0.5% cross-currency tolerance, the dangling-rate arm, the ten-business-day provisional age gate, and the class-dependent receipt grace ladder. A superseded threshold cannot now be reinstated by accident. One suspected pair was verified NOT a duplicate and hardened instead. Category confidence on transactions and match confidence on invoice_transactions share a tolerance and look alike; they watch different populations at different floors. Both rows now name their population in the title and carry a do-not-merge clause. Also fixes two rows whose name and bucket cells were swapped, carrying a seventh column where the table has six, so they rendered shifted; and removes two ids from the empty_is_pass list that this consolidation deleted, both of whose canonicals were already listed. Finding 4 stays REDUCED rather than closed: two orphan and tenancy pairs remain unadjudicated and are named. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two adversarial re-audit passes raised R1-R11 on top of the original 23 dry-run findings. This closes the nine that were still open. Deduplication (R3, R4). Five control points applied a canonical's predicate to the canonical's population under a second id, and three of those five carried a *different* threshold than the canonical -- so which file the agent read decided the verdict. BANK-txn-categorized, BANK-txn-ledger-mapped, BANK-txn-no-external-id-dup, BOOK-document-right-kind and BOOK-document-tenancy are removed against a named canonical, each leaving a cross-reference that records the superseded threshold so it cannot be re-added by accident. The orphan pair was a superset/subset, not an equality: DOC-08 filtered to receipt-capable MIME above 1KiB, GRAPH-orphan-documents-and-media covered documents *and* media unfiltered. Merging either direction would have dropped rows, so the split is by root instead -- DOC-08 owns orphan documents in two arms, GRAPH-orphan-media owns media. Branding assets no longer inflate the attachment error rate. Dead pagination (R11). nextCursor is null on every response at every limit, so the page-to-exhaustion loop was unimplementable. The loop, its budget and its resume point are gone; a truncated scan now reports SAMPLED rather than INCONCLUSIVE, which is a different verdict on nearly every large root. Six twin sentences taught the cursor loop elsewhere, including two worked examples showing a four-page exhaustive scan of a root that returns ~545 rows at most. Consistency (R5, R7, R8, R9, R10). Gate 1 names its control points as an explicit list -- a range kept the deleted SPINE-05 alive and hid SPINE-07. The records_url example demonstrates the omission rule instead of contradicting it. The no-cross-root-join limit is corrected against a live probe: filtering on a related field is one query, aggregating over a related set is not. Dated live measurements point at baseline-2026-07-29.md rather than inlining figures that decay. Family headings state their real gate. known-issues.md is recounted from its own rows, and its two tables no longer declare more columns than their rows carry -- the defect R6 was raised for.
The pagination rules were already right -- a control point may only return a bare pass when returned == totalCount in one response -- but the consequence was written down nowhere. A reader following the bucket rule would keep hunting for a bare pass on transactions or invoices, never find one, and conclude the sweep was broken. SKILL.md now says plainly that on a root bigger than one response the absence of a bare pass IS the honest result. Strikes the last live paging promise (mcp-surface-limits.md told the reader to re-test the [EXPENSIVE] RECON- caveats "before budgeting it as a paged job" -- there is no paged job to budget), and reconciles three stale twins in known-issues.md that the recounted header contradicted, including a standing instruction to set PAGE_BUDGET to 20 that R11 had already deleted. R1 and R2 were unnumbered, so the header's "34 findings" could not actually be recounted from the rows. Both are labelled now; the tally is checkable. Records the rule the renames exposed: the empty_is_pass enumeration in iteration-protocol.md B.1 is the only live instruction list outside the family files that names control points individually, so any future rename must sweep it. Deleted ids inside fold-in notes are deliberate history, not dangling -- stated explicitly so the next auditor does not "fix" them.
A third adversarial audit against skill-forge §6 scored 19 pass / 5 fail and raised nine findings. The Well MCP standard block came back clean; every surviving defect was stale load-bearing prose or plain absence -- which is what tends to outlive a long series of fixes. The blocker was the one line the file-layout table forces an executing agent to read *before trusting any verdict*, and it was false twice: it advertised an open duplication finding that had been closed two commits earlier, and it said "nine gates" where the gate table defines ten (0 through 9). A mandatory pre-verdict line that reports phantom problems makes an accurate verdict get discounted. Two findings are pure absence -- required by the standard, never written: Visual output. The skill produced a per-control-point scorecard and never offered to render it. It now offers a grouped bar of red/amber counts per gate, and says plainly that a month-over-month trend is unavailable because every run is cold -- a chart implying one would misrepresent the sweep's own memory. The handoff. This skill is a pre-flight that is forbidden from answering the financial question behind it, so it dead-ends by construction and the next question is guaranteed. It now names the sibling that answers it -- the passing case hands off to cash-position or runway-calculator, the failing case to whichever sibling owns the specific gap. Also: the einvoicing family was skipped by the undated-measurement pass, so its prod-wide table now carries its date and the decay pointer the other families got; a dedup note counted four rows and named five; SKILL.md asserted an eight-value connector-status enum as settled where schema-facts records seven native plus degraded observed live; and a live instruction named DOC-01, which survives only through a redirect note.
…hable An executability audit walked the skill gate 0-9 as the executing agent and found the core loop could not produce a correct verdict for any control point on any non-empty root. Both blockers were introduced by the no-pagination fix that known-issues.md certifies as closed. The counting idiom and the truncation rule contradicted each other. B.1 told the agent to count with limit:1 and read totalCount -- so returned is 1 by construction -- and then to return SAMPLED whenever examined < total. Every root with more than one row therefore short-circuited to SAMPLED before the pass/fail branch was reachable. The sweep degraded to "(partial)" with a permanently empty red table, and a server-side-filtered count of 793 uncategorized transactions -- an exact, exhaustive answer -- surfaced as pass (partial) instead of a red. L1 now branches on query shape first. A defect predicate expressible in the whereClause makes totalCount the finding: exact, routed to fail, with only the example ids sampled. SAMPLED is reserved for predicates that need a client-side reduce over returned rows, and a hit found in a partial scan is still a real defect -- only absence over a partial scan proves nothing. An empty result meant two things and needed three. total == 0 was intercepted as INCONCLUSIVE-or-pass before the control point's own fail signal was read, which silently killed a dozen absence-detection reds -- a connector that has never synced, a bank connector with zero accounts, an entity class silently empty, a period with transactions and no entries. empty_is_fail is now a declared class alongside empty_is_pass, and both are closed enumerated sets. Membership was decided by substring-matching the control point's name. That matched ING-connector-never-synced on the word "never", whose fail signal is literally totalCount = 0 -- making its pass condition identical to its fail condition, so a bank that had never once synced would report pass while cash-position totalled a confident zero. It is also the string-matching the repo standard forbids. Ten absence-is-pass rows were missing from the list and one presence-assertion row was wrongly in it. Also: one duplicated bill was counted three times by three live ids in two different buckets -- consolidated onto EINV-02, the only one anchored on workspace_pk. The fx-rate check needed a home currency the surface does not carry and would have fired an unearned red on it. The last live "page" instruction is gone. Both worked examples now agree with the rules they illustrate: the partial suffixes partition across buckets instead of printing one figure twice, and SAMPLED counts toward not-evaluated.
The baseline stated ING- 35 / GRAPH- 23 / RECON- 26 against actual counts of 37 / 16 / 19 -- numbers that drift on every dedup pass and were never the baseline's job to carry. The families own their own totals. Also marks the soft-delete question resolved. It was recorded INCONCLUSIVE on 2026-07-29 and settled by a later live probe, but a reader reaching the baseline first would have believed five GRAPH- checks were still blocked.
…nstall known-issues.md was 18KB of pure fix history gated as mandatory reading before trusting any verdict -- 7% of the reference corpus and 0% instruction, and no workflow step sent anyone there. Renamed CHANGELOG.md and demoted to what it is: read it to learn why a rule is shaped the way it is, never before a run. Its one still-operative rule (a rename must sweep the empty_is_pass / empty_is_fail enumerations) already lives in iteration-protocol.md B.1 next to the lists it protects, so it is not duplicated here. SKILL.md's Purpose carried a 45-line definitional essay; the two definitions stay verbatim in the body and the bucketing test, symptoms and worked examples move to references/complete-vs-exhaustive.md. The no-pagination measurement no longer duplicates iteration-protocol.md B.1. Condensing that block reintroduced the blocker fixed one commit earlier: it told the agent that returned < totalCount is always SAMPLED and never pass, which is true of a truncated scan and false of a limit:1 count, where returned is 1 by construction. Both the step-5 instruction and the concurrent- sync edge case now branch on query shape -- a server-side totalCount is exact and routes to fail with the count stated; SAMPLED is for a client-side reduce that found nothing. The README's AI-assisted install copied a lone SKILL.md, so this skill arrived pointing at a references/ tree that never came -- failing silently mid-run rather than at install. Directory-form install now enumerates references/ from the GitHub contents API, verifies it is non-empty before reporting success, and falls back to the .skill archive where an agent cannot write nested directories. The twelve single-file skills are unchanged.
The published archives still carried known-issues.md, so a .skill install landed the old filename while SKILL.md pointed at CHANGELOG.md -- a broken reference on a fresh install.
First live execution of the rewritten L1 primitive against a real workspace. It validated the fix and surfaced a worse defect than the one it was testing. Validated: a server-side filtered count returns totalCount 795 with returned 2 at limit 1. Under the rule as it stood this morning that was SAMPLED and the sweep reported pass (partial); under the corrected primitive it is a fail with an exact count. The blocker is genuinely closed, and 795 against yesterday's 793 is the decay the baseline warns about. Found: an invalid field name returns success with totalCount 0 in the fan-out path this skill uses, while the same query with an explicit workspace_id returns a hard 400. So a typo, a renamed upstream column, or a field a connector does not populate produces a successful empty response that is indistinguishable from a clean workspace -- and the empty_is_pass class turns it into a green. A whole family could report clean because one column was renamed upstream. empty_verdict now checks schema validation before anything else and outranks empty_is_pass: an empty result is only evidence if the fields were validated against well_get_schema first. That is what makes step 3's schema pass load-bearing rather than hygiene. Also corrects records_url against measured behaviour. The rule said no confirmed URL pattern exists so the field must be omitted -- but the API returns records_url in its own response. Pass it through verbatim; still never construct one, and do not append filters or ids, since the returned URL points at the root's records view rather than a filtered set.
Caught by a second sweep run that reported a non-null nextCursor, contradicting this skill's own baseline. Re-probed: the second run is right. well_query_records paginates when the query is scoped to one workspace. A limit:500 call with an explicit workspace_id returned 500 of totalCount 1597 with a non-null cursor; passing that cursor back advanced the window and issued another. Pagination is fully live. Every probe behind the original claim ran in the multi-workspace fan-out path with small limits, where returned never reaches limit, so no cursor is ever issued. That is a property of one call shape, and it was mistaken for a property of the API -- then propagated into the loop primitive, which deleted the cursor loop outright and made SAMPLED the permanent verdict on every large root. L1 pages to exhaustion again, with PAGE_BUDGET restored and defined (20 pages x 500 rows) as a runaway backstop rather than the expected stop -- hitting it is SAMPLED and must be reported. Absence over a budget-truncated scan is SAMPLED; a hit is a fail however far the scan got; a scan that reaches a null cursor is a genuine pass. The binding rule this produces is the useful part: always enumerate workspaces and pass workspace_id on every query. The fan-out caps rows per workspace, scopes totalCount differently from its rows, and cannot be paged -- so it can only ever produce SAMPLED. Scoping per workspace is what makes an exhaustive scan, and therefore an honest pass, reachable at all.
Nine agents, each hypothesis passed through an adversarial refutation step. Three of the four reds are unbuilt or half-built capability; one is live broken behaviour; one is also a defect in the sweep's own control point. Records the finding that BOOK-invoice-has-document counts document_pk IS NULL globally with no join through Connector.data_domains, so it reports an unbuilt capability as an 86% compliance breach. The sweep's own documentation already forbids publishing that term alone.
…rification failures Seven agents with an adversarial refutation pass, which overturned the balance-verification hypothesis entirely. 63 of 66 accounts are Mercury counterparty payees stamped ownership=WORKSPACE and admitted into every canvas KPI scope. mercury-mcp has no generator config and no mcps/ shim, so every file-backed governor on the account target is a structural no-op and tool choice falls to the LLM jury. The 8 failing balance rows are not a pending-float artefact. On first sync a balance row is seeded, then the entire ~24-month backfill is FK'd into it, so verification compares a seconds-long delta against the whole transaction history. One permanently-failing row per connected Plaid account, re-failing on every scheduler run forever -- and cash-position sums them at face value because it never reads verified_at. Records the landmine: cross-connector account dedup does not exist anywhere. Cash is not double-counted today only because mercury-mcp does not sync account_balance. Enabling that target without a cross-rail dedup key would double-count every dual-rail Mercury account.
FR runs the same code against different connectors and is clean where US is broken: 3 accounts all with a currency (vs 63 of 66 null), 14.6% uncategorized (vs 46.9%), 3.5% never-attempted (vs 38.1%). FR has no Mercury MCP rail. That corroborates from data what the Mercury root-cause concluded from code -- the counterparty ingestion and currency-null defects are rail-specific, not general. The one red that IS shared is ledger_account_pk at 100% in both, which is consistent with an unbuilt writer no connector can populate. Notes the shape worth investigating: 671 invoices against 314 transactions and 3 accounts, so most invoices cannot have a matching payment in Well. Records gates 7-8 as not run, which is the most material omission for the French entity since e-invoicing compliance lives there.
… arm The root-cause pass refuted this morning's FR sweep headline. FR is not healthier than US; it is differently broken and the sweep could not see it. Its 314 transactions are 157 ingested twice (Qonto MCP + plaid_qonto_fr), so every per-transaction percentage carried a doubled denominator. 120 of 671 invoices are cross-source Pennylane/Qonto duplicates. Dedup misses both for two independent reasons: the Plaid rail stores date-only executed_at while the MCP rail stores full timestamps, and the Qonto MCP rail drops the amount sign. The worst finding is one no control point looked for: 460 of 500 sampled FR invoices sit at payment_status unknown with null balance_due, because the connector sync enqueues a status recompute only inside the invoice_transaction branch and the hourly sweeper requires a settlement link to exist. bills-due reports that as 'no bills currently due' -- a false all-clear against ~434 real payables. Also refutes the earlier hypothesis that Mercury's missing provider config starves US categorization: 62% of US rows cleared the sufficiency gate, and an unmapped slot would starve ~100% of a tool's rows, not 38%. Marks the FR sweep report as corrected rather than rewriting it, so the wrong headline and its refutation stay visible together.
zachary95
marked this pull request as draft
July 31, 2026 08:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defines a monthly sweep that answers the question behind every other Well skill: can today's numbers be trusted at all?
It separates two failure classes that look identical in a dashboard and route to completely different fixes:
Every control point declares its bucket, because COMPLETE routes to repair and EXHAUSTIVE routes to connect/backfill. Breadth runs first and scopes every depth finding beneath it.
Shape
SKILL.mdis 377 lines and executable on its own: the two definitions, the(workspace × month)sweep unit and its four states, the loop rules, the ten-gate order, and a File layout table saying when to open each of 14 references. Detail lives inreferences/— control-point families, the latent-assumption audit, MCP surface limits, tolerances, and the dated measurements.Three ideas doing the real work
The sweep unit is
(workspace × month). The month is what the business closes. Months carry their own four states (HAS_FINDINGS/UNSWEPT/CLEAN/EMPTY), andEMPTYis treated as a red when an enabled connector covered the period — false quiet is the most dangerous state here.A check must prove it examined every object. Four verdicts, never two:
pass/fail/inconclusive/sampled. A truncated scan reportsSAMPLED, never a pass. "For every X" checks enumerate the parent set to exhaustion first, and name the parents they never reached rather than averaging them away.A red re-labels later gates rather than halting them — depth findings computed under a breadth failure are scoped, not clean. Latent assumptions run last, because that gate audits the other gates: a red there means the greens are unverified, not verified.
Discipline
INCONCLUSIVE— never a pass. An unexpressible check is named, not silently dropped.references/baseline-2026-07-29.mdwith a decay warning, so staleness is visible at the link.Verification
Restructured from a 1,554-line single file. Losslessness checked mechanically: 189 distinct control-point ids and 204 table rows before and after, none lost. Zero placeholder text; all nine canonical sections present; frontmatter limited to
name+description(939 chars); 14 references linked with relative paths, 0 unlinked, 0 broken.Test plan
Documentation only — no code, no CI in this repo. Validated against the skill-forge master standard's §6 checklist: name/description limits, no placeholders, references linked with relative paths and no unreferenced bundled resources, MCP-endpoint check before account resolution, OAuth/DCR with retry-in-the-same-turn, connector-sufficiency gate, transient-failure retry-once, no unverified deep-link query parameter, and the optional compliance line.
🤖 Generated with Claude Code