Skip to content

Move every pin that can move, and make the rest checkable - #3082

Closed
mtoy-googly-moogly wants to merge 19 commits into
mainfrom
dependency-sweep
Closed

mtoy-googly-moogly wants to merge 19 commits into
mainfrom
dependency-sweep

Conversation

@mtoy-googly-moogly

Copy link
Copy Markdown
Collaborator

Move every dependency that can move, and make each one that can't say why in a form the next person can check.

Three of the five "held because it breaks this repo" entries turned out to rest on a wrong diagnosis, so they came out:

  • postgres (Postgres connector: pg / pg-cursor / pg-query-stream must move as a coupled set #2928) was never a pg/pg-cursor version skew. pg 8.8 added an exports map, and jest resolves subpaths through its own resolve.exports — 1.x, which cannot match subpath patterns. Node resolved pg/lib/result.js the whole time. Jest 29.7 (already in range) brought 2.x and the hold evaporated.
  • vscode-textmate (Stop reaching into vscode-textmate internals in generateMonarchGrammar — fix it properly #2918) deep-imported type modules that 9.3.2 doesn't export. The generator reads a TextMate grammar .json, so it now declares that file's shape itself — a contract that survives whatever upstream does with its internals. Generated malloy.monarch.ts is byte-identical.
  • vega was half-released already: vega moved to 6, vega-lite stayed on 5 and dragged its own vega 5 along, which is where the three high advisories lived. vega-lite 6 peers on vega ^6, so there is now one vega and no advisory. No renderer code changed.

Also taken: jest 30, vite 8, peggy 5, jasmine-core 7 (which retires the @types/jasmine tracking ignore), vscode-oniguruma 2, every in-range minor and patch, and the two advisories a direct dep could clear (mysql2, toml). Six dependencies nothing imports are deleted.

Prod npm audit goes 27 → 18, high 14 → 6. Every remaining high and critical reconciles to the snowflake or databricks hold.

Four things did not move, and the ledger now says why in terms that can be re-tested rather than believed:

  • BigQuery — the entry said jest could never load gaxios 7's ESM-only dynamic import. It can: name the importer and its target and the whole auth path runs live queries on bigquery 9. What actually blocks the bump is projectId — the connector reads it synchronously in a constructor and the v8/v9 SDK resolves it asynchronously. The hold waits on us, not on vitest.
  • Node — the 24.17 keep-alive regression is fixed in 24.18.0, but the BigQuery suite on any post-24.16 Node takes an intermittent jest-worker SIGSEGV: 0 in 16 runs on 24.16.0, 2 in 8 on 24.18.0, 2 in 16 on 24.21.0, same tree. precheck is clean on all three. Cause not established; the measurement is. This one now costs us a security release, so it wants an owner.
  • snowflake / databricks — boundaries re-verified rather than assumed. snowflake-sdk@3.3.0 bundles nine .node binaries with an empty optionalDependencies; @databricks/sql@2.1.0 uses the other shape, eight per-platform kernel packages. The ledger now carries the two commands that show this, since no metadata field does.
  • apache-arrow — new hold. @duckdb/duckdb-wasm and @motherduck/wasm-client both require ^17.0.0, so raising ours leaves the tree invalid and our typings a major ahead of the values duckdb-wasm returns, with the build green throughout.

On the open-PR limit hiding things: it does, and turning security auto-updates off is what gave it teeth. GitHub exempts security PRs from the cap — but there are none here, so the cap governs everything Dependabot produces, and full means silent. Slots go to whatever is oldest, so an un-ignored major nobody intends to take squats one forever; storybook alone held four of ten. Storybook, jsdom, lerna and gaxios get ignore rows with argued reasons, which frees the queue.

@types/node is left alone — #3002 already syncs it to the runtime major and adds a check so the two can't drift again.

Verified from a clean tree: npm ci, npm run build, npm run build-duckdb-db, npm run precheck (4176 passing of 4270, unchanged), npm run lint. The one snapshot change is jest 30 rewriting its own header URL.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf

mtoy-googly-moogly and others added 19 commits September 9, 2026 14:13
Moves each range floor to the version npm was already resolving, so
`package.json` and the lockfile agree on one number instead of two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
mysql2 3.24.4 closes the compressed-protocol decompression-bomb DoS.
toml 5.0.0 closes the prototype-pollution and uncontrolled-recursion
pair in connections.toml parsing; snowflake-sdk keeps its own toml 3
copy, so those advisories still ride the snowflake-sdk pin.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
pg 8.23.0 + pg-query-stream 4.17.0 + pg-cursor 2.22.0, ranges back to
carets, ignore entries gone.

The hold's stated cause was a pg/pg-cursor version skew breaking
`pg/lib/result.js`. It isn't. pg 8.8 added an `exports` map, and jest
resolves subpaths through its own `resolve.exports` — 1.x, which cannot
match subpath patterns. Node resolved the path all along; only jest
failed. Jest moves to 29.7.0 (already in range, brings resolve.exports
2.x) and the deep import resolves.

Ledger records the symptom so the next deep-import failure is read as a
resolver question.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
The Monarch generator deep-imported vscode-textmate's internal type
modules; 9.3.2 ships one .d.ts and exports none of them, so no import
path reaches those names.

The generator reads a TextMate grammar .json file, so it now declares
that file's shape itself, in scripts/textmate-grammar.ts. That is the
contract it actually depends on, and it holds whatever vscode-textmate
does with its own types. The generated malloy.monarch.ts is
byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
vega moved to 6 a while back; vega-lite stayed on 5 and dragged its own
vega 5 copy along, which is where the three high vega advisories lived.
vega-lite 6 peers on vega ^6, so the tree now has one vega and the
advisories are gone.

Nothing in the renderer's types or specs needed changing. The vega 6
line is ESM-only, so the four vega packages a spec loads at runtime join
transformIgnoreModules; @malloydata/render bundles them, so consumers
still get CJS.

The ledger's devDep/runtime-dep split had no room for this — a runtime
dep of a package that bundles leaks nothing. Added as the third case,
since it is also the exit for the uuid and @noble/hashes holds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
BigQuery: the entry said jest could never load gaxios 7's ESM-only
dynamic import without --experimental-vm-modules, and that babel could
not help. Naming both the importer and its target in
transformIgnoreModules — gaxios, node-fetch, teeny-request — and writing
transformIgnorePatterns to reach nested copies runs the whole auth path,
live queries included, on bigquery 9.0.3. What actually blocks the bump
is projectId: the connector reads it synchronously in its constructor
and the v8/v9 SDK resolves it asynchronously, so every query fails
"ProjectId must be non-empty". The hold waits on us, not on vitest.

apache-arrow: @duckdb/duckdb-wasm and @motherduck/wasm-client both
require ^17.0.0, so raising ours leaves the tree invalid and the typings
a major ahead of the values duckdb-wasm returns. The build stays green
either way, which is why it needed writing down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
dotenv-cli, fs-extra, @types/fs-extra, cross-os and unified at the root,
and gaxios from db-trino. No source, script, config or workflow
references any of them.

db-bigquery keeps gaxios — it uses GaxiosError. Worth knowing that the
check is broken: the declared ^4.2.0 resolves a different copy from the
gaxios 6 that google-auth-library throws from, so the instanceof is
always false and the auth-error rewrite never fires. Not fixed here;
it changes what errors users see.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
Verified from a clean tree — femto-build's digest covers grammar text,
not the generator's version, so an in-place rebuild would have kept the
peggy 4 output.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
…nore

The karma monarch suite passes 28/28 on jasmine 7, so the ignore that
kept @types/jasmine at 5 has nothing left to track.

That suite is run by hand only — `karma start` has no --single-run, so
it never exits and no workflow invokes it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
Verified from a clean tree: femto-build hashes grammar and source text,
not the toolchain, so an incremental build would have reused the vite 6
bundle and told us nothing. Two forward-looking warnings left alone —
vite.config.base.mts still uses __dirname, and vite 8 can do tsconfig
path resolution without vite-tsconfig-paths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
Same 4176 passing of 4270 across 122 suites as jest 29, no config or
test changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
toBeCalledTimes and toThrowError were removed in jest 30. The three
files using them are all connector suites precheck cannot reach —
postgres and bigquery need a live database — so the break only shows in
a CI job with credentials.

Also records what a Node bump now costs. The 24.17 keep-alive regression
the hold was written for is fixed in 24.18.0, but the BigQuery suite on
any post-24.16 Node takes an intermittent jest-worker SIGSEGV: 0 in 16
runs on 24.16.0, 2 in 8 on 24.18.0, 2 in 16 on 24.21.0, same tree.
precheck is clean on all of them. Cause unknown; .node-version stays.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
…bites

Storybook (four PRs a month) and gaxios get ignore rows; both now have
ledger entries saying what would have to happen to take them.

Two mechanics the ledger was missing:

Security auto-updates being off is what gives open-pull-requests-limit
teeth. GitHub exempts security PRs from the cap — but there are none
here, so the cap governs everything Dependabot produces, and full means
silent. The fix is ignoring squatters, not raising the number.

And a boundary pin can be checked rather than trusted: npm pack the
candidate and grep the tarball for .node, then read
optionalDependencies. snowflake-sdk 3.3.0 hides nine binaries where no
metadata field shows them; @databricks/sql 2.1.0 uses the other shape,
eight per-platform kernel packages. Both were verified this way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
The last member of Dependabot's minor-and-patch group, held back because
0.37 -> 0.331 is a 0.x jump and a 0.x minor is allowed to break. It
doesn't: from a clean tree every committed .js.flow regenerates
byte-identical.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
`ci-core` carried a bare `--reporters` with no value ahead of its real
one. jest 29's yargs swallowed it; jest 30 answers with a usage dump and
"Not enough arguments following: reporters", so the whole core job died
before running a test.

ci-test-sanity-check.sh globbed *.spec.ts out of node_modules, so
`entities` — which parse5 7 pulls under test's jsdom, and which ships
its own specs — was reported as a test CI had forgotten to run. The
message points at jest.config.ts, which would have been the wrong place
to look. The find now prunes node_modules, and any future dependency
that ships a spec file is silent instead of alarming.

The lockfile also had test/'s @types/jsdom at 30 against a ^21.1.1
declaration; npm keeps such an entry across installs rather than
correcting it, so it needed removing by hand.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
"Everything ignored is written down" had no way to be checked. Fifteen
of the seventeen ignores were described in prose, but under display
names — the BigQuery row says "paginator" where dependabot.yml says
"@google-cloud/paginator" — so a reader could confirm the rule and a
script could not. @types/jsdom was absent outright.

The table lists the exact dependency-name strings and their scope, so it
diffs against dependabot.yml directly. That is the point of it: an
ignore with no row is a decision nobody recorded, and the next person
re-makes it from scratch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
The entry claimed post-24.16 Node segfaults the BigQuery suite, from 0
crashes in 16 runs on 24.16.0 against 4 in 24 on 24.18/24.21. That is
not what the data supports.

macOS crash reports show all six crashes share one signature —
v8::internal::ClearStaleLeftTrimmedPointerVisitor::VisitRootPointers, a
V8 garbage-collector fault with no native addon on the stack — and the
node image UUID on the 8 Sep report is 3EBB7B1A, which is 24.16.0's
binary. So 24.16.0 produces the same crash; sixteen runs simply didn't
sample one. The comparison was never strong enough to publish either:
p was around 0.05 by my own arithmetic.

What survives is an observation, not a cause, and the hold loses its
only remaining reason: the 24.17 regression it was written for is fixed
in 24.18.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NuuXJ8g133X9wL6MBiyDUf
@mtoy-googly-moogly

Copy link
Copy Markdown
Collaborator Author

Closing unmerged. The branch dependency-sweep stays pushed if any of it is wanted later.

What's in it: 17 dependencies moved, 6 unused ones removed, 3 holds released (pg #2928, vscode-textmate #2918, vega), prod npm audit 27→18 with highs 14→6, and two CI bugs jest 30 exposed — a bare --reporters in ci-core and ci-test-sanity-check.sh globbing spec files out of node_modules. CI was green across every dialect plus consumer-canary.

Not landing because the pin-ledger half of it is the wrong shape: three of five documented holds turned out to have the wrong cause recorded, so the document was costing more than it returned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant