fix: use NOT EXISTS instead of NOT IN to avoid cross join in pruneCache

[waterWang]

Summary

The pruneCache method in table_metadata_updater.go deletes orphaned rows from system.table_metadata with a NOT IN (SELECT id FROM system.namespace) predicate. Because system.namespace.id is nullable, the optimizer must preserve three-valued logic and lowers the predicate to (table_id = id) IS NOT false. That is not an equality, so it cannot be hash- or lookup-joined and the plan degrades to a cross join (anti) — costing O(rows(table_metadata) × rows(namespace)). On a 9-node cluster with 100k tables that is ~11.1B comparisons (~38 minutes per run).

Fix

Replace the NOT IN subquery with a correlated NOT EXISTS, which the optimizer can lower to an equality anti-join:

WHERE table_id NOT IN (
  SELECT id FROM system.namespace
)

→

WHERE NOT EXISTS (
  SELECT 1 FROM system.namespace WHERE id = table_id
)

This changes the plan from cross join (anti) (~38 min) to hash join (anti) (~0.2s), verified by the reporter. NOT EXISTS also eliminates the NULL-hazard of NOT IN: a single NULL in system.namespace.id would otherwise cause NOT IN to return the empty set for every input, silently disabling pruning cluster-wide. table_metadata.table_id is NOT NULL, so the result set is identical.

Fixes #173143

[blathers-crl]

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

Before a member of our team reviews your PR, I have some potential action items for you:

• Please ensure your git commit message contains a release note.
• When CI has completed, please ensure no errors have appeared.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[cockroachlabs-cla-agent]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}