Skip to content

[PXT-477] Add migration for existing primary key tables - #4

Open
Chrisclone wants to merge 1 commit into
christopherpestano:b-treefrom
Chrisclone:fix/pr1203-migration
Open

[PXT-477] Add migration for existing primary key tables#4
Chrisclone wants to merge 1 commit into
christopherpestano:b-treefrom
Chrisclone:fix/pr1203-migration

Conversation

@Chrisclone

Copy link
Copy Markdown

Migration Strategy for Existing Primary Key Tables

Implements the migration path for tables created with primary_key columns before the PK enforcement PR.

Problem

PR pixeltable#1203 adds a partial unique B-tree index for new tables, but existing tables with is_pk=True columns have:

  • No PrimaryIndexMd in their metadata
  • No physical unique index in PostgreSQL
  • Potentially duplicate live rows (the bug this PR fixes)

Solution: Schema Version 48 → 49

Converter convert_48.py runs during catalog load and for each base table with PK columns but no index:

  1. Generates PrimaryIndexMd (matching create_initial_md() logic)
  2. Attempts CREATE UNIQUE INDEX inside a savepoint
  3. If succeeds → persists updated metadata with primary_index_md, PK enforcement is active
  4. If fails (duplicates exist) → rolls back savepoint, strips is_pk from all columns, logs warning:

    "Table X had primary_key columns with duplicate rows. Primary key constraint removed."

Design Decisions

  • No silent deduplication — removing PK designation is safe; user data is untouched
  • Savepoint isolation — failed index creation doesn't abort the migration transaction
  • String PKs use left(col, 256) truncation matching BtreeIndex.MAX_STRING_LEN
  • Views skipped — only base tables get the physical index
  • Idempotent — checks for primary_index_md is None before running

Files

  • pixeltable/metadata/converters/convert_48.py — migration logic
  • pixeltable/metadata/__init__.py — version bump 48 → 49
  • pixeltable/metadata/notes.py — version note
  • tests/test_pk_migration.py — 4 test cases (happy path, duplicates, composite PK, idempotency)

Generated by OpenClaw

…bles

When upgrading from schema version 48 to 49, existing tables with
is_pk=True columns but no primary_index_md get a PrimaryIndexMd
generated and a physical partial unique index created. If index
creation fails due to duplicate live rows, is_pk is stripped from
all columns to preserve data without silent deduplication.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants