Skip to content

[auto-review #1430] fix CI failures in DB backend benchmark - #32

Open
Chrisclone wants to merge 1 commit into
christopherpestano:benchfrom
Chrisclone:auto-review/pr-1430-20260701T140001Z
Open

[auto-review #1430] fix CI failures in DB backend benchmark#32
Chrisclone wants to merge 1 commit into
christopherpestano:benchfrom
Chrisclone:auto-review/pr-1430-20260701T140001Z

Conversation

@Chrisclone

Copy link
Copy Markdown

Auto-review fixes for #1430 ([Ignore] WIP Benches). Targets bench so it flows into your PR.

Issues found

All CI failures on pixeltable#1430 trace to the new tests/test_db_backend_bench.py; pixeltable/env.py (external-Postgres pgvector bootstrap) is sound and left untouched.

  1. CI failure — test jobs (minimal / standard-ubuntu / standard-windows / validate gate). The benchmark was missing the @pytest.mark.benchmark marker, so it ran in the default CI matrix instead of being excluded. Every other benchmark in the repo (tests/benchmarks/*, test_query.py) carries this marker, and pyproject.toml addopts deselects benchmark. Running it spun a docker pgvector container per config and ate ~30-60 min per job before failing.

  2. CI failure — "FATAL: sorry, too many clients already" at [local-1cpu-8-16]. The busiest sweep point opens max(_N_PROCESSES) * max(_N_THREADS) = 8x16 = 128 pooled connections at once (each process has pool_size=n_threads), which exceeds Postgres's default max_connections=100. Real bug in the benchmark — that data point could never complete.

  3. CI failure — static-checks (ruff PLW0603). global _PROC_ENGINE in the ProcessPoolExecutor initializer tripped the lint rule. static-checks runs ruff on all files regardless of markers, so this needed fixing independently.

Changes made

Validated locally

  • python3 -m py_compile on both changed files — OK.
  • ruff check and ruff format --check on both files — pass ("All checks passed" / "already formatted").
  • Could not run pytest/collection or spin the docker backend here — the available interpreter has no pixeltable install, and I didn't run docker. Marker-based deselection is mechanically identical to the repo's other benchmarks, so I'm confident CI's default matrix will now skip it.

False-positive risk

  • max_connections fix only covers the local-1cpu docker backend. The planetscale backend (gated behind PXT_BENCH_PLANETSCALE_URL) opens the same 128 connections and could hit PlanetScale's own connection ceiling — not something I can set server-side. If you run that backend at 8x16, watch for the same error.
  • Intent check: if you wanted this benchmark to run as a gating test in CI (rather than on-demand), the @pytest.mark.benchmark marker is the wrong call — but given it's a timing benchmark with only an assert n > 0, excluding it matches how the repo treats benchmarks.
  • I did not run the benchmark end-to-end, so I haven't empirically confirmed 128 connections succeed at max_connections=144; the arithmetic is straightforward but worth a sanity check on your first real run.

🤖 Generated with Claude Code

- Add `@pytest.mark.benchmark` to test_db_backend_direct_bench so the
  docker-spinning benchmark is excluded from the default CI matrix, like
  every other benchmark in the repo (addopts excludes `benchmark`). This
  is what was causing the minimal/standard/windows test jobs to run it and
  fail (~30-60min each).
- Raise the local pgvector container's max_connections above the busiest
  sweep's peak (max(_N_PROCESSES)*max(_N_THREADS)=128 > default 100),
  fixing the "FATAL: sorry, too many clients already" failure at [1cpu-8-16].
- Add `# noqa: PLW0603` on the ProcessPoolExecutor initializer's `global`
  statement (repo convention) to fix the static-checks ruff failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant