[auto-review #1430] fix CI failures in DB backend benchmark - #32
Open
Chrisclone wants to merge 1 commit into
Open
[auto-review #1430] fix CI failures in DB backend benchmark#32Chrisclone wants to merge 1 commit into
Chrisclone wants to merge 1 commit into
Conversation
- Add `@pytest.mark.benchmark` to test_db_backend_direct_bench so the docker-spinning benchmark is excluded from the default CI matrix, like every other benchmark in the repo (addopts excludes `benchmark`). This is what was causing the minimal/standard/windows test jobs to run it and fail (~30-60min each). - Raise the local pgvector container's max_connections above the busiest sweep's peak (max(_N_PROCESSES)*max(_N_THREADS)=128 > default 100), fixing the "FATAL: sorry, too many clients already" failure at [1cpu-8-16]. - Add `# noqa: PLW0603` on the ProcessPoolExecutor initializer's `global` statement (repo convention) to fix the static-checks ruff failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-review fixes for #1430 ([Ignore] WIP Benches). Targets
benchso it flows into your PR.Issues found
All CI failures on pixeltable#1430 trace to the new
tests/test_db_backend_bench.py;pixeltable/env.py(external-Postgres pgvector bootstrap) is sound and left untouched.CI failure — test jobs (minimal / standard-ubuntu / standard-windows / validate gate). The benchmark was missing the
@pytest.mark.benchmarkmarker, so it ran in the default CI matrix instead of being excluded. Every other benchmark in the repo (tests/benchmarks/*,test_query.py) carries this marker, andpyproject.tomladdoptsdeselectsbenchmark. Running it spun a docker pgvector container per config and ate ~30-60 min per job before failing.CI failure — "FATAL: sorry, too many clients already" at
[local-1cpu-8-16]. The busiest sweep point opensmax(_N_PROCESSES) * max(_N_THREADS)= 8x16 = 128 pooled connections at once (each process haspool_size=n_threads), which exceeds Postgres's defaultmax_connections=100. Real bug in the benchmark — that data point could never complete.CI failure — static-checks (ruff
PLW0603).global _PROC_ENGINEin theProcessPoolExecutorinitializer tripped the lint rule.static-checksruns ruff on all files regardless of markers, so this needed fixing independently.Changes made
@pytest.mark.benchmarktotest_db_backend_direct_bench— right-layer fix that keeps this docker-spinning benchmark out of the default suite (fixes [CI Fix] PXT-1063 GC dispatch — fix premature gc_targets for cross-context slots #1 and thevalidategate).max_connectionstomax(_N_PROCESSES) * max(_N_THREADS) + 16(derived from the sweep, so it stays correct if the sweep changes) via-c max_connections=...ondocker run(fixes [CI Fix] [PXT-1065 + PXT-1066] Add export_csv and export_json #2).# noqa: PLW0603on theglobalstatement, matching repo convention (6+ existing uses) (fixes [Quality Review] PXT-1047 string to_sql — fix replace_re backreference divergence #3).Validated locally
python3 -m py_compileon both changed files — OK.ruff checkandruff format --checkon both files — pass ("All checks passed" / "already formatted").pytest/collection or spin the docker backend here — the available interpreter has nopixeltableinstall, and I didn't run docker. Marker-based deselection is mechanically identical to the repo's other benchmarks, so I'm confident CI's default matrix will now skip it.False-positive risk
max_connectionsfix only covers thelocal-1cpudocker backend. Theplanetscalebackend (gated behindPXT_BENCH_PLANETSCALE_URL) opens the same 128 connections and could hit PlanetScale's own connection ceiling — not something I can set server-side. If you run that backend at8x16, watch for the same error.@pytest.mark.benchmarkmarker is the wrong call — but given it's a timing benchmark with only anassert n > 0, excluding it matches how the repo treats benchmarks.max_connections=144; the arithmetic is straightforward but worth a sanity check on your first real run.🤖 Generated with Claude Code