Repository navigation
Run the geography suite in the benchmark and commit geography ground-truth answers #134
Description
Follow-up to #131, which adds the geography query suite as SQL and docs only. The benchmark harness, the CI matrix and benchmark/answers/ are untouched by it, so the geography queries are defined but never timed or verified.
Why the two suites need separate answers
verify_results.py resolves ground truth per query: benchmark/answers/sf<n>/q<n>.parquet. The geography suite reuses the query IDs q1..q12 but returns different values for the same IDs — a 0.45° planar radius and a 50 000 m geodesic radius simply select different rows. So the answers cannot be shared, and a per-suite (or per-engine) answer path is needed rather than a flat per-query one.
Concretely, at SF1: Q1 returns a different 100 rows, Q9's areas are m² instead of square degrees, and Q7's line_distance_m is a real geodesic length rather than degrees divided by 0.000009. Q2 (541) and Q4 (258 zones, identical counts and order) happen to agree with the planar answers, which makes them useful cross-checks but does not generalise.
Work involved
- Runner — teach
benchmark/run_benchmark.pyto select the suite. Either a new engine key (sedonadb_geography, which keeps the existing<engine>_<query>_result.csvdump naming working unchanged) or a--suite {geometry,geography}flag.get_sql_queries()currently hardcodes the two geometry dialect classes. - Answers —
benchmark/answers/geography/sf{1,10}/q<n>.{csv,parquet}, generated the same way as the existing ones, plus a note inbenchmark/answers/README.mdexplaining why the two sets differ. - Verify —
verify_results.pyneeds a per-engine (or per-suite) answer-directory mapping, and itsLIMIT_QUERIESboundary-tie set should be re-derived for the geography suite rather than assumed identical. - CI — a geography job block in
.github/workflows/benchmark.yml, mirroring the existing per-query matrix. Note the workflow already triggers onspatialbench-queries/**, so Add geography variants of the SpatialBench queries #131 runs the full geometry matrix without exercising any of the new SQL. - Summarize —
summarize_results.pyhas a hardcoded engine list and icon map.
QUERY_COUNT = 12 is duplicated across run_benchmark.py, verify_results.py, generate_answers.py and summarize_results.py, and the CI matrix lists q1..q12 literally per engine; the geography suite carries 11 queries (no ST_KNN yet — apache/sedona-db#1086, apache/sedona#3178), so whichever mechanism is chosen has to tolerate a suite with a gap rather than assume a contiguous 1..N.
Blocked on
Two SedonaDB defects would corrupt any answers generated today:
- apache/sedona-db#1084 — Q9 cannot complete: a geography scalar function over geography spatial-join output fails with
Array input must not be empty. - apache/sedona-db#1085 — one zone polygon matches points anywhere on Earth, inflating Q4, Q6, Q10 and Q11.
See also #133 for committing the notebook's executed outputs, which is blocked on the same two.
Activity