[jiangxt2]

Description

Apply Ray-provided per-task row budgets to Doris worker SELECTs so eligible bounded Dataset reads can avoid returning and converting unnecessary rows. Carry each budget on an immutable task-local split, append SQL LIMIT after the complete WHERE clause, and retain Ray's ReadTask and exact global limit enforcement.

Validate budgets before discovery, preserve cached planning across bounded and unbounded reads, return a schema-carrying empty task for zero, and leave budgets above Doris's signed 64-bit range to Ray. Both MySQL and Flight SQL use the same budget. Document the native Dataset operations that receive one and add unit, native contract, real Doris and distributed coverage.

Pin the slow-suite HAProxy default to 3.4.7-alpine and skip pulling it when the configured image is cached locally.

Related issues

Fixes #8.

How tested

• python -m pytest tests/unit tests/contract -q --tb=short: 651 passed on Python 3.12.12, Ray 2.59.0 and PyArrow 26.0.0. The three changed unit-test files also pass the local submission checks.
• python -m pytest tests/contract/test_read_limit.py -q --tb=short: all 11 distinct contract cases verified on each of Ray 2.49.2, 2.57.0 and 2.58.0 with Python 3.12.12.
• Real Doris 4.0.6 integration evidence covers all 64 distinct cases, including MySQL/Flight/auto execution, residual filters, empty projections and existing writes. Reproduce with python -m pytest tests/integration -q --tb=short. The validated BE profile uses a temporary 3 GiB memory override; the checked-in 2 GiB profile is outside this successful evidence.
• RAY_DORIS_SLOW_PROFILE=full tests/slow_integration/run.sh: 15 passed on Ray 2.59.0 and Doris 4.0.6, with three Ray workers and three 3 GiB BEs. Cached base images were retained and no images were downloaded.
• Ruff lint/format, strict mypy, python tools/check_docs.py, and Sphinx strict/spelling checks passed.

Additional information

No public parameter or dependency-range change. On Ray 2.59, row-wise map and column rename followed by limit can receive a source budget; a fully pushed predicate followed by map can also receive one. Direct limit and pure column selection followed by limit currently receive no V1 source budget, while residual filtering, sort and flat-map block it. MySQL remains production-candidate; Flight SQL and auto remain experimental. Unordered reads do not guarantee a particular subset or a measured speedup.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] This PR appears safe to merge; no actionable issues were found.

Summary

Pushes Ray-provided task row budgets into Doris worker queries while keeping Ray's own limit enforcement.

• Doris worker queries now use Ray's per-task row budgets.
• The slow test runner reuses HAProxy 3.4.7 when it is cached.

No actionable issues found. Tests were not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Ray supplies a task budget] --> B[Validate budget]
  B --> C[Reuse schema and tablet discovery]
  C --> D{Budget is zero?}
  D -->|Yes| E[Return empty task with schema]
  D -->|No| F[Copy budget onto each task split]
  F --> G[MySQL or Flight SQL]
  G --> H[Append LIMIT within Doris range]
  H --> I[Ray enforces task and global limits]

Loading

_{Reviews (1) · Last reviewed commit: "[Data] Push native row budgets into Dori..." · Reviewed by Greptile}