feat(batch): support native bounded OCR actor pools

[jioffe502]

Batch OCR currently accepts a fixed worker count, so a pool explicitly set to two cannot grow when upstream actors release resources. Add opt-in ocr_min_workers, ocr_initial_workers, and ocr_max_workers to batch tuning and ingest plans, forwarding them to Ray Data's native bounded actor pool.

All three bounds must be positive integers with min <= initial <= max; combining them with ocr_workers is rejected. Resource preflight admits the initial population while preserving the maximum for later growth. Existing fixed and automatic defaults remain unchanged. Native Ray handles growth, shrinking, and completed-input retirement; this adds no custom controller.

Stacked on #2748 (jioffe502/batch-nvtx-instrumentation). Review this PR against that branch; merge/rebase after #2748.

Evidence

In the full BO767 batch experiment on four L40S GPUs, changing only OCR from fixed two workers to min/initial/max 2/2/4 reduced ingestion from 1,854.864 to 1,751.188 seconds (-5.59%). NVTX and Ray logs show new OCR workers using GPU 0 after Page Elements retires. All 767 files, 54,730 pages and 991 queries completed. This is a single-run observation, not a universal performance promise. The later embedding-sharing gain uses existing resource settings and is not attributed to this patch alone.

On two H200 GPUs, bounded OCR at 2/2/4 did not grow while two embedding actors each reserved 0.5 GPU; ingest took 1,448.776 and 1,429.860 seconds, versus 1,432.518 seconds for the traced fixed-two control. With embedding reservations set to 0.25 GPU each, two bounded runs took 1,312.690 and 1,296.527 seconds, 117–152 seconds faster than the bounded 0.5 GPU runs. Both reached four OCR actors and processed OCR work on GPU 1. A fixed-two OCR run at 0.25 GPU per embedding actor took 1,453.430 seconds, supporting OCR growth as the source of the gain. All BO767 file, page, query, and indexed-row gates passed. Recall@5 was 0.880928 in both bounded 0.25 GPU runs versus 0.881937 in the bounded 0.5 GPU controls; exact OCR text and some query rankings varied between repeats. OCR also churned through 47 and 41 useful actor lifetimes. These tuned runs support the opt-in capability on this host, not a global allocation default or exact-output parity claim. The BO767 traces were captured on frozen pre-rebase head d1c0ed7; the feature patch is unchanged after rebasing onto the current #2748 head, but the full benchmark has not been repeated on the rebased head.

Validation

• After rebasing onto feat(profiling): add semantic batch NVTX ranges #2748 head 41554f45 and addressing review feedback, the focused OCR pool, ingest plan, pipeline graph/resource preflight, parameter model, and harness runfile tests passed: 247 passed, 3 skipped.
• Resolved node overrides and automatic-node selections were byte-identical between the frozen feat(profiling): add semantic batch NVTX ranges #2748 base and this PR for unspecified OCR and fixed ocr_workers=2 on 56-CPU/2-GPU and 64-CPU/4-GPU resource snapshots (SHA-256 943796e044f92e1e875b225752eb59a1f6a6c02f19db61566fb69770c28a61be for each JSON result). This is a plan-level compatibility check, not an unchanged-default end-to-end performance run.
• Completed traced BO767 runs on L40S and H200 exercise bounded growth, useful OCR work, and finished-input retirement.
• git diff --check and pre-commit run --all-files pass on the current head. The earlier inherited Black formatting failure was corrected in feat(profiling): add semantic batch NVTX ranges #2748 before this rebase.
• On the review-fix head 796ac497, GitHub checks pass: pre-commit, Retriever unit tests, slim imports, Windows/macOS installs, Docker build and test, aggregate PR validation, and automated review.
• Performance-guide documentation reviewed against code and native Ray behavior. MkDocs strict build unavailable because the installed environments lack MkDocs.

Scope: batch-mode Ray Data graphs that contain OCRActor (the PDF extraction graph). Mixed batches apply the bounds to the PDF branch's OCR pool. A run whose built graphs contain no OCRActor (non-PDF-only input, batch auto, method="nemotron_parse", or PDF extraction with all OCR-dependent outputs disabled) rejects the bounds with a clear ValueError. No Helm changes, GPU-fraction default changes, or downscaling-threshold override.

Review follow-up

The current head rejects bounded OCR settings on routes without a dedicated OCR actor, documents the supported extraction mode, adds field descriptions and the required test-file SPDX header, and tests that (min, max, initial) reaches Ray Data's map_batches call. The unchanged-default and fixed-two resolved plans remain byte-identical to the frozen #2748 base after this fix.

Bounded OCR validation now runs once per ingest against the extraction graphs actually built (require_ocr_actor_for_bounded_ocr), instead of per extraction mode inside batch_tuning_to_node_overrides. This fixes mixed PDF + text/image batches, which previously failed on the non-PDF branch, and also rejects PDF graphs that omit OCRActor. Regression tests cover PDF + txt, PDF + image, image + txt (rejected), Nemotron Parse, and OCR-disabled PDF extraction.

Qualification before shipping

The two-H200 qualification is complete. The one-H100 and RTX 6000-class systems remain untested, so this PR makes no broad cross-SKU performance or quality claim. Review the opt-in API, default/fixed compatibility, admission, and terminal/error cleanup separately from the tuned H200 and L40S performance results. An unchanged-default end-to-end base/candidate performance contrast remains unmeasured. Hardware-specific GPU fractions belong in examples, not library defaults; this PR does not change them.

Link to Devin session: https://nvidia-cloud.devinenterprise.com/sessions/51fd6ebb97044cf2882845b5127a5c62
Open in Devin Desktop: https://nvidia-cloud.devinenterprise.com/desktop/session/51fd6ebb97044cf2882845b5127a5c62?variant=devin
Requested by: @jioffe502

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge based on this review; no new findings or outstanding previous findings remain.

Summary

The PR adds opt-in minimum, initial, and maximum worker counts for the dedicated batch PDF OCR actor pool. It validates the bounds, passes them to Ray Data, admits resources for the initial population, and rejects bounds when the built extraction graphs contain no OCR actor. Tests cover the handoff and mixed-input routing. There are no code changes since the previous review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  B[OCR bounds] --> V[Validate values]
  V --> G[Build extraction graphs]
  G --> C{OCR actor present?}
  C -->|No| E[Reject bounds]
  C -->|Yes| P[Preflight initial workers]
  P --> R[Ray Data bounded actor pool]

Loading

_{Reviews (8) · Last reviewed commit: "Validate bounded OCR before starting Ray..." · Reviewed by Greptile}

[devin-ai-integration]

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

Devin Review found 2 potential issues.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[devin-ai-integration]

Devin Review found 1 new potential issue.