[codex] add batched collection traversal#66
Conversation
Add fx.eachBatch for processing iterables in sequential batches while preserving result order and global item indexes. Validate invalid batch sizes synchronously with a RangeError and reuse existing concurrency options within each batch. Cover ordering, bounded concurrency, type inference, docs, examples, and changeset for Issue #55.
Greptile SummaryAdds
Confidence Score: 5/5The implementation is correct and the one previous concern about generator exhaustion on re-run is properly resolved with Effect.suspend. The core logic is sound: the No files require special attention.
|
| Filename | Overview |
|---|---|
| src/concurrency.ts | Adds validateBatchSize, batchIterable generator, and eachBatch export; batchIterable is correctly constructed inside Effect.suspend so the iterator is fresh on each run; normalizeConcurrency is called once per batch rather than once per run (minor style issue) |
| src/index.ts | Exports eachBatch from concurrency module — straightforward one-liner addition |
| test/runtime.test.ts | Adds four tests: result order, per-batch concurrency bound, task re-runnability (covers the Effect.suspend fix), and invalid-batch-size rejection; coverage is solid |
| test/types.test.ts | Adds compile-time type assertion that eachBatch returns boolean[] — correct |
| examples/batch-job.ts | Replaces fx.sequence(ids.map(...)) with fx.eachBatch(ids, 100, ...) — straightforward migration to the new helper |
| examples/collections.ts | Adds a batched task demonstrating eachBatch with per-batch concurrency; exports result correctly |
| docs/concurrency.md | Documents batch traversal semantics, concurrency behaviour, and synchronous batch-size validation — accurate and consistent with implementation |
Reviews (2): Last reviewed commit: "fix: make eachBatch task reusable" | Re-trigger Greptile
Create the batch iterator inside Effect.suspend so repeated executions of the same eachBatch task reprocess re-iterable inputs instead of seeing an exhausted generator. Keep batch-size validation synchronous and add regression coverage for running the same task twice.
Summary
Adds
fx.eachBatch(items, batchSize, f, options?)for processing collections in sequential batches while preserving result order and global item indexes.Why
Issue #55 calls out that callers currently need to either collect all traversal work at once or manually chunk large inputs before using
fx.each/fx.sequence. This helper gives the public API a first-class batch traversal path for large workloads that need to limit memory pressure and external API load.Implementation
RangeError: batch size must be a positive finite integer.concurrencyoption inside each batch.Task<A, E, R>style.fx.eachBatch.Docs and Examples
docs/concurrency.mdanddocs/api-reference.md.fx.eachBatchusage toexamples/collections.ts.examples/batch-job.tsto the new batch traversal helper.Validation
bun fmtbun lint:fixbun lintbun testbun typecheckbun run examples:smokeCloses #55.