refactor!: unify the two pagination stacks into one (#202)

PR: #202

Author: OmarAlJarrah

CLAUDE.md

@@ -49,10 +49,14 @@ unpublished and not listed below: `sdk-shrink-test`, a test-only R8 shrink-survi
 Key `sdk-core` packages (`org.dexpace.sdk.core.*`): `client` (the `HttpClient`/`AsyncHttpClient` transport
 SPIs), `http.request` / `http.response` / `http.common` (immutable models), `http.context` (context promotion
 chain), `http.sse` (WHATWG Server-Sent Events),
-`http.paging` (`PagedIterable`), `http.pipeline` (+`.steps` — stage-based sync/async pipeline runtime),
+`http.pipeline` (+`.steps` — stage-based sync/async pipeline runtime),
 `auth` (credentials, RFC 7235 challenges, Digest), `pipeline` (+`.step`, `.step.retry` — recovery-aware
 Request/Response/Execution pipeline primitives),
-`pagination` (Paginator + 4 strategies), `serde` (incl. `Tristate`), `instrumentation` (+`.metrics`), `io`,
+`pagination` (unified paging: `Page` (raw per-page `Response`, `Closeable`) / `PageInfo`,
+`Paginator`/`AsyncPaginator` with item- and page-level views, the auto-closing `CloseablePages`
+view returned by `byPage()`, `PagedIterable`, 3 strategies, internal `PageWalker` driver),
+`serde` (incl. `Tristate`),
+`instrumentation` (+`.metrics`), `io`,
 `config`, `util`, `generics`. The full package map with highlights is in `README.md`.
 
 `docs/` (read before structural changes): `architecture.md`, `http.md`, `io.md`, `pipelines.md`,

README.md

@@ -270,9 +270,8 @@ See [docs/pipelines.md](docs/pipelines.md) for the step-author walkthrough.
 | `http.pipeline` | Sync (`HttpStep` / `HttpPipeline` / `HttpPipelineBuilder` / `PipelineNext` / `Stage`) and async (`AsyncHttpStep` / `AsyncHttpPipeline` / `AsyncHttpPipelineBuilder` / `AsyncPipelineNext`) pipeline machinery, plus `AsyncPipelineBridges`. |
 | `http.pipeline.steps` | Concrete steps: `RetryStep`, `RedirectStep`, `AuthStep`, `KeyCredentialAuthStep`, `BearerTokenAuthStep`, `InstrumentationStep`, `SetDateStep`, and their `*Options` / `*Condition` types. |
 | `http.sse` | `ServerSentEventReader` (WHATWG spec), `ServerSentEvent`, `ServerSentEventListener`, `BufferedSource.readServerSentEvents()`. |
-| `http.paging` | `PagedIterable<T>`, `PagedResponse<T>`, `PagingOptions` with `byPage()` and `stream()` accessors. |
 | `auth` | `Credential` sealed hierarchy (`KeyCredential`, `NamedKeyCredential`, `BearerToken`), `BearerTokenProvider`, `AuthScheme`, per-operation `AuthRequirement` / `AuthDescriptor` with `AuthDescriptorResolver` precedence ladder, RFC 7235 challenge parser, `BasicChallengeHandler`, `DigestChallengeHandler`, `CompositeChallengeHandler`. |
-| `pagination` | `Paginator<T>` (with a `maxPages` safety cap) over cursor / page-number / link-header `PaginationStrategy` implementations, plus `Page<T>` / `SimplePage<T>`. Token-style APIs use `CursorPaginationStrategy` with the query-param name set (e.g. `"page_token"`). |
+| `pagination` | Unified paging surface: `Page<T>` (exposes the raw per-page `Response`; `Closeable`) / `PageInfo<T>`; `Paginator<T>` / `AsyncPaginator<T>` (strategy-driven, sync + async, each carrying a `maxPages` safety cap) exposing item-level (`iterateAll` / `streamAll`, which eager-close each page) and page-level views — sync `byPage` returns the auto-closing `CloseablePages` view (wrap in `use {}` / try-with-resources), async `forEachPageAsync` delivers a live page valid only during the consumer callback — over cursor / page-number / link-header `PaginationStrategy` implementations; `PagedIterable<T>` (functional, transport-agnostic first/next-page fetchers); and the internal `PageWalker` driver shared by the sync paths. Token-style APIs use `CursorPaginationStrategy` with the query-param name set (e.g. `"page_token"`). |
 | `operation` | `OperationParams` — SPI projecting an operation's typed inputs (path / query / header / body) into a `Request` and the context chain, via `toRequest(baseUrl)` / `toRequestContext(baseUrl, dispatch)`. |
 | `pipeline` | Recovery-aware primitives: `RequestPipeline`, `ResponsePipeline`, `ExecutionPipeline` over a sealed `ResponseOutcome`, with steps (`pipeline.step`, `pipeline.step.retry`) like `RetryStep`, `ResponseRecoveryStep`, `IdempotencyKeyStep`, `ClientIdentityStep`. |
 | `serde` | `Serde`, `Serializer`, `Deserializer` abstractions, `Tristate<T>` (absent / null / present), and `SerdeException` (the unchecked failure adapters translate codec errors into). |

docs/architecture.md

@@ -79,12 +79,11 @@ java-sdk/
         http/response/                Response, ResponseBody, LoggableResponseBody, Status, typed exception hierarchy
         http/common/                  Headers, MediaType, Protocol, CommonMediaTypes, ETag, HttpRange, conditions
         http/context/                 Call/dispatch/request/exchange contexts + ContextStore
-        http/paging/                  PagedIterable, PagedResponse, PagingOptions
         http/pipeline/                Stage-based sync/async pipeline runtime (+ .steps)
         http/sse/                     WHATWG Server-Sent Events reader/listener/events
         auth/                         Credentials, RFC 7235 challenge parsing, Basic/Digest/Composite handlers
         pipeline/                     Recovery-aware Request/Response/Execution pipeline primitives (+ .step, .step.retry)
-        pagination/                   Paginator + cursor/page-number/token/link-header strategies
+        pagination/                   Unified paging: Page (raw per-page Response, Closeable)/PageInfo, Paginator/AsyncPaginator (item- + page-level views), CloseablePages, PagedIterable, strategies, PageWalker driver
         client/                       HttpClient + AsyncHttpClient interfaces (transport SPI)
         serde/                        Serialization abstractions + Tristate
         instrumentation/              Tracing, spans, scopes, logging (+ .metrics)
@@ -344,16 +343,21 @@ for usage examples.
 
 ### Pagination
 
-**Packages**: `org.dexpace.sdk.core.pagination`, `org.dexpace.sdk.core.http.paging`
+**Package**: `org.dexpace.sdk.core.pagination`
 
-Two complementary surfaces for walking multi-page responses.
+One unified package for walking multi-page responses, offering two complementary entry styles
+(strategy-driven and functional) over a single `Page` type that exposes the raw per-page `Response`.
 
 | Type                                                            | Role                                                                  |
 |-----------------------------------------------------------------|-----------------------------------------------------------------------|
-| `Paginator<T>`                                                  | Lazily iterates pages by re-issuing requests through an `HttpClient`; carries a `maxPages` safety cap |
-| `PaginationStrategy<T>`                                         | Computes the next-page request (or stops) from the current page       |
+| `Page<T>` / `PageInfo<T>`                                       | `Page<T>` exposes the raw per-page `Response` and is `Closeable`: its `items` are materialized and its `statusCode` / `headers` / `request` survive `close()`, while the raw response body/connection is live only while the page is current. `PageInfo<T>` is the strategy's parse result (items + next request) |
+| `CloseablePages<T>`                                             | Auto-closing, single-use page-level view returned by `byPage()`: closes each page as you advance past it and closes the page still held on view-`close()`; exposes `iterator()` / `stream()`. Wrap in `use {}` (Kotlin) / try-with-resources (Java) so an early break still releases the held page |
+| `Paginator<T>`                                                  | Strategy-driven sync paginator over an `HttpClient`; exposes item-level (`iterateAll` / `streamAll`, which eager-close each page so item consumers carry no close burden) and page-level (`byPage`, returning a `CloseablePages`) views; carries a `maxPages` safety cap |
+| `AsyncPaginator<T>`                                             | Non-blocking counterpart over an `AsyncHttpClient` (`forEachAsync` / `collectAllAsync` / `forEachPageAsync`); `forEachPageAsync` delivers a live page valid only during the consumer callback (the driver closes it as soon as the callback returns) |
+| `PaginationStrategy<T>`                                         | Parses a response into a `PageInfo` — items plus the next-page request (or `null` to stop) |
 | `CursorPaginationStrategy` / `PageNumberPaginationStrategy` / `LinkHeaderPaginationStrategy` | The shipped strategies |
-| `PagedIterable<T>`                                              | First/next-page fetcher abstraction over `PagedResponse`, with its own `maxPages` cap |
+| `PagedIterable<T>`                                              | Functional, transport-agnostic first/next-page fetcher abstraction, with its own `maxPages` cap |
+| `PageWalker<T>` (internal)                                      | Shared driver behind the sync paths, exposing a live-page iterator, an eager-closing item iterator, and an item stream |
 
 Token-style APIs (`next_page_token`, `pageToken`, …) are handled by `CursorPaginationStrategy`
 constructed with the query-param name set (e.g. `"page_token"`), so no separate token strategy is needed.
@@ -730,14 +734,13 @@ they should construct a fresh one.
 | `http.response.exception` | HttpException, HttpExceptionFactory, Retryable, RequestTimeoutException (and siblings), NetworkException |
 | `http.common`        | Headers, MediaType, CommonMediaTypes, Protocol, ETag, HttpRange, RequestConditions             |
 | `http.context`       | CallContext, DispatchContext, RequestContext, ExchangeContext, ContextStore                     |
-| `http.paging`        | PagedIterable, PagedResponse, PagingOptions                                                      |
 | `http.pipeline`      | HttpPipeline, HttpPipelineBuilder, HttpStep, Stage, AsyncHttpPipeline (+ `.steps`)              |
 | `http.sse`           | ServerSentEvent, ServerSentEventReader, ServerSentEventListener                                 |
 | `auth`               | Credential, KeyCredential, BearerToken, ChallengeHandler, Basic/Digest/CompositeChallengeHandler, AuthChallengeParser |
 | `pipeline`           | RequestPipeline, ResponsePipeline, ExecutionPipeline, ResponseOutcome                           |
 | `pipeline.step`      | PipelineStep, RequestPipelineStep, ResponsePipelineStep, ResponseRecoveryStep, ClientIdentityStep, IdempotencyKeyStep |
 | `pipeline.step.retry`| RetryStep, RetrySettings, BackoffCalculator, RetryAfterParser                                   |
-| `pagination`         | Paginator, PaginationStrategy, Cursor/PageNumber/Token/LinkHeader strategies, Page              |
+| `pagination`         | Page (Closeable; raw per-page Response), PageInfo, CloseablePages, Paginator, AsyncPaginator, PaginationStrategy, Cursor/PageNumber/LinkHeader strategies, PagedIterable, PageWalker (internal) |
 | `client`             | HttpClient, AsyncHttpClient                                                                      |
 | `serde`              | Serde, Serializer, Deserializer, Tristate                                                        |
 | `instrumentation`    | InstrumentationContext, Span, NoopSpan, NoopInstrumentationContext, Tracer, TracingScope, TraceIdType, ClientLogger |

docs/implementation-plan.md

@@ -372,6 +372,8 @@ defaults (per Square: `FAIL_ON_UNKNOWN_PROPERTIES=false`, `WRITE_DATES_AS_TIMEST
 
 ### WU-9: Pagination primitives
 
+> Superseded by #30 (pagination unification): `Page` now exposes the raw per-page `Response` and is `Closeable` (materialized `items` and derived `statusCode` / `headers` / `request` survive `close()`), strategies return `PageInfo` (`nextRequest == null` = end of stream), and `SimplePage` was removed.
+
 **Status: shipped.** `Page`, `Paginator`, `PaginationStrategy`, and the three strategies
 (`Cursor` / `PageNumber` / `LinkHeader`) are in `sdk-core/.../pagination`, alongside
 helper types `SimplePage` and `RequestRebuilder`. `Paginator` gained a `maxPages` safety cap

docs/refs-comparison.md

@@ -230,7 +230,7 @@ below records where each scheme's design was sourced from:
 ### Pagination
 
 `Paginator<T>` + `Page<T>` ship in `sdk-core`, driven by a `PaginationStrategy` (cursor,
-page-number, token, link-header), and `http.paging.PagedIterable` wraps the result. Reference
+page-number, token, link-header), and `pagination.PagedIterable` wraps the result. Reference
 designs we drew on:
 
 - **Square**: `SyncPagingIterable<T>` (`Iterable<T>` lazy iterator), `SyncPage<T>` (per-page holder), `BiDirectionalPage<T>` (forward + backward cursors), `CustomPager<T>` (user-implementation stub for HATEOAS).