Problem
list_jobs (app/api/v1/endpoints/jobs.py):
query = db.query(Job).order_by(Job.created_at.desc())
...
return [_serialize_job(j) for j in query.limit(limit).all()]
with limit bounded at 200. There is no cursor, offset, or next-page link.
Consequences:
- Older jobs are unreachable: after 200 jobs (a busy SLA/compute day), engineers cannot list or inspect earlier jobs at all through the API; the retention UI and ops tooling are capped at the newest slice.
- Filtered queries share the same cap:
job_type/status filters still apply limit — a filtered view of a large failure class shows only the newest 200, hiding older failures from exactly the debugging workflows the endpoint exists for.
- The pattern exists elsewhere to do it right:
app/utils/cursor.py (encode_cursor/decode_cursor, CursorPage) is already used by the outage repository — the jobs endpoint predates or ignores it.
Root cause
The list endpoint was written with a hard limit and no pagination mechanism, before the cursor utilities existed.
Why this is architecturally hard
- Adding cursor pagination means a stable sort key (
created_at DESC, id DESC) and the CursorPage envelope — a response-schema change that the frontend job views and the OpenAPI snapshot must absorb.
- The
_sync_job_status_from_celery interaction matters: cursor keys must be based on DB fields, and status syncing (a Redis call per row) already makes deep pages expensive — pagination exposes that cost per page and must be factored in.
- A test must assert stable ordering across pages (no skips/duplicates under concurrent job creation), following the pattern in
tests/test_outage_db_pagination.py.
Proposed design
Add cursor pagination to list_jobs using app/utils/cursor.py (keep limit as page size), preserve filters, and add a multi-page test asserting stable, gapless ordering.
Acceptance criteria
Service
Tests
Out of scope
Job retention (already implemented) and retry semantics.
Getting started
pytest tests/test_outage_db_pagination.py -q
make typecheck
Good first files to read: app/api/v1/endpoints/jobs.py, app/utils/cursor.py.
Problem
list_jobs(app/api/v1/endpoints/jobs.py):with
limitbounded at 200. There is no cursor, offset, or next-page link.Consequences:
job_type/statusfilters still applylimit— a filtered view of a large failure class shows only the newest 200, hiding older failures from exactly the debugging workflows the endpoint exists for.app/utils/cursor.py(encode_cursor/decode_cursor,CursorPage) is already used by the outage repository — the jobs endpoint predates or ignores it.Root cause
The list endpoint was written with a hard
limitand no pagination mechanism, before the cursor utilities existed.Why this is architecturally hard
created_at DESC, id DESC) and theCursorPageenvelope — a response-schema change that the frontend job views and the OpenAPI snapshot must absorb._sync_job_status_from_celeryinteraction matters: cursor keys must be based on DB fields, and status syncing (a Redis call per row) already makes deep pages expensive — pagination exposes that cost per page and must be factored in.tests/test_outage_db_pagination.py.Proposed design
Add cursor pagination to
list_jobsusingapp/utils/cursor.py(keeplimitas page size), preserve filters, and add a multi-page test asserting stable, gapless ordering.Acceptance criteria
Service
Tests
Out of scope
Job retention (already implemented) and retry semantics.
Getting started
Good first files to read:
app/api/v1/endpoints/jobs.py,app/utils/cursor.py.