Skip to content

feat: in-memory content capture for request/response inspection#360

Merged
elena-oaklight[bot] merged 3 commits into
masterfrom
feat/content-capture
Jul 8, 2026
Merged

feat: in-memory content capture for request/response inspection#360
elena-oaklight[bot] merged 3 commits into
masterfrom
feat/content-capture

Conversation

@clementine-oaklight

Copy link
Copy Markdown
Contributor

Summary

Implements in-memory content capture (#359) — an ephemeral request/response inspector for the admin Dashboard, following the same pattern as the existing ProfilerState profiling feature.

What it captures

For each proxied request (when enabled), the system records:

  1. Original request — the client body as received
  2. Converted body — the target-format body after pipeline conversion
  3. Upstream response — the provider's response body (or accumulated stream chunks for streaming requests)

Design

  • No SQLite, no file I/O — purely in-memory, lost on restart (by design)
  • Same enable-for-N-requests pattern as profiling: POST /admin/api/capture/enable with {"requests": 5}
  • Thread-safe CaptureState with atomic slot claiming via should_capture()
  • Capped at 20 results (rolling), configurable via MAX_RESULTS
  • Streaming captures accumulate raw upstream chunks in the generator's finally block

New files

File Purpose
observability/capture.py CapturedRequest dataclass + CaptureState controller
admin/routes/capture.py 6 API endpoint handlers

Modified files

File Change
proxy.py Record captures in handle_non_streaming and handle_streaming
app.py Consume capture slots, pass capture_state to handlers
admin/__init__.py Instantiate CaptureState on app
admin/routes/__init__.py Register 6 capture API routes
admin.html Dashboard UI: enable/disable toggle, results table, detail viewer (opens in new tab with collapsible JSON sections)
observability/__init__.py Re-export CapturedRequest, CaptureState

API endpoints

Method Path Description
GET /admin/api/capture/status Current capture status
POST /admin/api/capture/enable Enable for N requests
POST /admin/api/capture/disable Disable capture
GET /admin/api/capture/results List result summaries
GET /admin/api/capture/results/<index> Full detail for one result
DELETE /admin/api/capture/results Clear all results

Closes #359

Add CaptureState (same pattern as ProfilerState) for capturing the
full three-stage request flow in memory:
- original_request (client body as received)
- converted_body (target-format after pipeline conversion)
- upstream_response (provider response body)

Enable for N requests via admin API, view results in Dashboard.
No SQLite, no file I/O — ephemeral by design.

New files:
- observability/capture.py — CaptureState + CapturedRequest dataclass
- admin/routes/capture.py — 6 API endpoint handlers

Modified:
- proxy.py — record captures in handle_non_streaming and
  handle_streaming (accumulated chunks for stream responses)
- app.py — consume capture slots and pass to handlers
- admin/__init__.py — instantiate CaptureState on app
- admin/routes/__init__.py — register capture API endpoints
- admin.html — Dashboard UI section with enable/disable toggle,
  results table, and collapsible detail viewer

Closes #359

@elena-oaklight elena-oaklight Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Mostly LGTM, one concurrency issue to fix, a couple of notes.

Overall this is clean work — follows the ProfilerState pattern well, good separation between core state and route handlers, streaming capture in finally is the right approach.

Should fix

CaptureState.status() doesn't hold the lock, but reads mutable state

enable() and disable() call self.status() while holding self._lock. Since threading.Lock is non-reentrant, status() can't also acquire it — so it reads _enabled, _remaining, _results unlocked. This works today, but it's fragile:

  • If someone later adds self._lock to status() → deadlock.
  • External callers (like get_capture_status route handler) call status() without the lock, getting a potentially inconsistent snapshot.

Fix: split into _status_unlocked() (used inside locked sections) and status() (acquires lock for external callers). Or just use threading.RLock().

Notes (non-blocking)

  1. Slot consumed before handler runs: should_capture() is called at the top of _proxy_handler. If the request fails before reaching handle_streaming/handle_non_streaming (e.g. conversion error), a slot is consumed but nothing is recorded. Acceptable for "best effort" capture, but worth a comment.

  2. Unbounded per-capture memory: Streaming captures accumulate all upstream chunks in a list. A long code-gen response could store a lot of data × 20 results. Consider either documenting the memory implication or adding a per-capture byte cap.

  3. Detail viewer hardcodes dark theme: The viewCapture() popup uses background:#1a1a2e regardless of the dashboard's theme setting. Minor UX inconsistency.

status() now acquires the lock for external callers.
enable()/disable() call _status_unlocked() internally since
they already hold the lock. threading.Lock is not reentrant,
so the previous pattern would have deadlocked if status()
ever tried to acquire the lock.
Both error return paths in handle_non_streaming (UpstreamConnectionError
and upstream 4xx/5xx) now record capture data. Error responses are often
the most valuable for debugging/inspection.

@milo-oaklight milo-oaklight Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — PR #360

整体实现和 ProfilerState 一脉相承,CaptureState / CapturedRequest 结构清晰,流式在 generator finally 里攒 chunks 的做法正确。两个需要修的点(已在后续 commit 修复):

1. status() 锁问题(已修 — c6e90f1

status() 读可变状态但不持锁,而 enable()/disable() 持锁时调用 status()threading.Lock 不可重入,所以 status() 没法直接加锁。修法:拆成 _status_unlocked() + status(),或换 RLock()

2. 非流式错误路径不捕获(已修 — 60bc3e2

handle_non_streaming 里 upstream 返回错误时,dump_error 之后直接 return,跳过了 capture_state.record()。流式没这个问题(finally 兜底)。对 inspection 用途来说,错误响应往往更需要看完整 body。

Notes(非阻塞)

  • should_capture() slot 浪费:在 handler 执行前就扣 slot,如果 handler 早期失败(conversion 阶段异常),slot 被消耗但没有对应 record。非流式尤其明显——exception handler 里的 dump_error 路径完全没碰 capture。
  • 流式 chunks 无大小上限:streaming 捕获的 upstream_chunks 没有 cap,大响应 × MAX_RESULTS(20) 条可能占不少内存。

@elena-oaklight elena-oaklight Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All review items addressed. status() lock fix (c6e90f1) and error-path capture (60bc3e2) both verified. CI green. LGTM ✅

@elena-oaklight elena-oaklight Bot merged commit 4329cb0 into master Jul 8, 2026
4 checks passed
@elena-oaklight elena-oaklight Bot deleted the feat/content-capture branch July 8, 2026 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: in-memory content capture for request/response inspection

0 participants