feat: in-memory content capture for request/response inspection#360
Conversation
Add CaptureState (same pattern as ProfilerState) for capturing the full three-stage request flow in memory: - original_request (client body as received) - converted_body (target-format after pipeline conversion) - upstream_response (provider response body) Enable for N requests via admin API, view results in Dashboard. No SQLite, no file I/O — ephemeral by design. New files: - observability/capture.py — CaptureState + CapturedRequest dataclass - admin/routes/capture.py — 6 API endpoint handlers Modified: - proxy.py — record captures in handle_non_streaming and handle_streaming (accumulated chunks for stream responses) - app.py — consume capture slots and pass to handlers - admin/__init__.py — instantiate CaptureState on app - admin/routes/__init__.py — register capture API endpoints - admin.html — Dashboard UI section with enable/disable toggle, results table, and collapsible detail viewer Closes #359
There was a problem hiding this comment.
Review: Mostly LGTM, one concurrency issue to fix, a couple of notes.
Overall this is clean work — follows the ProfilerState pattern well, good separation between core state and route handlers, streaming capture in finally is the right approach.
Should fix
CaptureState.status() doesn't hold the lock, but reads mutable state
enable() and disable() call self.status() while holding self._lock. Since threading.Lock is non-reentrant, status() can't also acquire it — so it reads _enabled, _remaining, _results unlocked. This works today, but it's fragile:
- If someone later adds
self._locktostatus()→ deadlock. - External callers (like
get_capture_statusroute handler) callstatus()without the lock, getting a potentially inconsistent snapshot.
Fix: split into _status_unlocked() (used inside locked sections) and status() (acquires lock for external callers). Or just use threading.RLock().
Notes (non-blocking)
-
Slot consumed before handler runs:
should_capture()is called at the top of_proxy_handler. If the request fails before reachinghandle_streaming/handle_non_streaming(e.g. conversion error), a slot is consumed but nothing is recorded. Acceptable for "best effort" capture, but worth a comment. -
Unbounded per-capture memory: Streaming captures accumulate all upstream chunks in a list. A long code-gen response could store a lot of data × 20 results. Consider either documenting the memory implication or adding a per-capture byte cap.
-
Detail viewer hardcodes dark theme: The
viewCapture()popup usesbackground:#1a1a2eregardless of the dashboard's theme setting. Minor UX inconsistency.
status() now acquires the lock for external callers. enable()/disable() call _status_unlocked() internally since they already hold the lock. threading.Lock is not reentrant, so the previous pattern would have deadlocked if status() ever tried to acquire the lock.
Both error return paths in handle_non_streaming (UpstreamConnectionError and upstream 4xx/5xx) now record capture data. Error responses are often the most valuable for debugging/inspection.
There was a problem hiding this comment.
Review — PR #360
整体实现和 ProfilerState 一脉相承,CaptureState / CapturedRequest 结构清晰,流式在 generator finally 里攒 chunks 的做法正确。两个需要修的点(已在后续 commit 修复):
1. status() 锁问题(已修 — c6e90f1)
status() 读可变状态但不持锁,而 enable()/disable() 持锁时调用 status()。threading.Lock 不可重入,所以 status() 没法直接加锁。修法:拆成 _status_unlocked() + status(),或换 RLock()。
2. 非流式错误路径不捕获(已修 — 60bc3e2)
handle_non_streaming 里 upstream 返回错误时,dump_error 之后直接 return,跳过了 capture_state.record()。流式没这个问题(finally 兜底)。对 inspection 用途来说,错误响应往往更需要看完整 body。
Notes(非阻塞)
should_capture()slot 浪费:在 handler 执行前就扣 slot,如果 handler 早期失败(conversion 阶段异常),slot 被消耗但没有对应 record。非流式尤其明显——exception handler 里的dump_error路径完全没碰 capture。- 流式 chunks 无大小上限:streaming 捕获的
upstream_chunks没有 cap,大响应 × MAX_RESULTS(20) 条可能占不少内存。
Summary
Implements in-memory content capture (#359) — an ephemeral request/response inspector for the admin Dashboard, following the same pattern as the existing
ProfilerStateprofiling feature.What it captures
For each proxied request (when enabled), the system records:
Design
POST /admin/api/capture/enablewith{"requests": 5}CaptureStatewith atomic slot claiming viashould_capture()MAX_RESULTSfinallyblockNew files
observability/capture.pyCapturedRequestdataclass +CaptureStatecontrolleradmin/routes/capture.pyModified files
proxy.pyhandle_non_streamingandhandle_streamingapp.pycapture_stateto handlersadmin/__init__.pyCaptureStateon appadmin/routes/__init__.pyadmin.htmlobservability/__init__.pyCapturedRequest,CaptureStateAPI endpoints
/admin/api/capture/status/admin/api/capture/enable/admin/api/capture/disable/admin/api/capture/results/admin/api/capture/results/<index>/admin/api/capture/resultsCloses #359