Skip to content

Feature request: Workspace Control MCP extension — agent-driven GUI/daemon control and glass-box, human-interactive subagent tabs #30

Description

@Broccolito

Please explain the motivation behind the feature request.

Two problems, one mechanism.

1. The agent cannot see or shape its own workspace. The daemon already runs many concurrent agents (AgentManager LRU registry keyed by session id, crates/biorouter/src/execution/manager.rs), and the GUI already renders them as tabs and split panes — but the agent has no tool surface over any of it. It cannot open a sibling conversation in a tab, read what another conversation concluded or what tools it called, inject a prompt into another conversation, or change which extensions/knowledge bases a conversation may use. Every one of those operations already exists as an HTTP route or a renderer reducer action; none is reachable from a tool call.

2. Subagents are black boxes. When the main agent delegates via the subagent tool (crates/biorouter/src/agents/subagent_tool.rs), the child runs a full agent loop with its own session, but:

  • its event stream is deliberately not user-visible (subagent_handler.rs — only the final summary reaches the parent; nothing reaches the human);
  • its session is persisted but excluded from the browsable session list (session_manager.rs list_session_summaries filters WHERE session_type IN ('user','scheduled')), so a sub_agent transcript cannot be opened even after the fact;
  • the human cannot see the spawn prompt/context, the granted extensions/skills/KBs, or the live tool calls, and cannot inject course corrections. The only affordance is discover-and-kill via active_work (whose GUI panel is itself deferred).

Flagship embodiment this request is built around: whenever the agent spawns a subagent, the GUI opens a new tab bound to the child's session. The tab shows the exact prompt and context the subagent was started with, streams every command/tool call/function execution live, and displays the extensions, skills, and knowledge bases made available to it. The human can talk to the subagent through that tab's ordinary chat box; the subagent reports findings back to the parent; parent and human can both steer or abort it. Subagents become interoperable by the AI and by humans, symmetrically.

Describe the solution you'd like

📐 Plan of record — read this first if you are implementing. The full design document for this feature is committed in-repo as docs/agent-loop/designs/agent-workspace-control.md (BR-71) (landed in f911951a; permalink to the landed revision: here). It carries everything this issue summarizes plus the parts an implementer needs that an issue can't hold: exact file:line citations into the current code for every reuse point, the full tool JSON schemas, the WorkspaceBridge frame vocabulary, the session-model migrations, the drafted server-instructions block, the slice-by-slice test gates, and the binding reconciliation rulings against chatrecall/Memory/active_work/Agent Drafter. Any agent (human or AI) picking this up should treat that document as the plan of record and this issue as its summary — and should keep the document updated as slices ship, per the status-header convention in docs/agent-loop/designs/README.md, so no design work is lost or re-derived.

A workspace control surface in three parts — full specification in the design doc above (BR-71, continuing the campaign numbering).

Part 1 — A workspace platform extension (MCP tools for the agent)

A new in-process platform extension (like chatrecall: registered in PLATFORM_EXTENSIONS, default_enabled: false), exposing:

Tool What it does
workspace_list List conversations: id, name, type, running state (from active_turns), parent_session_id, enabled extensions, active KBs, and — when a GUI is attached — window/tab placement and focus.
workspace_open Open/focus an existing conversation or start a new one (working_dir, extensions = extension_overrides semantics of POST /agent/start, knowledge_bases, optional first prompt), with placement: tab | split | window and focus (default false — never steal the user's composer).
workspace_read_conversation Structured read of any conversation: view: transcript | tool_calls | summary | spawn_context. tool_calls is a projection of the MessageContent::ToolRequest/ToolResponse items already embedded in content_json — "what did that agent actually do" without transcript noise. spawn_context returns a subagent's exact rendered system prompt, task instructions, and grants.
workspace_send_prompt Inject into another conversation with mode: turn (start its agent on the text; uses the existing one-turn-per-session lock) | steer (existing /interrupt soft-interrupt path, mid-turn) | note (append context without triggering a turn). Optional wait: "final_message" parks the tool call (bounded, ui_ask-style) and returns the target's final message — a synchronous ask-another-agent primitive.
workspace_set_tools Add/remove extensions (wraps the existing runtime-mutable /agent/add_extension + /agent/remove_extension path with persistence) and set active knowledge bases (KnowledgeService::set_active_for_session) on a target session.
workspace_close scope: tab (GUI-only; session survives, matching today's close-tab semantics) | turn (cancel_turn) | agent (stop_agent: cancel + evict).
workspace_spawn_subagent Same parameters as today's subagent tool plus visible: true (default) and placement — the glass-box delegation surface below.

Provenance is structural, not cosmetic: every injected message carries metadata.provenance = { kind: "agent_injection", from_session_id, from_session_name } (extending MessageMetadata) and is rendered with a visible "injected by …" chip in the target transcript. Human interventions in a subagent tab are stamped { kind: "user_direct" }, and the parent's tool-call result reports whether the human intervened.

Part 2 — Backend spine (two additions, useful beyond this feature)

  1. Detached turn runner. Factor the turn-driving loop out of the POST /reply handler so a turn can run server-side with no attached HTTP response (same active_turns lock; events published to the broadcast below). Needed by workspace_send_prompt mode:"turn", workspace_open.new.prompt, and subagent turns.
  2. Per-session event broadcast + observer endpoint. A tokio::sync::broadcast publisher per live session into which every turn (reply-driven, detached, or subagent) publishes its AgentEvents, plus GET /sessions/{id}/events (read-only SSE observer; joins mid-turn with an UpdateConversation snapshot; reuses the existing MessageEvent wire enum so the generated TS client parses it unchanged). This is what lets a subagent tab, a second window, or a parent-watching-child render a turn none of them started. Today, events flow only inside the /reply response that started the turn — that asymmetry is the root blocker.

Session model: add sessions.parent_session_id (sibling of diverged_from); give list_session_summaries an opt-in include_subagents flag (default false, existing behavior preserved); persist the subagent's rendered spawn context as its first message with metadata { user_visible: true, agent_visible: false } so the tab can show it without polluting the child's model context.

Part 3 — Daemon→GUI command channel (WorkspaceBridge)

  • New GET /ui/workspace WebSocket; each Electron window connects once at startup with a stable window_id, authenticated like every other route.
  • A WorkspaceBridge registry modeled on Agent Drafter's proven UiBridge (agent_drafter/control.rs / routes/apps.rs): generation-guarded attach/detach, pending-request map for blocking round trips, cancel_all on disconnect. This is the same "in-process MCP server pushes frames over a socket to a UI, split socket accepts inbound frames" pattern that already works for apps — instantiated at workspace scope.
  • Outbound frames: open_tab / activate_tab / close_tab / open_window / notify / annotate_tab (subagent badge + parent link). Inbound: workspace_echo (debounced layout report: groups, tabs, session bindings, focus) and workspace_result (resolves parked round trips).
  • Renderer side: a workspaceCommandRegistry (same shape as the existing newTabRegistry / closeActiveTabRegistry seams) maps frames onto existing ChatGroups reducer dispatches — openTab already dedupes by session id ("open or focus session X" is one existing dispatch), split maps to moveTabToGroup (refused with a clear message at MAX_GROUPS = 6), window maps to the existing create-chat-window IPC. No parallel tab lifecycle.
  • A tab opened for a session the renderer isn't driving subscribes its ChatStreamController to GET /sessions/{id}/events instead of owning a /reply stream.
  • Headless degradation is a requirement: every tool works with no GUI attached (session-level effect only) and says so in its result (gui_attached: false).

Glass-box subagents (the integration)

With the three parts in place: spawn exactly as today (fresh Agent, parent's provider, extensions minus exclusions, subagent_system.md override, SessionType::SubAgent, active-work registration, recursion guard intact) → run the child through the detached runner so its events are observable → announce open_tab + annotate_tab over the bridge. The tab header shows spawned-by link, expandable spawn context, and the child's extensions/skills/KBs (via the existing GET /sessions/{id}/extensions + KB state). The human steers mid-turn via the existing soft-interrupt path; parent and human can both abort (closing the tab alone never kills the child; an explicit Stop resolves the parent's tool call as Incomplete with partial summary). The parent still receives only the final-summary envelope — transparency lives on the observation plane, not in the parent's context window — and can audit the child with workspace_read_conversation view:"tool_calls", exactly as the human can.

Permissions and safety (must-haves, not nice-to-haves)

  • Extension is off by default; enabling is an explicit user decision with a capability summary. Per .github/copilot-instructions.md, this permission-relevant code requires human review.
  • Manual/smart-approval modes: every mutating workspace_* call is confirmable like any sensitive tool. Autonomous mode: mutations proceed but are always GUI-visible (toasts + provenance chips) — silent cross-session action is not a supported configuration.
  • workspace_read_conversation refuses Hidden sessions; cross-session reads are themselves logged as tool calls in the reading session (auditable by the same projection).
  • Subagents never receive the workspace extension (extension of the existing "subagents cannot create subagents" guard) — no fan-out of workspace control, no child steering its parent.
  • Bounded fan-out: reuse BIOROUTER_SUBAGENT_MAX_CONCURRENT/MAX_INFLIGHT; respect MAX_GROUPS; cap concurrently injected detached turns per caller.
  • workspace_set_tools targeting the caller's own session follows the caller's permission mode exactly as a user-initiated toggle would; removing security-relevant or adding process-spawning extensions on any target always confirms.

System-prompt integration (explicit requirement)

Once the extension is executed, the agent must receive full context on how to use it via the standard server-instructions channel — get_info().instructionsExtensionInfoSystemPromptBuilder → the system.md extension loop — kept within the injection budget (~≤2.5k chars). The instruction block must teach: the workspace model (tabs ↔ sessions), which tool for which job, that injections are permanently provenance-labeled, that subagent tabs are human-interactive and the completion result reports human intervention, the headless degradation, and the routing rules to neighboring tools (below). Per-tool JSON-schema descriptions carry parameter detail.

Please make sure the implementing agent fully considers the existing overlapping tools before building — these reconciliations are binding:

Existing surface Ruling
chatrecall platform extension (agents/chatrecall_extension.rs, FTS5 search + load of past sessions; default_enabled: false) Reuse, never duplicate. Workspace implements no search; instructions route content questions ("what did we conclude about X?") to chatrecall, live control + structured reads to workspace_*. workspace_read_conversation subsumes load-mode via the same SessionManager::get_session read path. BR-17 (cross-session memory) keeps ownership of recall.
GET /sessions/{id}, /reply, /interrupt, /agent/cancel, /agent/stop, /agent/add_extension, /agent/remove_extension, /agent/tools, /sessions/{id}/extensions Reuse — the tools are a thin uniform surface over these exact paths; one storage read path, one turn lock.
active_work registry + routes Reuse & feed — workspace-spawned work registers there too.
Memory MCP server, platform__ingest_conversation, platform__read_session_blob Orthogonal — instructions name them so the model doesn't misroute (facts → Memory; fold-into-KB → ingest; blobs → read_session_blob).
Agent Drafter appcontrol + UiBridge + ui_ask Pattern donor, unchanged — the workspace bridge copies its anatomy at workspace scope.
Agent Drafter consult worker profiles; ACP crate Out of scope, deliberately enabled laterconsult turns could route through the same observation plane; ACP is a natural future transport for workspace observation.

Describe alternatives you've considered

  • Drive the GUI via OS-level automation (computercontroller/AppleScript/Playwright): brittle, platform-specific, invisible to permissions/provenance, and can't work headless. Rejected.
  • Extend Agent Drafter's appcontrol to workspace scope: its bridge is per-app-session and its vocabulary is widget trees; workspace commands are tab/session verbs across windows. Same pattern, separate bridge.
  • SSE-only (no WS) for the GUI channel: the channel needs inbound layout echoes and blocking round-trip results from the renderer; a split WS (as in handle_agent_socket) is the established shape.
  • Making subagent transcripts visible without interactivity (knowledge-loop-style SSE progress only): solves observation but not the core ask — humans must be able to talk to and steer the subagent through the same chat surface.
  • Auto-enabling the extension: rejected on safety grounds; cross-session read/inject/tool-mutation power must be an explicit opt-in.

Additional context

Suggested phasing (each slice ships and verifies independently):

  1. Backend spine + headless tools — detached turn runner; session event broadcast + GET /sessions/{id}/events; parent_session_id; spawn-context persistence; workspace extension with list/read_conversation/send_prompt/set_tools/close (session-level only). Route + platform-extension tests; OpenAPI regen (just generate-openapi && cd ui/desktop && npm run generate-api).
  2. WorkspaceBridge + renderer applier/ui/workspace, per-window registry (reconnect/generation tests modeled on the routes/apps.rs ones), workspaceCommandRegistry + reducer wiring, layout echo, observer-backed ChatStreamController, provenance chips + set-tools toasts.
  3. Glass-box subagents — route subagent execution through the detached runner; workspace_spawn_subagent; auto-open + badge; tab header (spawn context, extensions, KBs); human steer + intervention flag; Stop control; include_subagents History grouping. E2E on the mock-daemon harness pattern (scripts/agent-drafter/ui-control-harness.mjs).
  4. Polish + docs — instruction tuning against real model behavior; user docs (docs/agent-loop/subagents.md, docs/extensions/built-in/workspace.md); tool-routing table update for the chatrecall/workspace split.

Open questions for discussion: focus etiquette (an "announce-only, never auto-open tabs" user setting?); long-term convergence of workspace_send_prompt wait:"final_message" with Agent Drafter consult; cross-window targeting default; observer backpressure/resync cost on very long transcripts; whether Slice 1 should also ship biorouter sessions watch/send CLI commands as free verification tooling.

  • I have verified this does not duplicate an existing feature request

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions