Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
8ea9c66
Allow framing in dor iframe surfaces and fix blank-pane footguns
nedtwigg Jun 12, 2026
bbbff8c
Document dor iframe surface limitations and provisional status
nedtwigg Jun 12, 2026
cf72ab0
Spec the dor agent-browser surface as the iframe alternative
nedtwigg Jun 12, 2026
287d25b
dor ab: passthrough CLI with --key→session translation (Phase 1)
nedtwigg Jun 12, 2026
8d4f671
Wall: surface.agentBrowser control method with session↔surface reuse …
nedtwigg Jun 12, 2026
2725ed2
AgentBrowserPanel: live canvas viewer with stream-native input and ta…
nedtwigg Jun 12, 2026
e9abbbb
Record agent-browser implementation status in plan doc
nedtwigg Jun 12, 2026
9fa38f1
Fix dogfood round 1: input mapping, keyboard focus, host binary resol…
nedtwigg Jun 12, 2026
93b0137
Keyboard passthrough: real VK codes, modifier chords, clipboard paste…
nedtwigg Jun 12, 2026
e9263b4
Draft upstream issue: stream input_keyboard drops CDP commands field
nedtwigg Jun 15, 2026
691994f
agent-browser: emulate macOS native editing chords via host edit channel
nedtwigg Jun 15, 2026
95299aa
Spec: rework viewport section to SYNCED/SCALED indicator + edit-chann…
nedtwigg Jun 16, 2026
76828f4
Didn't mean to commit this.
nedtwigg Jun 16, 2026
00f7536
agent-browser: Phase 8 — SYNCED/SCALED screen indicator + viewport modal
nedtwigg Jun 16, 2026
55bc7d3
agent-browser: display crisp HiDPI screenshots, paced by stream-frame…
nedtwigg Jun 16, 2026
e5f1077
spec: headed pop-out + document the shipped HiDPI-screenshot pipeline
nedtwigg Jun 16, 2026
258cf93
Merge remote-tracking branch 'origin/main' into cli-expanded
nedtwigg Jun 16, 2026
cabab63
spec: describe the built surface; move pop-out to Future Expansions
nedtwigg Jun 16, 2026
297fc6b
agent-browser: don't parse discarded screencast frames
nedtwigg Jun 16, 2026
1b706cd
agent-browser: binary screenshot transport + extract panel modules
nedtwigg Jun 16, 2026
1f8cd0f
stories: AgentBrowserScreenModal (Sync / Device / Custom)
nedtwigg Jun 16, 2026
d684354
agent-browser: icon the screen indicator (FrameCorners=synced, Resize…
nedtwigg Jun 16, 2026
f2d4f88
Remove DOR_AGENT_BROWSER.md implementation plan
nedtwigg Jun 16, 2026
fbae5f8
agent-browser: tokenize the VS Code stream relay (one-use grants)
nedtwigg Jun 16, 2026
dae3e43
spec: propose browser-chrome header (nav + URL + connection)
nedtwigg Jun 16, 2026
72de860
agent-browser: fix stale "base64" screenshot-transport docs (PR #140 …
nedtwigg Jun 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
619 changes: 619 additions & 0 deletions docs/specs/dor-agent-browser.md

Large diffs are not rendered by default.

54 changes: 54 additions & 0 deletions docs/specs/dor-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,57 @@ Invariants:
- Workspace/window refs and target flags will be added only when Dormouse
actually supports them.

## Iframe Surface: Limitations And Status

> Status: **provisional.** The `dor iframe` surface works for displaying a page,
> but has structural limitations (below) that may keep it from shipping. The
> likely path forward is to ship an **agent-browser** surface instead — a
> Dormouse-controlled browser view (see `lib/src/components/wall/AgentBrowserPanel.tsx`)
> that does not embed a foreign page as a peer browsing context — and to drop or
> hide `dor iframe`. That surface is specified in
> [dor-agent-browser.md](dor-agent-browser.md). Do not build features on top of
> the iframe surface until this is resolved.

An `<iframe>` pointed at an arbitrary URL is a **separate browsing context** from
the Wall. Dormouse's input, focus, and attention model assumes a single
same-document context, so the iframe surface conflicts with it in ways that are
inherent to the browser, not bugs we can patch away:

- **Focus leaves Dormouse entirely.** When the iframe gains focus, a focused
cross-origin frame owns the keyboard. Its keystrokes fire in *its* document and
never reach the parent. Dormouse's global shortcuts are a capturing `window`
keydown listener (`lib/src/components/wall/use-wall-keyboard.ts`), so dual-tap-⌘,
pane navigation, split, and kill all go dead until focus returns to the Wall.
The same-origin policy means the parent cannot observe or intercept those keys —
this cannot be fixed, only designed around (e.g. a click-to-interact overlay, or
an accept-focus model with a mouse-driven escape affordance).

- **The app reads iframe-focus as being backgrounded.** Focusing the iframe fires
a `blur` on the parent `window`. Current handlers treat that as the whole app
losing focus: `Wall.tsx` clears cross-session attention, and
`use-window-focused.ts` flips `windowFocused` to `false`, which drives the
active styling (e.g. `SurfacePaneHeader.tsx`'s `isActiveHeader`). The result is
every header/focus-ring goes inactive and attention clears the instant the
iframe is focused. This part *is* fixable (distinguish `document.activeElement`
being one of our own iframes from a real window blur) but is not yet done.

- **No programmatic focus handle.** `focusSession` (`lib/src/lib/terminal-lifecycle.ts`)
only knows xterm terminals in a registry. The iframe pane is not registered, so
`onClickPanel → enterTerminalMode → focusSession(iframeId)` is a no-op: Dormouse
cannot focus the iframe programmatically and cannot tell when it is focused.

- **Some sites refuse to be framed, with no error signal.** Servers that send
`X-Frame-Options` or a CSP `frame-ancestors` directive cannot be embedded at
all, yielding a blank pane. Cross-origin frames do not report load errors to the
embedder (`onError` never fires; `onLoad` fires even for a blocked frame), so the
surface cannot reliably distinguish "loading", "blocked", and "broken". The
current `IframePanel` shows a best-effort stall hint after a timeout only.

- **The VS Code webview must opt into framing.** The webview CSP
(`vscode-ext/src/webview-html.ts`) is `default-src 'none'`; without a `frame-src`
directive every `<iframe>` is blocked outright (blank white pane). A
`frame-src http: https:` allowance is required for the surface to render at all.

## Current Implemented Commands

Implemented commands call private `surface.*` control methods. `surface.list`
Expand Down Expand Up @@ -165,5 +216,8 @@ from `command-detail`.
- `dor read` [impl](../../dor/src/commands/read.ts) [docs](../../dor/test/snapshots/help/read.md)
- `dor kill` [impl](../../dor/src/commands/kill.ts) [docs](../../dor/test/snapshots/help/kill.md)
- `dor iframe` [impl](../../dor/src/commands/iframe.ts) [docs](../../dor/test/snapshots/help/iframe.md)
- `dor agent-browser` / `dor ab` — delegates to the user's `agent-browser`,
rendered in a Dormouse-native surface; see [dor-agent-browser.md](dor-agent-browser.md)
(the chosen alternative to the iframe surface)
- `dor list-panes` [impl](../../dor/src/commands/list-panes.ts) [docs](../../dor/test/snapshots/help/list-panes.md)
- `dor list-pane-surfaces` [impl](../../dor/src/commands/list-pane-surfaces.ts) [docs](../../dor/test/snapshots/help/list-pane-surfaces.md)
29 changes: 28 additions & 1 deletion dor/src/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
type ApplicationText,
type StricliProcess,
} from '@stricli/core';
import { agentBrowserCommand, runAgentBrowserCli } from './commands/agent-browser.js';
import { ensureCommand } from './commands/ensure.js';
import { iframeCommand } from './commands/iframe.js';
import { killCommand } from './commands/kill.js';
Expand All @@ -27,6 +28,10 @@ import type {
} from './commands/types.js';

export type {
AgentBrowserExec,
AgentBrowserExecResult,
AgentBrowserSurfaceRequest,
AgentBrowserSurfaceResponse,
CliEnv,
CliOptions,
CliResult,
Expand Down Expand Up @@ -64,6 +69,7 @@ const COMMANDS = [
readCommand,
killCommand,
iframeCommand,
agentBrowserCommand,
listPanesCommand,
listPaneSurfacesCommand,
] as const satisfies readonly Command[];
Expand All @@ -76,6 +82,7 @@ const ROUTES = {
read: readCommand.command,
kill: killCommand.command,
iframe: iframeCommand.command,
'agent-browser': agentBrowserCommand.command,
'list-panes': listPanesCommand.command,
'list-pane-surfaces': listPaneSurfacesCommand.command,
};
Expand Down Expand Up @@ -122,7 +129,16 @@ interface CaptureProcess extends StricliProcess {
};
}

export async function runCli(argv: string[], options: CliOptions = {}): Promise<CliResult> {
export async function runCli(rawArgv: string[], options: CliOptions = {}): Promise<CliResult> {
const argv = normalizeAgentBrowserAlias(rawArgv);

// `dor ab <args...>` forwards args verbatim to agent-browser, so they must
// never reach stricli's flag parser. Only a bare `--help`/`-h` (or
// `dor help agent-browser`, normalized above) falls through to stricli.
if (argv[0] === 'agent-browser' && !isAgentBrowserHelpInvocation(argv)) {
return runAgentBrowserCli(argv.slice(1), options);
}

const helpTarget = getHelpTarget(argv);
const [commandName, ...args] = rewriteHelpArgv(argv);

Expand All @@ -147,6 +163,17 @@ export async function runCli(argv: string[], options: CliOptions = {}): Promise<
};
}

/** `ab` is the documented short alias for `agent-browser`, in any help form. */
function normalizeAgentBrowserAlias(argv: string[]): string[] {
if (argv[0] === 'ab') return ['agent-browser', ...argv.slice(1)];
if (argv[0] === 'help' && argv[1] === 'ab') return ['help', 'agent-browser', ...argv.slice(2)];
return argv;
}

function isAgentBrowserHelpInvocation(argv: string[]): boolean {
return argv.length === 2 && (argv[1] === '--help' || argv[1] === '-h');
}

type HelpTarget =
| { scope: 'root' }
| { scope: 'command'; commandName: string };
Expand Down
260 changes: 260 additions & 0 deletions dor/src/commands/agent-browser.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,260 @@
/**
* `dor agent-browser` (alias `dor ab`) — near-transparent passthrough to the
* user's own agent-browser binary, plus Dormouse surface management.
*
* Dormouse intercepts exactly two flags: `--key` (managed, workspace-scoped,
* default "default") and `--session` (raw escape hatch). Everything else is
* forwarded verbatim to `agent-browser --session <resolved> <args...>`. After
* a successful forwarded command, the stream WebSocket port is read via
* `stream status --json` and a `surface.agentBrowser` control request asks the
* Wall to create or reuse the browser surface bound to that session.
*
* The stricli command registered here exists only to serve `--help`; real
* invocations are intercepted in `runCli` before stricli parses argv, because
* forwarded agent-browser args (e.g. `open --headed`) must not be parsed as
* dor flags.
*/

import { buildCommand } from '@stricli/core';
import { spawn } from 'node:child_process';
import { existsSync } from 'node:fs';
import type {
CliEnv,
AgentBrowserExecResult,
CliOptions,
CliResult,
Command,
DorCommandContext,
ParseResult,
} from './types.js';
import { fail, requireControlClient, stringParser } from './shared.js';

/** Hardcoded until Dormouse exposes real workspaces; encoded now to avoid a rename. */
const WORKSPACE_ID = '1';

const INSTALL_HINT = 'npm i -g agent-browser';

// agent-browser session names become filesystem paths (socket dir), so `/` is
// not usable as a namespace separator — the daemon fails to start. Dots keep
// the managed namespace readable: dormouse.<workspaceId>.<key>.
const KEY_PATTERN = /^[A-Za-z0-9._-]+$/;

export function sessionForKey(key: string): string {
return `dormouse.${WORKSPACE_ID}.${key}`;
}

export const agentBrowserCommand: Command = {
name: 'agent-browser',
helpPatches: [
{
scope: 'root',
findReplace: ['agent-browser [--key name] [--session name]<TO-EOL>', 'agent-browser [--key name|--session name] [args...]\n'],
},
{
scope: 'command-usage',
findReplace: ['agent-browser [--key name] [--session name]<TO-EOL>', 'agent-browser [--key name|--session name] [args...]\n'],
},
],
command: buildCommand<{ key?: string; session?: string }, [...args: string[]], DorCommandContext>({
docs: {
brief: 'Drive a browser surface via your agent-browser install (alias: dor ab).',
fullDescription: `Forwards all arguments verbatim to your own agent-browser binary and binds the session to a Dormouse browser surface.

dor intercepts exactly two flags:
--key <name> Managed, workspace-scoped browser identity (default "default").
Maps to agent-browser session dormouse.1.<name>.
--session <name> Attach to a raw agent-browser session by its literal name.
Mutually exclusive with --key.

Everything else — subcommands, flags, selectors — is agent-browser's own
command surface. The binary is resolved from PATH (override with
DORMOUSE_AGENT_BROWSER_BIN) and is never bundled; install it with:
${INSTALL_HINT}

After a successful command, dor opens (or reuses) the browser surface bound to
the session: one session is always exactly one surface.

Examples:
dor ab open http://localhost:5173 # key "default"
dor ab --key storybook open http://localhost:6006
dor ab click @e3 # drives key "default"
dor ab --key storybook reload # drives key "storybook"`,
},
parameters: {
flags: {
key: { kind: 'parsed', parse: stringParser, brief: 'Workspace-scoped browser key (default "default").', optional: true, placeholder: 'name' },
session: { kind: 'parsed', parse: stringParser, brief: 'Raw agent-browser session name (mutually exclusive with --key).', optional: true, placeholder: 'name' },
},
positional: {
kind: 'array',
parameter: { parse: stringParser, brief: 'Arguments forwarded verbatim to agent-browser.', placeholder: 'args' },
minimum: 0,
},
},
func: async function (this: DorCommandContext, _flags: { key?: string; session?: string }, ..._args: string[]): Promise<void | Error> {
// runCli intercepts every non-help agent-browser invocation before
// stricli; reaching this func means that interception regressed.
return new Error('internal: agent-browser passthrough was not intercepted');
},
}),
};

interface ResolvedSessionFlags {
/** Managed key; undefined when the caller attached via raw --session. */
key?: string;
session: string;
rest: string[];
}

export function extractSessionFlags(args: string[]): ParseResult<ResolvedSessionFlags> {
let key: string | undefined;
let session: string | undefined;
const rest: string[] = [];

for (let index = 0; index < args.length; index += 1) {
const arg = args[index] ?? '';
const flag = arg === '--key' || arg.startsWith('--key=')
? '--key'
: arg === '--session' || arg.startsWith('--session=')
? '--session'
: null;
if (!flag) {
rest.push(arg);
continue;
}

let value: string | undefined;
if (arg.includes('=')) {
value = arg.slice(arg.indexOf('=') + 1);
} else {
value = args[index + 1];
index += 1;
}
if (!value || value.startsWith('-')) {
return { ok: false, message: `${flag} requires a value` };
}
if (flag === '--key') key = value;
else session = value;
}

if (key !== undefined && session !== undefined) {
return { ok: false, message: '--key and --session are mutually exclusive' };
}
if (key !== undefined && !KEY_PATTERN.test(key)) {
return { ok: false, message: `--key must match ${KEY_PATTERN} (it becomes part of an agent-browser session name)` };
}

if (session !== undefined) {
return { ok: true, value: { session, rest } };
}
const resolvedKey = key ?? 'default';
return { ok: true, value: { key: resolvedKey, session: sessionForKey(resolvedKey), rest } };
}

export async function runAgentBrowserCli(args: string[], options: CliOptions): Promise<CliResult> {
const flags = extractSessionFlags(args);
if (!flags.ok) return fail(flags.message);
const { key, session, rest } = flags.value;

const env = options.env ?? {};
const binary = env.DORMOUSE_AGENT_BROWSER_BIN || 'agent-browser';
const exec = options.execAgentBrowser ?? execAgentBrowserProcess;

let result: AgentBrowserExecResult;
try {
result = await exec(binary, ['--session', session, ...rest]);
} catch (error) {
if (isMissingBinaryError(error)) {
return fail(`agent-browser was not found (looked for '${binary}'). Install it with: ${INSTALL_HINT}`);
}
return fail(error instanceof Error ? error.message : String(error));
}

let stderrSuffix = '';
if (shouldManageSurface(result.exitCode, rest)) {
const client = requireControlClient(options);
// Outside a Dormouse terminal there is no control endpoint; stay a pure
// passthrough rather than nagging about the missing surface.
if (!(client instanceof Error)) {
try {
const status = await exec(binary, ['--session', session, 'stream', 'status', '--json']);
const wsPort = parseStreamPort(status.stdout);
// The Dormouse host (e.g. a GUI-launched VS Code extension host) may
// not share this terminal's PATH, so resolve the binary to an
// absolute path here, where the user's environment is authoritative,
// and pass it along for host-side tab/close commands.
const binaryPath = resolveBinaryPath(binary, env);
await client.agentBrowserSurface({
key,
session,
wsPort,
...(binaryPath ? { binaryPath } : {}),
});
} catch (error) {
stderrSuffix = `Warning: could not open the Dormouse browser surface: ${error instanceof Error ? error.message : String(error)}\n`;
}
}
}

return {
exitCode: result.exitCode,
stdout: result.stdout,
stderr: result.stderr + stderrSuffix,
};
}

function shouldManageSurface(exitCode: number, rest: string[]): boolean {
if (exitCode !== 0 || rest.length === 0) return false;
if (rest.includes('--help') || rest.includes('-h')) return false;
// `close` tears the session down; the Wall notices the stream dropping and
// placeholders the surface, so opening one here would be self-defeating.
const subcommand = rest.find((arg) => !arg.startsWith('-'));
return subcommand !== undefined && subcommand !== 'close';
}

export function resolveBinaryPath(binary: string, env: CliEnv): string | undefined {
if (binary.includes('/') || binary.includes('\\')) return binary;
const pathVar = env.PATH;
if (!pathVar) return undefined;
const isWindows = process.platform === 'win32';
const names = isWindows ? [`${binary}.cmd`, `${binary}.exe`, `${binary}.bat`] : [binary];
for (const dir of pathVar.split(isWindows ? ';' : ':')) {
if (!dir) continue;
for (const name of names) {
const candidate = `${dir}${isWindows ? '\\' : '/'}${name}`;
if (existsSync(candidate)) return candidate;
}
}
return undefined;
}

function parseStreamPort(stdout: string): number | undefined {
try {
const parsed = JSON.parse(stdout) as { port?: unknown; data?: { port?: unknown } };
const port = parsed.data?.port ?? parsed.port;
return typeof port === 'number' && Number.isFinite(port) ? port : undefined;
} catch {
return undefined;
}
}

function isMissingBinaryError(error: unknown): boolean {
return !!error && typeof error === 'object' && (error as { code?: unknown }).code === 'ENOENT';
}

// agent-browser talks to a daemon, so forwarded commands return quickly;
// buffering output until exit keeps this transport-agnostic with runCli's
// captured stdout/stderr at the cost of not streaming long-running output.
function execAgentBrowserProcess(binary: string, args: string[]): Promise<AgentBrowserExecResult> {
return new Promise((resolve, reject) => {
const child = spawn(binary, args, { stdio: ['ignore', 'pipe', 'pipe'] });
let stdout = '';
let stderr = '';
child.stdout.on('data', (chunk: unknown) => { stdout += String(chunk); });
child.stderr.on('data', (chunk: unknown) => { stderr += String(chunk); });
child.on('error', reject);
child.on('close', (code: number | null) => {
resolve({ exitCode: code ?? 1, stdout, stderr });
});
});
}
Loading
Loading