feat: improve semantic monitoring and Pi handoffs

CHANGELOG.md

@@ -9,10 +9,12 @@ All notable changes to the `pi-interactive-shell` extension will be documented i
 - Enforce separate policy for extracted Yes/No and comma-qualified Yes/No confirmations, stop pending and later dynamic choices after automation stops, and keep unchanged-screen approvals valid across elapsed/quiet time drift (#72).
 
 ### Added
+- Add one bounded quiet semantic reassessment per inactivity episode, contextual redacted handoffs with content-sensitive dedupe, and a state-bound single-line reply action guarded by fresh runtime state and trusted global permission (#75).
 - Add opt-in goal-driven selection among conservative choices discovered in fresh visible CLI output, with code-owned exact inputs, trusted global allow/ask/deny policy, and a one-time Pi confirmation prompt for `ask` (#71).
 - Add explicitly activated global exact-command policy for launches through `interactive_shell`; deny blocks before construction, ask requires Pi confirmation, and allow proceeds without changing existing-session controls (#71).
 
 ### Security
+- Fail semantic replies closed on stale identity, changed observation, takeover/reload/exit, secret prompts, restricted text, deny/rejection, or unavailable confirmation UI; quiet reassessment remains non-actionable (#75).
 - Fail dynamic terminal choices closed on headless/background operation, unavailable UI, rejection, stale state, reload/takeover/exit, secret or lifecycle content, ambiguity, and evaluator failure. Project and tool configuration cannot weaken the global user policy (#71).
 - Reject file-watch requests that also supply a raw command or structured spawn, so only the generated watcher command can satisfy launch authorization and no unused spawn can create a worktree (#71).
 

README.md

@@ -514,6 +514,8 @@ Example global opt-in (the default is `false`):
 
 Per-session `monitor.semantic` supports: `goal` (optional task context, sent bounded to 1,000 characters), `attention` (built-in events, default `false`), `watches` (safe unique IDs, nonempty conditions, optional threshold 0–1 with default `0.8`), `minIntervalMs` (default `1000`, clamped 250–60,000), `uncertain` (`"continue"` by default or `"notify"`), optional configured `actions`, and optional `dynamicChoices`. Dynamic choices require literal `enabled: true` and a nonblank `goal`; they are unavailable in headless/background supervision and can execute at most once per session. Actions require literal `enabled: true`, 1–10 items, session `maxActions` default 1/max 10, safe unique IDs, descriptions up to 500 characters, exactly one text input (1–2,000 characters, optional `submit`) or strict key array (1–32 keys), encoded bytes up to 4,096, cooldown 0–86,400,000 ms, and per-action executions 1–10. A code-owned process-wide cap permits at most 10 semantic action attempts across all sessions; refused or throwing writes consume an attempt, and the cap is not caller-configurable.
 
+Quiet-state integration adds `quietIntervalMs`: one observe-only reassessment per inactivity episode (default 2,000ms, bounded 250–60,000 and still subject to `minIntervalMs`). It also covers sessions that produce no output. An unchanged quiet observation can notify but cannot execute fixed/dynamic actions or write terminal bytes.
+
 `semanticPermissions` is also the explicit opt-in boundary for commands launched through `interactive_shell`. Omitting the field preserves existing launch behavior. Once present, each raw command or resolved structured-spawn command is matched exactly before PTY, session, process, or worktree creation. `deny` blocks, `ask` requires Pi's confirmation dialog, and `allow` proceeds; an empty array asks for every launch. Unavailable UI, rejection, or dialog failure blocks an `ask`. Query, input, attach, and lifecycle calls for existing sessions are unaffected. This is an `interactive_shell` launch policy, not a shell, Bash, Pi, or operating-system sandbox.
 
 ```json
@@ -597,6 +599,8 @@ interactive_shell({
 
 The extension conservatively extracts a fresh sequential numbered/lettered menu or an explicitly keyboard-navigable menu from the visible viewport. Wrapped descriptions and a single visible selection marker are supported; recognized navigation/cancel help, bounded visual separator chrome, and unsupported custom-input/chat rows may remain visible without becoming executable choices. This foreground-only feature can execute one dynamic choice per session. Jev receives a dedicated choice containing only code-owned opaque visible-option IDs plus `none`; it never receives or supplies terminal input, and dynamic options do not compete with configured actions or action controls. Code resolves the chosen ID to the exact extracted selector or navigation bytes. A two-option menu whose labels are `Yes`/`No`, or those words followed by a comma and qualifier (for example, `Yes, proceed`/`No, go back`), is code-classified as `dynamic-terminal-confirmation`; other supported menus use `dynamic-terminal-choice`. Inline prompts such as `(Y/n)`, free text, unsafe secret/lifecycle menus, ambiguous layouts, and other unsupported interactions produce no dynamic input. Global `jev.semanticPermissions` rules are the trusted policy source for each classification; project/tool configuration cannot add or weaken them. Prefer `ask` for both operation kinds unless a narrower trusted rule is intended. `deny` always blocks; `ask` opens Pi's confirmation dialog and grants one request bound to the session, operation, generation, and observation hash; `allow` skips the dialog. Elapsed/quiet time alone does not invalidate that hash, but terminal, session, or operation changes do. `stop_automation` prevents later and pending dynamic choices for the session. This is not general unattended CLI operation or a command sandbox. Existing configured fixed actions are unchanged and retain their ownership, freshness, secret, cooldown, dedupe, and budget gates.
 
+Semantic events carry a bounded, source-grounded, already-redacted terminal excerpt and a content-sensitive `handoffIdentity`; the same unresolved state dedupes while a new same-type question wakes Pi. For an ordinary input handoff, Pi may send one `semanticReply` with that event's exact session, decision, generation, identity, and one bounded single-line response. Immediately before writing, runtime rechecks ownership, current observation, reload/takeover/exit state, secret state, response restrictions, and trusted global `semantic-reply` permission. Unmatched permission asks, deny wins, and unavailable UI fails closed. Ordinary manual `input` is unchanged.
+
 Inspect semantic decisions with `interactive_shell({ semanticDecisions: true, semanticSessionId: sessionId })`. Inspect delivered events with `interactive_shell({ monitorEvents: true, monitorSessionId: sessionId })`; `monitorStatus: true` returns monitor lifecycle state.
 
 Optional diagnostics write private per-process JSONL journals under Pi's agent directory. Each process owns and bounds its own files, so writers need no shared lock. Records contain only fixed schema/version identifiers, random run and incident IDs, timestamps, bounded timing/token counts, fixed decision/event/outcome categories, and sanitized model names. They never contain terminal text, commands, paths, session IDs, observation hashes, configured goals or watches, credentials, API keys, raw provider responses, or free-form notes. Files are mode `0600`; expired records are excluded from summaries immediately, and old journals are pruned on later writes. Diagnostics add no terminal polling, provider request, notification, action, or automatic tuning.

headless-monitor.ts

@@ -7,6 +7,7 @@ import type { JevClient } from "./jev-client.ts";
 import { SemanticSupervisor } from "./semantic-supervisor.ts";
 import type { SemanticActionRegistry } from "./semantic-actions.ts";
 import type { SemanticChoiceAuthorization } from "./semantic-choice-authorization.ts";
+import type { SemanticReplyBinding } from "./semantic-reply.ts";
 
 export interface MonitorMatchInfo {
 	strategy: MonitorStrategy;
@@ -418,6 +419,9 @@ export class HeadlessDispatchMonitor {
 
 	pauseSemantic(): void { this.semanticSupervisor?.pause(); }
 	resumeSemantic(): void { this.semanticSupervisor?.resume(); }
+	submitSemanticReply(binding: SemanticReplyBinding, response: string, permissionAllowed: () => boolean): { ok: true } | { ok: false; reason: string } {
+		return this.semanticSupervisor?.submitReply(binding, response, permissionAllowed) ?? { ok: false, reason: "semantic-supervision-unavailable" };
+	}
 	rebindSemanticEpoch(isEpochCurrent: () => boolean): void { this.semanticSupervisor?.rebindEpoch(isEpochCurrent); }
 
 	activateBackgroundLifecycle(options: { autoExitOnQuiet: boolean; timeout?: number; onComplete: (info: HeadlessCompletionInfo) => void }): void {

index.ts

@@ -232,7 +232,7 @@ function compileSemanticRuntime(sessionId: string, mode: "hands-free" | "dispatc
 		const diagnostics = jev.diagnostics?.enabled
 			? createSemanticDiagnosticsSession({ config: jev.diagnostics, sessionId, mode, model: jev.model })
 			: undefined;
-		let lastAttentionTriggerId: string | undefined;
+		let lastAttentionIdentity: string | undefined;
 		return {
 			ok: true as const,
 			runtime: {
@@ -245,16 +245,17 @@ function compileSemanticRuntime(sessionId: string, mode: "hands-free" | "dispatc
 					const deliveries: SemanticDeliveryDiagnostic[] = [];
 					for (const candidate of candidates) {
 						if ((candidate.semantic?.kind === "attention" || candidate.semantic?.kind === "uncertain")
-							&& candidate.triggerId === lastAttentionTriggerId) {
+							&& (candidate.semantic.handoffIdentity ?? candidate.triggerId) === lastAttentionIdentity) {
 							deliveries.push({ eventType: diagnosticEventType(candidate), outcome: "suppressed-unchanged" });
 							continue;
 						}
-						const delivered = coordinator.getMonitor(sessionId)?.submitMonitorCandidate(candidate, `${recorded.generation}:${candidate.triggerId}`) === true;
+						const delivered = coordinator.getMonitor(sessionId)?.submitMonitorCandidate(candidate, candidate.semantic?.handoffIdentity ?? `${recorded.generation}:${candidate.triggerId}`) === true;
 						deliveries.push({ eventType: diagnosticEventType(candidate), outcome: delivered ? "delivered" : "suppressed-monitor" });
 					}
 					diagnostics?.recordDecision(recorded, deliveries);
 					if (recorded.kind === "observation") {
-						lastAttentionTriggerId = candidates.find((candidate) => candidate.semantic?.kind === "attention" || candidate.semantic?.kind === "uncertain")?.triggerId;
+						const attention = candidates.find((candidate) => candidate.semantic?.kind === "attention" || candidate.semantic?.kind === "uncertain");
+						lastAttentionIdentity = attention?.semantic?.handoffIdentity ?? attention?.triggerId;
 					}
 				},
 				onDiagnostic: (outcome: "stale-response" | "cancelled-response") => diagnostics?.recordRequest(outcome),
@@ -294,6 +295,22 @@ async function authorizeLaunchCommand(
 	}
 }
 
+async function authorizeSemanticReply(
+	config: InteractiveShellConfig,
+	ctx: Pick<ExtensionContext, "ui"> & { hasUI?: boolean },
+): Promise<{ allowed: true; acceptedDecision: "allow" | "ask" } | { allowed: false; reason: "denied" | "ui-unavailable" | "rejected" }> {
+	const decision = config.jev?.semanticPermissions.evaluate({ kind: "semantic-reply" }) ?? "ask";
+	if (decision === "allow") return { allowed: true, acceptedDecision: "allow" };
+	if (decision === "deny") return { allowed: false, reason: "denied" };
+	if (ctx.hasUI === false || typeof ctx.ui.confirm !== "function") return { allowed: false, reason: "ui-unavailable" };
+	try {
+		const approved = await ctx.ui.confirm("Allow semantic reply?", "Send this one state-bound response to the currently visible terminal prompt?");
+		return approved ? { allowed: true, acceptedDecision: "ask" } : { allowed: false, reason: "rejected" };
+	} catch {
+		return { allowed: false, reason: "ui-unavailable" };
+	}
+}
+
 function describeSemanticMonitor(config: SemanticConfig): string {
 	const watches = config.watches?.map((watch) => watch.id).join(", ") || "none";
 	const actions = config.actions?.enabled === true
@@ -1616,6 +1633,7 @@ export default function interactiveShellExtension(pi: ExtensionAPI) {
 				semanticDiagnosticDays,
 				semanticDiagnosticLimit,
 				semanticIncident,
+				semanticReply,
 				handsFree,
 				handoffPreview,
 				handoffSnapshot,
@@ -1629,13 +1647,38 @@ export default function interactiveShellExtension(pi: ExtensionAPI) {
 				? { text: input, keys: inputKeys, hex: inputHex, paste: inputPaste }
 				: input;
 			const normalizedSpawn = normalizeSpawnRequest(spawn);
-			const hasExistingSessionAction = Boolean(sessionId || sourceId || outputView || attach || listBackground || dismissBackground || monitorEvents || monitorStatus || semanticDecisions || semanticDiagnostics || semanticIncident);
+			const hasExistingSessionAction = Boolean(sessionId || sourceId || outputView || attach || listBackground || dismissBackground || monitorEvents || monitorStatus || semanticDecisions || semanticDiagnostics || semanticIncident || semanticReply);
 			if (outputSelection && hasExistingSessionAction) {
 				return { content: [{ type: "text", text: "outputSelection is launch-only and cannot be combined with an existing-session action." }], isError: true };
 			}
 			if (semanticDiagnostics && semanticIncident) {
 				return { content: [{ type: "text", text: "Choose semanticDiagnostics or semanticIncident, not both." }], isError: true };
 			}
+			if (semanticReply) {
+				if (sessionId || effectiveInput !== undefined || submit) return { content: [{ type: "text", text: "semanticReply cannot be combined with ordinary session input." }], isError: true };
+				const monitor = coordinator.getMonitor(semanticReply.sessionId);
+				const state = coordinator.getSemanticSessionState(semanticReply.sessionId);
+				const event = coordinator.getMonitorEvents(semanticReply.sessionId, { limit: 200 }).events.find((candidate) =>
+					candidate.semantic?.decisionId === semanticReply.decisionId
+					&& candidate.semantic?.generation === semanticReply.generation
+					&& candidate.semantic?.handoffIdentity === semanticReply.handoffIdentity);
+				const decision = coordinator.getSemanticDecisions(semanticReply.sessionId, { limit: 200 }).decisions.find((candidate) => candidate.decisionId === semanticReply.decisionId);
+				if (!monitor || !state || state.status !== "running" || !event || !decision || decision.generation !== semanticReply.generation) return { content: [{ type: "text", text: "Semantic reply binding is unavailable or stale; no input was sent." }], isError: true };
+				if (event.eventType !== "input-required" || event.semantic?.kind !== "attention" || event.semantic.attentionState !== "waiting_input") return { content: [{ type: "text", text: "Semantic replies are limited to ordinary input-required handoffs; no input was sent." }], isError: true };
+				const initialConfig = loadRuntimeConfig(ctx.cwd);
+				const authorization = await authorizeSemanticReply(initialConfig, ctx);
+				if (!authorization.allowed) return { content: [{ type: "text", text: `Semantic reply blocked (${authorization.reason}); no input was sent.` }], isError: true };
+				const result = monitor.submitSemanticReply({ sessionId: semanticReply.sessionId, decisionId: semanticReply.decisionId, observationHash: decision.observationHash, generation: semanticReply.generation }, semanticReply.response, () => {
+					try {
+						const current = coordinator.getSemanticDecisions(semanticReply.sessionId, { limit: 1 }).decisions[0];
+						if (!current || current.observationHash !== decision.observationHash || current.generation !== decision.generation) return false;
+						const currentDecision = loadRuntimeConfig(ctx.cwd).jev?.semanticPermissions.evaluate({ kind: "semantic-reply" }) ?? "ask";
+						return currentDecision === "allow" || (currentDecision === "ask" && authorization.acceptedDecision === "ask");
+					} catch { return false; }
+				});
+				if (!result.ok) return { content: [{ type: "text", text: `Semantic reply rejected (${result.reason}); no input was sent.` }], isError: true };
+				return { content: [{ type: "text", text: `State-bound reply sent to session ${semanticReply.sessionId}; semantic supervision remains active.` }], details: { sessionId: semanticReply.sessionId, decisionId: semanticReply.decisionId } };
+			}
 			const spawnForAction = (command || hasExistingSessionAction) && isEmptySpawnPlaceholder(spawn)
 				? undefined
 				: normalizedSpawn;

package.json

@@ -37,6 +37,7 @@
     "semantic-corpus.ts",
     "semantic-evaluator.ts",
     "semantic-diagnostics.ts",
+	"semantic-reply.ts",
     "selector-corpus.ts",
     "selector-evaluator.ts",
     "output-source-store.ts",