spec: add Pi coding agent trace provider design

Define the project-local Pi extension, lifecycle and tool hooks, JSONL v3 parsing, active-branch semantics, and evidence mapping for native Chainloop Trace support.

Refs: #3520

Assisted-by: pi
Signed-off-by: Vibhav Bobade <vibhav.bobde@gmail.com>

Author: waveywaves

docs/specs/003-pi-trace-provider.md

@@ -0,0 +1,154 @@
+---
+status: draft
+owner: waveywaves
+ticket: https://github.com/chainloop-dev/chainloop/issues/3520
+prd:
+---
+
+# Spec 003: Pi coding agent support in Chainloop Trace
+
+## Summary
+
+Chainloop Trace will support [Pi](https://pi.dev) as a coding agent. `chainloop trace init --pi` and `chainloop trace run --pi` will install one project-local Pi extension. The extension will translate Pi session and tool lifecycle events into the existing Chainloop Trace hook protocol. The CLI will copy and parse Pi's persisted JSONL session, attribute file changes, and emit the existing AI coding-session evidence type. No control-plane API or attestation schema changes are required.
+
+## Problem
+
+- Chainloop Trace supports Claude Code, Cursor, and OpenCode, but not Pi. A contribution produced with Pi cannot provide native Chainloop AI coding-session evidence.
+- Pi records a structured session and exposes lifecycle events, but Chainloop does not install an extension, receive those events, discover Pi sessions, or parse Pi JSONL.
+- Contributors using Pi currently need the `skip-ai-session` bypass even when their work otherwise follows the repository's Trace policy.
+- Treating Pi as Claude Code or OpenCode would be incorrect. Pi has its own session tree, extension lifecycle, tool event ordering, and transcript format.
+
+## Goals and Non-Goals
+
+- Goal: Pi users can select Pi during `trace init` or with a `--pi` flag and produce the same AI coding-session evidence type as other providers.
+- Goal: Pi lifecycle and built-in file mutation events use the existing best-effort Trace hook pipeline.
+- Goal: The captured transcript follows the active Pi session branch and includes model, usage, tool, and conversation summaries.
+- Goal: The extension works without corrupting Pi's stdout in interactive, RPC, JSON, or print mode.
+- Non-goal: Change Pi itself or require a Chainloop-maintained Pi fork.
+- Non-goal: Publish or install a separate Pi package. Chainloop owns one generated project extension.
+- Non-goal: Attribute arbitrary custom Pi tools in the first version. Tools outside the documented `write`, `edit`, and `bash` paths remain visible in the transcript but do not get file-line attribution unless they use those built-ins.
+- Non-goal: Capture a transcript for `pi --no-session`. Hooks remain best-effort, but a persisted Pi session is required for session evidence.
+- Non-goal: Change the default Trace provider. Claude Code remains the default when no provider is selected.
+
+## Requirements
+
+### R-001: Provider selection
+
+Pi MUST be registered as the provider `pi`. `trace init` and `trace run` MUST accept `--pi`, and interactive initialization MUST list Pi with the other providers. Existing default-provider behavior MUST remain unchanged.
+
+### R-002: Project-local extension lifecycle
+
+Initialization MUST install exactly one Chainloop-owned file at `.pi/extensions/chainloop-trace.ts`. Installation MUST be idempotent. Pi's `UninstallHooks` MUST remove only that file, preserve sibling Pi files, and tolerate an absent file. `trace run` MUST restore the previous file byte for byte when it finishes.
+
+### R-003: Session lifecycle
+
+For permanent installation, the extension MUST send Pi's session-header `id`, working directory, and persisted session-file path to the existing session-start and session-end hook actions. A tool event MUST also be able to create the session record when session-start was missed, so every recovery-capable tool payload MUST include the same session ID, working directory, and session-file path. Reload, new, resume, fork, and normal shutdown MUST remain safe when hooks repeat. `trace run` MUST omit session-end because it owns final attestation and cleanup after Pi exits. An abrupt termination MUST NOT block Pi or a later Git push.
+
+### R-004: File attribution
+
+The provider MUST provide best-effort attribution for Pi's built-in `write` and `edit` tools with paired before and after hooks. It MUST provide best-effort attribution for `bash` changes with the existing working-tree snapshot path. Relative paths from `event.input.path` MUST be resolved against `ctx.cwd` before they reach Chainloop. Every extension handler that invokes Chainloop, including lifecycle, `tool_call`, and `tool_result`, MUST catch command-spawn, timeout, output-parse, and hook failures. A `tool_call` handler MUST return `undefined`; it MUST never throw or return `{ block: true }`. Hook subprocess stdout and stderr MUST be captured. Snapshot-based line attribution is best-effort when concurrent calls touch the same file.
+
+### R-005: Transcript ownership and discovery
+
+Before parsing, the provider MUST copy the Pi session JSONL into the Trace store's raw-session directory. The reported session-file path MUST take precedence. When hooks did not record a session, discovery MUST find the newest persisted Pi session whose header working directory belongs to the repository. Fallback discovery precedence MUST be `PI_CODING_AGENT_SESSION_DIR`, project `sessionDir`, global `sessionDir`, then `${PI_CODING_AGENT_DIR:-~/.pi/agent}/sessions`. A process-only `--session-dir` cannot be discovered when no hook ran and is an explicit limitation.
+
+### R-006: Pi session tree
+
+The extension MUST append a Chainloop custom marker after `session_tree` so the selected leaf is persisted as the marker's parent. The parser MUST support Pi JSONL session version 3, use the latest valid marker when available, and otherwise fall back to the last persisted tree entry. It MUST follow `parentId` links to the root, reverse that path, preserve entries on the selected branch, ignore abandoned branches, and ignore unknown fields. An invalid header, unsupported or absent version, or malformed interior entry MUST return a useful error. A single truncated final line after a valid v3 header MAY be ignored with a warning.
+
+### R-007: Evidence mapping
+
+The provider MUST emit the existing AI coding-session evidence schema with:
+
+- agent name `pi`;
+- Pi session ID; start time from the session header; end time from the last selected-branch entry, falling back to the header time; and duration as their non-negative difference;
+- primary model and provider from the last assistant message on the selected branch, plus all models used by assistant messages;
+- input, output, cache-read, cache-write, total-token, and cost totals from assistant messages;
+- tool invocation counts from assistant `toolCall` content blocks;
+- user and assistant message counts; and
+- a raw timeline under `RawSession["main"]` for the selected branch, excluding Chainloop's plain custom tree markers while retaining messages that entered model context.
+
+Missing optional usage or agent-version data MUST produce best-effort evidence rather than fail the Git push.
+
+### R-008: Session-start communication
+
+The Pi extension MUST deliver the existing session-start instruction to the model exactly once per persisted session. It MUST scan all session entries, not only the active branch, for its instruction marker and keep an in-memory guard before injecting. In interactive or RPC mode it MUST also show the existing Trace banner to the user. JSON mode MUST remain valid JSONL, and neither JSON nor print mode may receive extension diagnostics on stdout.
+
+### R-009: Compatibility and safety
+
+Pi hooks MUST require no credentials, and local recording MUST NOT depend on network success. Adding Pi MUST NOT change Claude Code, Cursor, or OpenCode behavior, the default provider, the control-plane API, protobufs, or the attestation schema.
+
+## Constraints
+
+- Pi extensions run with the user's operating-system permissions. The generated file MUST contain only the documented Chainloop hook bridge. Chainloop MUST NOT bypass Pi's project-trust mechanism and MUST document trust as an installation prerequisite.
+- Provider instances are stateless registry singletons. Repository and session state MUST stay in the existing Trace store.
+- The extension MUST capture child-process output instead of forwarding it to Pi's stdout. Diagnostics may use stderr or guarded Pi notifications.
+- Pi can execute sibling tool calls concurrently. Hook correlation MUST use Pi's tool-call ID in the extension; it MUST NOT depend on event order. Correlation does not make overlapping working-tree snapshots isolated.
+- Pi may shut down through quit, reload, session replacement, SIGHUP, or SIGTERM. Cleanup MUST be idempotent. SIGKILL and process crashes remain best-effort cases.
+
+## Proposal
+
+`trace init --pi` writes a TypeScript extension into the repository. Pi discovers the extension after the user trusts the project. The extension invokes provider-specific `chainloop trace hook pi` commands and sends JSON on stdin. Chainloop reuses its generic session tracking, snapshots, attribution, commit trailers, pre-push aggregation, and evidence upload.
+
+```mermaid
+sequenceDiagram
+    participant P as Pi extension
+    participant H as chainloop trace hook pi
+    participant S as Trace store
+    participant G as Git pre-push
+
+    P->>H: session-start(session ID, cwd, JSONL path)
+    H->>S: create or resume session record
+    H-->>P: banner and model instruction
+    P->>H: pre-tool-use(write/edit/bash)
+    H->>S: file or worktree snapshot
+    P->>H: post-tool-use(write/edit/bash)
+    H->>S: record attributed line ranges
+    P->>H: session-end
+    H->>S: copy JSONL and mark inactive
+    G->>S: load recorded sessions and attribution
+    G->>G: parse Pi active branch and create evidence
+```
+
+**Extension installation.** Chainloop generates one dependency-free TypeScript file. It uses Pi's documented `session_start`, `session_shutdown`, `session_tree`, `before_agent_start`, `tool_call`, and `tool_result` events. Permanent installation handles `session_shutdown`; the temporary `trace run` installation omits it because `trace run` attests and cleans up after Pi exits. Existing backup and restore logic can therefore treat the extension as one settings file.
+
+**Session identity.** The Pi session manager supplies the session-header `id`, working directory, and optional session-file path. The extension sends those values on every lifecycle event needed for recovery. It keeps a pending session-start instruction in memory and injects one hidden custom message during the first `before_agent_start`. On resume or reload it scans all session entries for that custom message before injecting another. On `session_tree` it appends one custom marker as a child of the selected leaf so external parsing can recover that choice before any later user message.
+
+**Tool attribution.** The extension recognizes `write`, `edit`, and `bash`. It keys pending calls by Pi's tool-call ID because sibling tools may finish out of order. File paths are made absolute from the event context. Chainloop's existing generic handlers snapshot file content for `write` and `edit`, snapshot the working tree for `bash`, and compute the resulting line ranges after the tool result.
+
+**Transcript capture.** A hook-reported session file is copied directly. Fallback discovery checks `PI_CODING_AGENT_SESSION_DIR`, project and global `sessionDir` settings, then Pi's documented default, reads session headers, filters by repository working directory, and chooses the newest match. A process-only `--session-dir` is discoverable only when a hook reports the session file. `--no-session` has no source file to copy and therefore cannot produce session evidence.
+
+**Transcript parsing.** The parser reads the v3 header and tree entries as typed structures while allowing unknown fields. A `session_tree` marker persists an otherwise in-memory branch selection; without one, the last persisted tree entry is the fallback leaf. Following its parent chain excludes abandoned branches without discarding messages that predate compaction. Model, usage, cost, tool, and conversation summaries are aggregated from selected-branch messages. A truncated final line is skipped with a warning, while an invalid header, unsupported version, or malformed interior entry fails that session cleanly.
+
+**User and model messages.** The provider returns one JSON response from session-start. The extension shows its banner through Pi UI only when UI exists and injects its instruction into model context once. The provider initially reports post-command user announcements as unsupported; adding pending-link delivery later does not block session evidence.
+
+## Decision Record
+
+| ID | Decision | Choice | Why (and what we rejected) | Source |
+|----|----------|--------|----------------------------|--------|
+| D-001 | Pi integration point | Project-local TypeScript extension | Pi documents lifecycle and tool events. We rejected a Pi fork and transcript file watcher because both add more moving parts and a watcher cannot reliably observe the live active branch. | drafting |
+| D-002 | Installed artifact | `.pi/extensions/chainloop-trace.ts` | Pi auto-discovers this trusted project location, and one Chainloop-owned file fits existing backup, restore, and uninstall behavior. | Pi extension docs |
+| D-003 | Provider identifier | `pi` | Short, stable, and matches the command and product name. | drafting |
+| D-004 | Session identifier | Pi session-header `id` | It is stable across process restart and available through the session manager. We rejected a Chainloop-generated parallel identifier. | Pi session format |
+| D-005 | File attribution scope | Built-in `write`, `edit`, and `bash` | These have documented event payloads and map to existing snapshot paths. Arbitrary custom tools have no common file-mutation contract. | Pi extension docs |
+| D-006 | Transcript source | Copied Pi JSONL v3 | The persisted session is authoritative and keeps provider/model/usage/tool data. We rejected an extension-maintained duplicate transcript. | Pi session format |
+| D-007 | Branch selection | Latest Chainloop tree marker, otherwise last persisted entry, then `parentId` to root | The marker persists `/tree` selection before later work. Parsing all entries would mix abandoned branches. | Pi session manager |
+| D-008 | Session-start instruction | Hidden custom message injected once | It persists in the session and reaches the model without requesting a separate turn. The extension scans all entries so an abandoned branch cannot cause reinjection. | Pi extension docs |
+| D-009 | Headless behavior | Hooks run; UI output is omitted | JSON stdout must stay valid JSONL, and print stdout must not receive extension diagnostics. | Pi mode docs |
+| D-010 | Evidence schema | Existing AI coding-session material | Provider differences are represented inside the existing model. No server contract change is needed. | Chainloop provider architecture |
+
+## Open Questions
+
+None.
+
+## Risks
+
+| Risk | Mitigation |
+|------|------------|
+| Pi changes an extension event or JSONL field. | Parse typed required fields, ignore unknown fields, keep golden fixtures, and test the generated extension against a documented Pi version. |
+| A user has not trusted the repository, so Pi does not load the extension. | `trace init` next steps explicitly name the generated file and Pi's project-trust requirement. Non-interactive Pi must have saved trust or run with `--approve`. |
+| Pi is killed before session-end. | Every tool event can establish the session, pre-push copies again, and inactive state is advisory. |
+| A custom tool changes files without `write`, `edit`, or `bash`. | The transcript still records the tool. File-line attribution for arbitrary custom tools is a documented non-goal for the first version. |
+| Parallel tools modify the same file. | Tool-call IDs correlate pairs, but snapshot attribution remains best-effort for overlapping mutations. Tests cover interleaved completion without assuming isolated per-call diffs. |
+| `/tree` changes the in-memory leaf without immediately appending an entry. | The extension appends a custom marker whose parent is the selected leaf, making the choice recoverable before later work. |
+| `--no-session` supplies no transcript file. | Do not block Pi; document that persisted sessions are required for Trace session evidence. |