fix(trace): do not capture a conversation as a plan

The spec capture instructions now define a plan as one that comes from
an external source or from the plan mode of the agent. An agreement in
the chat is not captured, a chat follow-up does not count as a change of
a source, and a pasted image is not captured also when the agent has a
file path for it. The agent tells the user in one line when it captures
a file, instead of being told to say nothing. When the agent is not
sure, it captures nothing.

Closes #3580

Assisted-by: Claude Code
Signed-off-by: Miguel Martinez Trivino <miguel@chainloop.dev>

Chainloop-Trace-Sessions: ffad01c1-46ff-482a-8596-c4bc07cdd21f

Author: migmartri

app/cli/internal/trace/README.md

@@ -62,7 +62,7 @@ What the session was asked to build, and which skills it used.
 
 | Feature | What it means | Claude Code | Cursor | OpenCode 1.x | OpenCode 2 |
 |---------|---------------|-------------|--------|--------------|------------|
-| Spec capture instruction | At session start, the agent is told to capture the specs of the session (tickets, documents, plans, images) in the spec folder. | Yes | Yes | Yes | Yes |
+| Spec capture instruction | At session start, the agent is told to capture the specs of the session (tickets, documents, plans, images) in the spec folder. A plan is captured only when it comes from an external source or from the plan mode of the agent: an agreement in the chat is not a plan. A pasted image is not captured, also when the agent has a file path for it. The agent tells the user in one line when it captures a file. | Yes | Yes | Yes | Yes |
 | Spec capture reminder | At each user prompt, the agent is reminded to capture new or changed specs. | Yes | No | Yes | Yes |
 | Local spec sources read at push | A capture whose source is a local file holds only a header and a placeholder. Each push reads the file and records its current content, so the evidence follows the edits of the file. Only a regular text file of up to 1 MiB is read. Otherwise the push keeps the body that the agent wrote, or drops the capture with a warning when there is no body (spec issue-3561). | Yes | Yes | Yes | Yes |
 | Skills tracking | The evidence lists the skills that the session used, with a copy of each skill as it ran. | Yes | No | Partial | Partial |
@@ -75,7 +75,7 @@ Why not Yes:
 - **Skills tracking, Cursor:** the provider does not track skills. The evidence has no skill entries, which means "not recorded", not "no skill used".
 - **Skills tracking, OpenCode:** only the skills that the model starts with the `skill` tool are counted. Skills that the user starts, and skills used in subagents, are not.
 - **Spec source pointers, Cursor and OpenCode:** the provider has no finder for the copies in its transcript yet. For OpenCode, the rebuilt transcript holds no tool outputs, so only the write copies would apply.
-- **Pasted image pointers, Cursor and OpenCode:** the provider has no finder for pasted images yet. Each agent puts pasted images in a different block shape. The pasted images stay inline, and the agent can still capture them in the spec folder.
+- **Pasted image pointers, Cursor and OpenCode:** the provider has no finder for pasted images yet. Each agent puts pasted images in a different block shape. The pasted images stay inline in the transcript. The agent is told not to capture them in the spec folder.
 
 ## Security
 

app/cli/pkg/action/trace_spec.go

@@ -85,14 +85,20 @@ func sessionSpecReminder(repoRoot, sessionID string, log zerolog.Logger) string
 // evidence follows its edits with no help from the agent. Only a remote
 // source still depends on the agent, so the rules to keep it current are
 // about that one.
+//
+// It says that a conversation is not a plan, because an agent left to define
+// "approved plan" itself took a chat agreement for one, wrote the plan itself
+// and rewrote it at each follow-up (issue-3580). The stakes come with a way
+// out for when the agent is not sure, and the agent tells the user what it
+// captured, so that a capture is never hidden from the user.
 func specCaptureInstruction(specDir string) string {
 	return fmt.Sprintf(`Write the specification this session is working from into %[1]s now, in your next tool call, before you start the work.
 
-These files are attested evidence of what this session was asked to build, like a commit. Each push of this session records them. A partial, outdated or summarized capture is a false record.
+These files are attested evidence of what this session was asked to build, like a commit. Each push of this session records them. A partial, outdated or summarized capture is a false record. When you are not sure, capture nothing.
 
-Use your file-writing tool, one call per file — not a shell redirect or heredoc, which a sandboxed or worktree-isolated session refuses. It creates the directory for you.
+Use your file-writing tool, one call per file — not a shell redirect or heredoc, which a sandboxed or worktree-isolated session refuses. It creates the directory for you. Each time you capture a file, tell the user in one line, for example "Captured ticket-eng-1234.md as the task."
 
-Resolve whatever the task points at — a ticket, a design document, an image file, a plan approved in this session — and write its actual text. Not a link to it, not your summary of it. Use whatever tool reaches it: an issue tracker, a URL fetch, a local file. One file per source, named for what it is (ticket-eng-1234.md, design-proposal.md). Begin each file with a frontmatter block like this one, then the text:
+Resolve whatever the task points at — a ticket, a design document, an image file, a plan that the user approved in the plan mode of your agent — and write its actual text. Not a link to it, not your summary of it. Use whatever tool reaches it: an issue tracker, a URL fetch, a local file. One file per source, named for what it is (ticket-eng-1234.md, design-proposal.md). Begin each file with a frontmatter block like this one, then the text:
 
 ---
 kind: ticket
@@ -107,7 +113,7 @@ Set kind to one of ticket, document, image or text. An approved plan, or spec te
 Set role to the purpose of the source. If a source has more than one purpose, pick the role that tells why it is in this session. Use one of these values:
 - task: the item that states the work to do.
 - spec: a document that defines what to build.
-- plan: a plan that the user approved.
+- plan: a plan for the work that comes from an external source (a ticket, a design document, a spec file) or that the user approved in the plan mode of your agent. A conversation is not a plan.
 - reference: supporting material, such as a screenshot or a background document.
 Set title to a short name for the source, such as the ticket title. Set description to one or two sentences that tell what the source holds.
 
@@ -117,11 +123,11 @@ A local text file is the exception to writing text. When the source is a file on
 
 Each push reads the file and records its content at that time, so a later edit of the file needs no new capture.
 
-The transcript of this session already holds the conversation, so the user's request prompt is not a spec: do not capture it. Spec text that the user pastes, such as a ticket or a design document, is a spec.
+Capture only a source that exists outside the chat: a ticket, a document, a file, a web page, spec text that the user pastes, or a plan approved in plan mode. The transcript of this session already holds the conversation. So the user's request prompt is not a spec, and a conversation is not a plan: a change that you and the user agree on in the chat is not a source, also when the user approves it. Never make a capture from the conversation.
 
-An image is the other exception to writing text. If you can reach the image as a file — on disk, or at a URL you can download — copy the file itself into the folder with a copy or download command (cp, curl -o), keeping its extension and adding no frontmatter. Then write the role, title and description lines in a second file next to it. Give that file the name of the image plus .meta.yaml, for example mockup.png.meta.yaml. Do not capture an image pasted into this conversation: you have no file for it.
+An image is the other exception to writing text. If you can reach the image as a file — on disk, or at a URL you can download — copy the file itself into the folder with a copy or download command (cp, curl -o), keeping its extension and adding no frontmatter. Then write the role, title and description lines in a second file next to it. Give that file the name of the image plus .meta.yaml, for example mockup.png.meta.yaml. Do not capture an image pasted into this conversation, also when you have a file path for it, such as a file in the folder where your agent keeps pasted images: the transcript holds it.
 
-For any other source — a ticket, a web page, pasted text, an approved plan — write its full text. If the task changes, or such a source changes, overwrite its file with the current content in the same turn as the change, or add another. Before you push, check that each of these files holds the current full text of its source. If there is nothing to capture — a one-line request, a typo fix, a question, a passing remark — write nothing at all.
+For any other source — a ticket, a web page, pasted text, an approved plan — write its full text. If the task changes, or such a source changes, overwrite its file with the current content in the same turn as the change, or add another. A reply in the chat does not change a source. Before you push, check that each of these files holds the current full text of its source. If there is nothing to capture — a one-line request, a typo fix, a question, a passing remark — write nothing at all.
 
 The files are removed when the session ends. They are git-ignored and will never appear in a commit.`, specDir, spec.Placeholder)
 }
@@ -133,16 +139,17 @@ The files are removed when the session ends. They are git-ignored and will never
 // distract the agent from the task. It repeats the folder and the header
 // format, because a resumed session may not hold the full instruction any
 // more. Like the instruction, it states the stakes and gives a local file the
-// placeholder instead of its text (spec issue-3561).
+// placeholder instead of its text (spec issue-3561), and like the
+// instruction it does not take a conversation for a plan (issue-3580).
 func specCaptureReminder(specDir string) string {
 	return fmt.Sprintf(`Chainloop spec capture. Folder: %s
 These files are attested evidence: a partial or outdated capture is a false record. Capture only in these cases, one file per source. Write each text file with your file-writing tool, starting with frontmatter (kind: ticket, document, image or text; uri: the source, when there is one). Copy an image file as it is, with no frontmatter:
 - The user pasted or gave a new spec: write its full text.
 - The user gave an image as a file path or a URL: copy the file.
-- The user approved a plan: write it as kind text.
+- The user approved a plan in plan mode: write it as kind text. A conversation is not a plan.
 - A spec is a local text file, also one this session writes: set uri to its path. Each push reads the file. Write only this line below the frontmatter: %s
-- A spec that is not a local file changed: overwrite its file with the current full text in the same turn. Check these files before a push.
-Do not capture the user's request prompt or a pasted image. If no case applies, do nothing and say nothing about it.`, specDir, spec.Placeholder)
+- A spec that is not a local file changed: overwrite its file with the current full text in the same turn; a reply in the chat is not a change. Check these files before a push.
+Do not capture the user's request prompt or a pasted image, also one saved to a file. When you capture a file, tell the user in one line. If no case applies or you are not sure, capture nothing.`, specDir, spec.Placeholder)
 }
 
 // readSessionSpecs reads the spec files of a session for a push. Most

app/cli/pkg/action/trace_spec_test.go

@@ -69,10 +69,16 @@ func TestSpecCaptureInstruction(t *testing.T) {
 		{name: "the text, not a link", want: "Not a link to it, not your summary of it", why: "a link is worthless as evidence once the source moves"},
 		{name: "one file per source", want: "One file per source", why: "several sources become several entries, not one blob"},
 		{name: "an image file is copied", want: "copy the file itself into the folder", why: "an image the agent can reach as a file is stored as the image, not as a description"},
-		{name: "a pasted image is not captured", want: "Do not capture an image pasted into this conversation", why: "the agent cannot copy the bytes of a pasted image, and a description of it is not the image"},
-		{name: "an approved plan", want: "a plan approved in this session", why: "a plan the session wrote and the user approved is what the work ran against"},
+		{name: "a pasted image is not captured", want: "Do not capture an image pasted into this conversation", why: "the transcript holds a pasted image, and a description of it is not the image"},
+		{name: "a pasted image file is still pasted", want: "also when you have a file path for it", why: "some agents save a pasted image to a file and give the agent its path (issue-3580)"},
+		{name: "a plan is defined", want: "plan: a plan for the work that comes from an external source (a ticket, a design document, a spec file) or that the user approved in the plan mode of your agent", why: "an agent left to define a plan took a chat agreement for one (issue-3580)"},
+		{name: "a source exists outside the chat", want: "Capture only a source that exists outside the chat", why: "an agreement in chat is in the transcript, and a copy of it is text the agent wrote (issue-3580)"},
+		{name: "a chat agreement is not a plan", want: "a conversation is not a plan", why: "the user approving a change in chat does not make it a plan"},
+		{name: "a chat follow-up is no change", want: "A reply in the chat does not change a source", why: "rewriting a capture after each follow-up turns it into a log of the chat (issue-3580)"},
+		{name: "the way out when unsure", want: "When you are not sure, capture nothing", why: "the stakes must not push the agent into a capture it doubts"},
+		{name: "the user is told", want: "tell the user in one line", why: "a capture the user finds only in the tool calls looks hidden (issue-3580)"},
 		{name: "no request prompt", want: "the user's request prompt is not a spec", why: "the transcript already holds the prompt, so a copy of it adds nothing"},
-		{name: "a pasted spec still counts", want: "Spec text that the user pastes", why: "a ticket or design document pasted into the chat is still a spec"},
+		{name: "a pasted spec still counts", want: "spec text that the user pastes", why: "a ticket or design document pasted into the chat is still a spec"},
 		{name: "a changed spec", want: "overwrite its file with the current content", why: "the push records what is on disk, so the final version replaces its drafts"},
 		{name: "a spec the session writes", want: "a design note outside the repository", why: "a plan the session keeps outside the working tree is a spec that changes"},
 		{name: "each push", want: "Each push of this session records them", why: "the files stay for the whole session, and every push records them"},
@@ -120,6 +126,8 @@ func TestSpecCaptureInstruction(t *testing.T) {
 	// The captures are evidence, not bookkeeping to keep quiet about
 	// (spec issue-3561, R-006).
 	assert.NotContains(t, got, "no need to mention")
+	// An agreement in chat is not an approved plan (issue-3580).
+	assert.NotContains(t, got, "a plan approved in this session")
 }
 
 // TestSpecCaptureReminder pins the reminder given at each user prompt. It goes
@@ -138,15 +146,19 @@ func TestSpecCaptureReminder(t *testing.T) {
 		{name: "the destination", want: specDir, why: "a resumed session may not hold the session-start instruction any more"},
 		{name: "a new spec", want: "pasted or gave a new spec", why: "a spec given after the start must reach the evidence"},
 		{name: "an image file", want: "image as a file path or a URL", why: "an image the agent can reach is copied as it is"},
-		{name: "an approved plan", want: "approved a plan", why: "the plan is what the work ran against"},
+		{name: "a pasted image file is still pasted", want: "a pasted image, also one saved to a file", why: "some agents save a pasted image to a file and give the agent its path (issue-3580)"},
+		{name: "a plan from plan mode", want: "approved a plan in plan mode", why: "the plan is what the work ran against"},
 		{name: "kind text for a plan", want: "kind text", why: "an approved plan has no other kind"},
+		{name: "a conversation is not a plan", want: "A conversation is not a plan", why: "an agreement in chat is in the transcript already (issue-3580)"},
+		{name: "a chat follow-up is no change", want: "a reply in the chat is not a change", why: "rewriting a capture after each follow-up turns it into a log of the chat (issue-3580)"},
+		{name: "the user is told", want: "tell the user in one line", why: "a capture the user finds only in the tool calls looks hidden (issue-3580)"},
 		{name: "a changed spec", want: "overwrite its file with the current full text in the same turn", why: "the push cannot read a remote source, so the agent updates it at the change"},
 		{name: "a local path as the source", want: "set uri to its path", why: "the push reads a local file from its path"},
 		{name: "the placeholder", want: spec.Placeholder, why: "the push fills the body of a local file"},
 		{name: "the stakes", want: "attested evidence", why: "a capture read as bookkeeping gets skipped (spec issue-3561, R-006)"},
 		{name: "a check before a push", want: "before a push", why: "nothing else checks a remote capture"},
 		{name: "no request prompt", want: "Do not capture the user's request prompt", why: "the transcript already holds it"},
-		{name: "the way out", want: "do nothing", why: "most turns have nothing to capture"},
+		{name: "the way out", want: "If no case applies or you are not sure, capture nothing", why: "most turns have nothing to capture, and the stakes must not push the agent into a capture it doubts"},
 		{name: "the tool to use", want: "file-writing tool", why: "a shell heredoc is refused in a worktree-isolated session"},
 		{name: "an image keeps no header", want: "Copy an image file as it is, with no frontmatter", why: "a header written into an image file breaks the image"},
 	}
@@ -161,6 +173,9 @@ func TestSpecCaptureReminder(t *testing.T) {
 	// the full instruction.
 	assert.Less(t, len(got), len(specCaptureInstruction(specDir))/2, "the reminder must stay short")
 	assert.LessOrEqual(t, strings.Count(got, "\n"), 8, "the reminder must stay a few lines")
+
+	// The capture is not hidden from the user (issue-3580).
+	assert.NotContains(t, got, "say nothing")
 }
 
 func TestSessionSpecReminder(t *testing.T) {

pkg/attestation/crafter/materials/aicodingsession/aicodingsession.go

@@ -97,7 +97,9 @@ const (
 	SpecRoleTask = "task"
 	// SpecRoleSpec is a document that defines what to build.
 	SpecRoleSpec = "spec"
-	// SpecRolePlan is a plan for the work that the user approved.
+	// SpecRolePlan is a plan for the work from an external source, or one
+	// that the user approved in the plan mode of the agent. An agreement in
+	// the chat is not a plan.
 	SpecRolePlan = "plan"
 	// SpecRoleReference is supporting material: a screenshot, a mockup, an
 	// example, or a background document.