feat: write file output, config-relative paths, SGR routing support

README.md

@@ -165,6 +165,41 @@ Always wrap jq programs in single quotes: unquoted YAML silently truncates at `#
 
 jq programs are validated when the config loads. Transform steps run instantly, produce regular JSONL, and don't trigger the external-CLI warning. See the [dataset-pipeline example](./examples/v1/dataset-pipeline/README.md), which uses fan-out and fan-in.
 
+### Local Files: `read` and `write`
+
+Process your own data end to end — no shell glue:
+
+```yaml
+steps:
+  - name: leads
+    read: ./leads.csv          # local files → rows
+  - name: classified
+    forEach: leads
+    prompt: Classify {{.item.company}}
+    jsonSchema: { ... }
+  - name: report
+    from: classified
+    write: ./enriched.csv       # rows → a deliverable file
+```
+
+- **`read:`** turns local files into rows. Format is inferred from the path (overridable with `format:`):
+  - a glob / directory / `.txt` / `.md` → one row per file: `{path, name, content}`
+  - `.csv` / `.tsv` → one row per record (columns become fields)
+  - `.jsonl` → one row per line
+- **`write:`** exports a step's rows to a file, format inferred from the extension: `.csv`, `.json` (array), `.md` (table), or `.jsonl`. It's terminal and doesn't change the intermediate JSONL that other steps read.
+- **`image:`** on a prompt step attaches a file as a vision image, e.g. `image: "{{.item.path}}"` after `read`-ing a folder of images.
+- Relative `read`/`write` paths resolve **relative to the config file's directory** (so a workflow's data travels with it and runs from any working directory); absolute paths are used as-is. See the [csv-enrichment](./examples/v1/csv-enrichment/README.md) and [process-my-files](./examples/v1/process-my-files/README.md) examples.
+
+### Schema-Guided Reasoning (SGR)
+
+Shape the JSON schema to steer *how* the model reasons, not just what it returns. Three patterns ([background](https://abdullin.com/schema-guided-reasoning/)):
+
+- **Cascade** — put reasoning before the conclusion (a `reasoning` field, or a `steps[]` array, ahead of the answer). The workhorse; works well even on small local models. See [sgr-reasoning](./examples/v1/sgr-reasoning/README.md), [document-classification](./examples/v1/document-classification/README.md), [inbox-triage](./examples/v1/inbox-triage/README.md).
+- **Routing** — a discriminated union (`anyOf` of object branches, each with a `const` discriminator) makes the model pick one branch and fill only its fields; datamatic validates the union and every branch for strict output. Branch-choice accuracy needs a capable model — small local models (≤3B) reliably mis-route, so use a cloud model for real routing.
+- **Cycle** — an `array` of a repeated sub-schema (optionally bounded with `minItems`/`maxItems`) emits N reasoning items. Bounds are honored by Ollama's grammar but rejected by OpenAI strict mode.
+
+Prompt steps send the schema as strict structured output (all properties required, `additionalProperties: false`); `datamatic validate` flags schemas that break those rules before you hit the API.
+
 ### Environment Variables
 
 Configure your pipelines dynamically using `$VAR` syntax:
@@ -288,6 +323,7 @@ See [`examples/v1/`](./examples/v1/) for the full feature matrix. Start with `ba
 | Example | Features shown | Backend |
 | --- | --- | --- |
 | [process-my-files](./examples/v1/process-my-files/README.md) | `read` local files (glob/dir/CSV/JSONL) into rows | Ollama |
+| [csv-enrichment](./examples/v1/csv-enrichment/README.md) | `read` CSV → LLM enrich → `write` CSV (full office loop) | Ollama |
 | [external-data](./examples/v1/external-data/README.md) | HuggingFace download + transform + shell tools | Ollama |
 | [env-and-workdir](./examples/v1/env-and-workdir/README.md) | env vars, `workDir`, `$PROVIDER`, DuckDB | Ollama |
 | [vision](./examples/v1/vision/README.md) | image → structured output (`imagePath`) | Ollama, LM Studio |

config/config.go

@@ -47,6 +47,7 @@ const (
 	ShellStepType     StepType = "shell"
 	TransformStepType StepType = "transform"
 	ReadStepType      StepType = "read"
+	WriteStepType     StepType = "write"
 	UnknownStepType   StepType = "unknown"
 )
 
@@ -61,6 +62,13 @@ const (
 	ReadFormatJSONL = "jsonl" // one row per line (parsed JSON)
 )
 
+const (
+	WriteFormatCSV      = "csv"   // one record per row; keys become columns
+	WriteFormatJSON     = "json"  // a single pretty-printed JSON array of all rows
+	WriteFormatMarkdown = "md"    // a Markdown table
+	WriteFormatJSONL    = "jsonl" // one JSON value per line (passthrough)
+)
+
 type Step struct {
 	Type           StepType    `yaml:"type,omitempty"`
 	Name           string      `yaml:"name"`
@@ -69,8 +77,9 @@ type Step struct {
 	Run            string      `yaml:"run"`
 	JQ             string      `yaml:"jq"`           // transform steps: jq program
 	Read           string      `yaml:"read"`         // read steps: file/glob/dir to load as rows
-	Format         string      `yaml:"format"`       // read steps: "files" | "csv" | "jsonl" (default: by extension)
-	From           string      `yaml:"from"`         // transform steps: source step name
+	Write          string      `yaml:"write"`        // write steps: file path to export the source rows to
+	Format         string      `yaml:"format"`       // read: "files"|"csv"|"jsonl"; write: "csv"|"json"|"md"|"jsonl" (default: by extension)
+	From           string      `yaml:"from"`         // transform/write steps: source step name
 	Limit          int         `yaml:"limit"`        // transform steps: cap output rows (0 = no cap)
 	Collect        bool        `yaml:"collect"`      // transform steps: jq sees an array of ALL source rows (fan-in)
 	SourceFormat   string      `yaml:"sourceFormat"` // transform steps: "jsonl" (default, line per row) or "json" (whole file is one value)

examples/v1/README.md

@@ -6,6 +6,8 @@ Each folder is a self-contained `config.yaml` + `README.md`. New to datamatic? R
 | --- | --- | --- |
 | [basics](./basics) | text generation, JSON schema | Ollama |
 | [process-my-files](./process-my-files) | `read` your own local files (glob/dir/CSV/JSONL) → rows | Ollama |
+| [csv-enrichment](./csv-enrichment) | `read` CSV → enrich with LLM → `write` CSV (the full office loop) | Ollama |
+| [inbox-triage](./inbox-triage) | `read` folder of emails → SGR triage → draft replies → `write` CSV + Markdown | Ollama |
 | [linked-steps](./linked-steps) | step chaining, native template values (`if`/`range`/`len`) | Ollama |
 | [structured-extraction](./structured-extraction) | nested schema, both schema formats (YAML / JSON-string), native templates | Ollama |
 | [dataset-pipeline](./dataset-pipeline) | transform fan-out, fan-in (`collect`, `$parent`), rating pipeline | Ollama |

examples/v1/csv-enrichment/README.md

@@ -0,0 +1,25 @@
+# CSV enrichment
+
+The full office loop with no shell: **read a CSV → enrich each row with an LLM → write a CSV**. Point it at your own spreadsheet of leads/rows to add labeled columns.
+
+**Features:** `read` · `forEach` · `jsonSchema` · `write`
+
+## Steps
+
+1. `leads` — `read: ./leads.csv` → one row per record (columns become fields)
+2. `classified` — `forEach` lead → `{company, industry, size}`
+3. `report` — `write: ./enriched.csv` → the enriched rows as CSV
+
+Output format is inferred from the extension (`.csv`); use `.json` for a JSON array, `.md` for a Markdown table, or set `format:` explicitly.
+
+## Requirements
+
+- `datamatic`
+- [Ollama](https://ollama.com/download) + `ollama pull qwen3:1.7b`
+
+## Run
+
+```bash
+datamatic --config ./config.yaml --verbose
+cat ./enriched.csv
+```

examples/v1/csv-enrichment/config.yaml

@@ -0,0 +1,33 @@
+version: 1.0
+
+# The full office loop, no shell: read a CSV, enrich each row with an LLM,
+# write a CSV back out.
+steps:
+  - name: leads
+    read: ./leads.csv
+
+  - name: classified
+    model: ollama:qwen3:1.7b
+    forEach: leads
+    modelConfig:
+      temperature: 0.2
+    prompt: |
+      Company: {{.item.company}} (website: {{.item.website}}).
+      Classify its industry and likely size. Echo the company name back.
+      Return as JSON.
+    jsonSchema:
+      type: object
+      properties:
+        company:
+          type: string
+        industry:
+          type: string
+        size:
+          type: string
+          enum: [startup, smb, enterprise]
+      required: [company, industry, size]
+      additionalProperties: false
+
+  - name: report
+    from: classified
+    write: ./enriched.csv

examples/v1/csv-enrichment/leads.csv

@@ -0,0 +1,4 @@
+company,website
+Acme Robotics,acme-robotics.example
+Globex Financial,globex.example
+Initech Software,initech.example

examples/v1/inbox-triage/README.md

@@ -0,0 +1,30 @@
+# Inbox triage
+
+A real support-desk loop, no shell: **read a folder of incoming emails →
+classify each with schema-guided reasoning → draft a suggested reply → write a
+triage board (CSV) and a drafts digest (Markdown)**. Drop your own `.txt`
+emails into `inbox/` and rerun.
+
+**Features:** `read` (folder of files) · `SGR` · `forEach` · `transform` · `write` (csv + md)
+
+## Steps
+
+1. `emails` — `read: ./inbox/*.txt` → one row per file (`{path, name, content}`)
+2. `triage` — `forEach` email → SGR `{reasoning, subject, category, priority, sentiment, summary}`
+3. `board_rows` — **transform** drops the reasoning, keeping the scannable columns
+4. `board` — `write: ./board.csv` → the triage board
+5. `drafts` — `forEach` triage row → `{subject, reply}` (drafted from the summary, not the raw email)
+6. `reply_digest` — `write: ./replies.md` → the suggested replies as a Markdown table
+
+## Requirements
+
+- `datamatic`
+- [Ollama](https://ollama.com/download) + `ollama pull qwen3:1.7b`
+
+## Run
+
+```bash
+datamatic --config ./config.yaml --verbose
+cat ./board.csv
+cat ./replies.md
+```

examples/v1/inbox-triage/config.yaml

@@ -0,0 +1,82 @@
+version: 1.0
+
+# Support inbox triage — the full office loop, no shell:
+# read a folder of incoming emails → classify each with SGR reasoning →
+# shape a scannable board → draft a suggested reply → write CSV + Markdown.
+steps:
+  - name: emails
+    read: ./inbox/*.txt
+
+  - name: triage
+    model: ollama:qwen3:1.7b
+    forEach: emails
+    modelConfig:
+      temperature: 0.2
+    prompt: |
+      You are a support-desk triage assistant. Read the incoming email, reason
+      through it first, then classify it.
+
+      Email ({{.item.name}}):
+      {{.item.content}}
+
+      Return as JSON.
+    jsonSchema:
+      type: object
+      properties:
+        reasoning:
+          type: string
+          description: "Brief reasoning behind the classification (SGR)."
+        subject:
+          type: string
+          description: "Copy the email's `Subject:` line verbatim."
+        category:
+          type: string
+          enum: [bug, billing, feature_request, account, other]
+        priority:
+          type: string
+          enum: [low, medium, high, urgent]
+        sentiment:
+          type: string
+          enum: [positive, neutral, negative]
+        summary:
+          type: string
+          description: "One sentence: what the customer wants."
+      required: [reasoning, subject, category, priority, sentiment, summary]
+      additionalProperties: false
+
+  # shape the board: drop the SGR reasoning, keep the scannable columns
+  - name: board_rows
+    from: triage
+    jq: '{subject, category, priority, sentiment, summary}'
+
+  - name: board
+    from: board_rows
+    write: ./board.csv
+
+  - name: drafts
+    model: ollama:qwen3:1.7b
+    forEach: triage
+    modelConfig:
+      temperature: 0.4
+    prompt: |
+      Draft a short, friendly support reply. Acknowledge the issue and give one
+      clear next step. Keep it under 120 words.
+
+      Subject: {{.item.subject}}
+      Category: {{.item.category}}
+      Customer wants: {{.item.summary}}
+
+      Return as JSON.
+    jsonSchema:
+      type: object
+      properties:
+        subject:
+          type: string
+        reply:
+          type: string
+      required: [subject, reply]
+      additionalProperties: false
+
+  - name: reply_digest
+    from: drafts
+    write: ./replies.md

examples/v1/inbox-triage/inbox/01-locked-out.txt

@@ -0,0 +1,11 @@
+From: dana.reyes@brightsail.io
+Subject: Locked out after the weekend maintenance
+
+Hi support,
+
+Since your Sunday maintenance window I can't log in to the dashboard at all.
+I reset my password twice, but after entering the code from the email the page
+just spins and eventually says "session expired". Three people on my team hit
+the same wall this morning and we have a client demo at 2pm. Please help ASAP.
+
+Dana

examples/v1/inbox-triage/inbox/02-double-charge.txt

@@ -0,0 +1,11 @@
+From: marcus@nordfeld-consulting.com
+Subject: Charged twice for the October invoice
+
+Hello,
+
+I was billed EUR 240 twice on October 3rd for the Team plan, but I only have a
+single subscription. Could you refund the duplicate charge and confirm it will
+not happen again? The two invoice numbers are INV-4471 and INV-4472.
+
+Thanks,
+Marcus

examples/v1/inbox-triage/inbox/03-feature-wishlist.txt

@@ -0,0 +1,11 @@
+From: priya.n@quantaleap.dev
+Subject: Any plans for dark mode or outbound webhooks?
+
+Hey team,
+
+Really enjoying the product so far. Two things on our wishlist: a dark theme for
+the console (we stare at it all day) and outbound webhooks so we can push events
+into our own Slack. Neither is urgent, just wanted to put them on your radar.
+
+Cheers,
+Priya

examples/v1/inbox-triage/inbox/04-thanks-and-seats.txt

@@ -0,0 +1,11 @@
+From: t.oyelaran@meridianhealth.org
+Subject: Thank you, and we would like to add seats
+
+Hi,
+
+Just wanted to say the new reporting export saved us hours this quarter, great
+work. On a related note, we would like to add 5 more seats to our account. What
+is the easiest way to do that mid-cycle?
+
+Best,
+Tunde