Skip to content

feat: unify image input into read: + an image: attach field (D.1 Phase 2) - #73

Merged
mirpo merged 2 commits into
mainfrom
feat/image-unify-read
Jul 19, 2026
Merged

feat: unify image input into read: + an image: attach field (D.1 Phase 2)#73
mirpo merged 2 commits into
mainfrom
feat/image-unify-read

Conversation

@mirpo

@mirpo mirpo commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Unify the image glob into read: + a clean image: attach field

imagePath conflated three jobs in one field (self-glob + modulo row-index + attach). Phase 2 splits them into the same model everything else uses:

- name: images
  read: ./images/*.jpg          # enumerate files into rows
- name: describe
  forEach: images
  image: "{{.item.path}}"        # attach the current row's file as a vision image
  prompt: Describe this image. Return as JSON.

Changed

  • New image: field on prompt steps — a templatable path rendered against the row (promptbuilder.RenderString, sharing buildValues with BuildPrompt), then base64-attached via the existing fs.ImageToBase64data:image/... path.
  • Removed config.ImagePath + Step.HasImages, utils.setImagePath, the resolveIterations image-count branch, and fs.PickImageFile / fs.CountFiles / fs.globFiles (fs/images.go is now just ImageToBase64).
  • The prompt+image are parsed together for field discovery, so {{.item.path}} loads from the read source.

Vision example migrated

Reads local images (ships 3 small sample .jpg) → forEachimage: — no HF download / magick, fully self-contained. Live-verified on qwen2.5vl:3b: 3 images → correct bar-chart / red-circle+blue-square / green-triangle descriptions with colors + tags, 0 retries.

Tests (TDD)

promptbuilder.RenderString; prompt-step image attach from a row path + missing-image failure; preprocess rejects image: on read steps; updated the affected existing tests. Full suite + golangci-lint clean.

Completes Block D.1 (file-in). Next: D.2 (non-JSONL output: csv/json/md).

mirpo added 2 commits July 18, 2026 17:23
…e 2)

Replace the imagePath prompt-step modifier (which self-globbed and row-indexed
images in one field) with a cleaner two-part model:

- `read:` enumerates image files into rows like any other files.
- a new `image:` field on prompt steps takes a templatable path
  (e.g. `image: "{{.item.path}}"`) and attaches that file as a vision image.

This composes with forEach and matches how everything else references rows.

Removed: config.ImagePath + Step.HasImages; utils.setImagePath; the
resolveIterations image-count branch; fs.PickImageFile / fs.CountFiles /
fs.globFiles (fs/images.go is now just ImageToBase64). promptbuilder gains
RenderString (BuildPrompt refactored to share buildValues) so the image path
renders against the same row context.

Migrated the vision example to read local images (ships 3 sample .jpg) →
forEach → `image: {{.item.path}}`; dropped the HF download + magick shell steps,
so it's self-contained. Live-verified on qwen2.5vl:3b (3 images → correct
color/shape/chart descriptions, 0 retries).

TDD: promptbuilder RenderString, prompt-step image attach + missing-image
failure, preprocess read-rejects-image; full suite + golangci-lint clean.
…image guard

- Fix TestPromptStepRun_AttachesImageFromRowPath on Windows: encode the row
  path with json.Marshal instead of string-concatenation (backslashes in
  Windows paths broke the hand-built JSON). This was the CI failure.

Post-/simplify (altitude findings):
- NewPromptBuilder takes variadic extra template sources for placeholder
  discovery; the image path is discovered/validated as its own template
  instead of being string-glued onto the prompt (`prompt+"\n"+image`).
  Parse errors are now attributed correctly, and image: placeholders are
  validated at config time (previously runtime-only).
- Hoist the 'image only on prompt steps' guard to a top-level type guard
  next to collect/sourceFormat, so it also rejects image: on shell/transform
  (was only checked inside the read branch).
@mirpo
mirpo merged commit 6fc445a into main Jul 19, 2026
3 checks passed
@mirpo
mirpo deleted the feat/image-unify-read branch July 19, 2026 04:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant