feat: unify image input into read: + an image: attach field (D.1 Phase 2) - #73
Merged
Conversation
…e 2)
Replace the imagePath prompt-step modifier (which self-globbed and row-indexed
images in one field) with a cleaner two-part model:
- `read:` enumerates image files into rows like any other files.
- a new `image:` field on prompt steps takes a templatable path
(e.g. `image: "{{.item.path}}"`) and attaches that file as a vision image.
This composes with forEach and matches how everything else references rows.
Removed: config.ImagePath + Step.HasImages; utils.setImagePath; the
resolveIterations image-count branch; fs.PickImageFile / fs.CountFiles /
fs.globFiles (fs/images.go is now just ImageToBase64). promptbuilder gains
RenderString (BuildPrompt refactored to share buildValues) so the image path
renders against the same row context.
Migrated the vision example to read local images (ships 3 sample .jpg) →
forEach → `image: {{.item.path}}`; dropped the HF download + magick shell steps,
so it's self-contained. Live-verified on qwen2.5vl:3b (3 images → correct
color/shape/chart descriptions, 0 retries).
TDD: promptbuilder RenderString, prompt-step image attach + missing-image
failure, preprocess read-rejects-image; full suite + golangci-lint clean.
…image guard - Fix TestPromptStepRun_AttachesImageFromRowPath on Windows: encode the row path with json.Marshal instead of string-concatenation (backslashes in Windows paths broke the hand-built JSON). This was the CI failure. Post-/simplify (altitude findings): - NewPromptBuilder takes variadic extra template sources for placeholder discovery; the image path is discovered/validated as its own template instead of being string-glued onto the prompt (`prompt+"\n"+image`). Parse errors are now attributed correctly, and image: placeholders are validated at config time (previously runtime-only). - Hoist the 'image only on prompt steps' guard to a top-level type guard next to collect/sourceFormat, so it also rejects image: on shell/transform (was only checked inside the read branch).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unify the image glob into
read:+ a cleanimage:attach fieldimagePathconflated three jobs in one field (self-glob + modulo row-index + attach). Phase 2 splits them into the same model everything else uses:Changed
image:field on prompt steps — a templatable path rendered against the row (promptbuilder.RenderString, sharingbuildValueswithBuildPrompt), then base64-attached via the existingfs.ImageToBase64→data:image/...path.config.ImagePath+Step.HasImages,utils.setImagePath, theresolveIterationsimage-count branch, andfs.PickImageFile/fs.CountFiles/fs.globFiles(fs/images.gois now justImageToBase64).{{.item.path}}loads from the read source.Vision example migrated
Reads local images (ships 3 small sample
.jpg) →forEach→image:— no HF download / magick, fully self-contained. Live-verified on qwen2.5vl:3b: 3 images → correct bar-chart / red-circle+blue-square / green-triangle descriptions with colors + tags, 0 retries.Tests (TDD)
promptbuilder.RenderString; prompt-step image attach from a row path + missing-image failure; preprocess rejectsimage:on read steps; updated the affected existing tests. Full suite +golangci-lintclean.Completes Block D.1 (file-in). Next: D.2 (non-JSONL output: csv/json/md).