Problem
The current detector/reader testing system has fundamental limitations:
- Labels in filenames — can't encode multi-dimensional ground truth (species + detector bools + sprites on one image)
- 1 image = 1 detector directory — prevents reuse across detectors/readers, forces duplication
- No comprehensive screen model — detectors added ad-hoc, no shared screen graph for auto-ladder state machine
- Re-labeling is destructive — new detector/reader means re-labeling everything from scratch
Solution
Replace with a manifest-based, screen-centric architecture:
screens.yaml — canonical screen graph defining all 13 screen states, transitions, detector/reader registration with typed field schemas
- Per-screen image directories with
manifest.json storing all reader labels
- Implicit negative testing — images not in a detector's registered screens are automatic negatives
- Web-based labeler on ash dashboard with OCR pre-population via ColePC job runner API
Key Decisions
| Decision |
Choice |
| Image storage |
Per-screen directories (13 screens + 1 overlay) |
| Labels |
manifest.json per directory |
| Negative tests |
Implicit from directory membership |
| Screen graph |
screens.yaml with transitions, detectors, readers, field schemas |
| Singles/doubles |
Separate directories for in-battle screens only |
| C++ test runner |
Add yaml-cpp + nlohmann/json, parse new structure |
| Labeler |
Web UI on ash dashboard |
| OCR suggestions |
Job runner API on ColePC (port 8422) |
| Migration |
Rip and replace CommandLineTests/ |
Full PRD
See plans/test_image_architecture.md for complete specification including:
- Screen inventory and state machine transitions
- screens.yaml and manifest.json schemas
- Directory structure
- Negative test model
- C++ test runner changes
- Web labeler design (inbox, screen, validation views)
- OCR suggestion API spec
Problem
The current detector/reader testing system has fundamental limitations:
Solution
Replace with a manifest-based, screen-centric architecture:
screens.yaml— canonical screen graph defining all 13 screen states, transitions, detector/reader registration with typed field schemasmanifest.jsonstoring all reader labelsKey Decisions
Full PRD
See
plans/test_image_architecture.mdfor complete specification including: