feat(add-meal): read a meal photo into the multi-item screen (tier 2b) - #650
Conversation
Pulls the endpoint, headers, forced tool call, response parsing and — most importantly — the tool schema out of AnthropicMealTextInterpreter into AnthropicMealItemsApi. The interpreter is now its prompt and nothing else. The schema is the reason. It is the app's provenance guarantee in executable form: it has no macro fields, so a model has nowhere to put a calorie count. That guarantee is only worth having if reviewing it is one question — "did anyone add a field to this schema?" — rather than one question per caller. A second path with its own copy would make it two, and the copies would drift. Also renames MealTextInterpreterException to MealInterpreterException, since nothing about it was ever text-specific. The old name is re-exported from meal_text_interpreter.dart so existing imports are untouched. No behaviour change: all 34 existing tests pass with only the mechanical rename applied to them.
Photograph a plate, get the foods back as review rows. The camera action appears on the multi-item add screen only when an API key is enabled; everything downstream — resolving against Open Food Facts / USDA / BLS, the review rows, the write path — is the pipeline tier 1b already uses. The reader changes, the row logic does not. A photo may count, never measure -------------------------------- The text path can return "100 g" because the user typed it. A photograph states nothing, so a gram figure read off a picture of a plate is estimation wearing the costume of a measurement — and it reaches the review screen looking exactly as confident as a number the user typed. Counting discrete items is a different act: two eggs are two eggs, and the user can see whether the count is right. So the prompt asks for counts only, and the interpreter discards any amount that comes back carrying a unit. Dropping only the *unit* would be worse than useless: an estimated `200 g` of rice stripped to a bare `200` reads downstream as a count, and BulkAddBloc turns a bare count into servings. The quantity goes with it, and the row falls back to the same default an unquantified item gets. That rule is enforced in code, not asked for in the prompt. Mutation- checked both ways: stripping only the unit, and skipping the filter entirely, each fail tests. Failures are shown, not swallowed --------------------------------- ReadMealTextUseCase falls back to the deterministic parser because text always has one underneath it. A photo does not — there is no offline way to turn a picture into food names — so ReadMealPhotoUseCase returns a sealed result instead, and the screen says what happened. A rejected key reads differently from a rate limit: "try again later" is the wrong advice for a wrong key, and following it never stops being wrong. An empty answer stays an answer, not a failure (#647): the model looked and found no food. The photo is never stored ------------------------- Encoded to WebP in memory and sent for one request. Deliberately not UserImageStorage, which this otherwise resembles — sharing that code would mean sharing its file write, and a meal photo landing in the documents directory is a photo the export zip picks up and the user never asked to keep. README privacy table and the settings disclosure updated to match, in all 9 locales. Adds no dependency: image_picker and flutter_image_compress are already here for meal and recipe photos. Refs #599
There was a problem hiding this comment.
Pull request overview
Adds tier 2b support for AI-assisted meal logging by letting the multi-item add screen read a user-selected meal photo (camera/gallery) via Anthropic vision, while preserving the existing “model identifies foods; databases supply nutrition” pipeline and updating privacy/disclosure copy across locales.
Changes:
- Introduces a photo-reading flow (UI entry point, encoder, bloc events/states, and
ReadMealPhotoUseCase) with explicit user-facing failure states (unavailable/auth/transient/unreadable). - Refactors Anthropic request/response/tool-schema handling into a shared
AnthropicMealItemsApiused by both text and photo interpreters. - Updates README + all 9 ARB locales to disclose photo sending behavior and add photo-specific strings; adds extensive new unit/widget tests.
Reviewed changes
Copilot reviewed 31 out of 31 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| test/unit_test/read_meal_text_usecase_test.dart | Updates tests to use the shared MealInterpreterException type. |
| test/unit_test/read_meal_photo_usecase_test.dart | New tests covering ReadMealPhotoUseCase result variants and auth/transient handling. |
| test/unit_test/meal_photo_encoder_test.dart | New tests for extension→media-type mapping and payload size cap. |
| test/unit_test/bulk_add_bloc_test.dart | Extends bloc tests to cover photo reading events, new source enum, and photo error states. |
| test/unit_test/anthropic_meal_text_interpreter_test.dart | Adjusts exception expectations after API refactor. |
| test/unit_test/anthropic_meal_photo_interpreter_test.dart | New tests for vision request shape, shared schema, counts-only enforcement, and failure mapping. |
| test/features/add_meal/presentation/bulk_add_screen_test.dart | Registers AiCredentialStorage/ReadMealPhotoUseCase so UI tests can keep the camera action hidden without a key. |
| README.md | Updates privacy table and AI disclosure to include explicit photo sending + ephemeral claim. |
| lib/l10n/intl_zh.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_uk.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_tr.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_sk.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_pl.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_it.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_en.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_de.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/l10n/intl_cs.arb | Adds photo-reading UI strings + expands AI disclosure to include photos. |
| lib/features/add_meal/util/meal_photo_encoder.dart | New in-memory encoder producing sendable MealPhoto bytes with size/type guards. |
| lib/features/add_meal/presentation/screens/bulk_add_screen.dart | Adds AppBar camera action (key-gated), bottom-sheet disclosure + camera/gallery picker, and photo error rendering. |
| lib/features/add_meal/presentation/bloc/bulk_add_state.dart | Replaces readByModel bool with BulkAddReadSource and adds BulkAddPhotoError + error state. |
| lib/features/add_meal/presentation/bloc/bulk_add_event.dart | Adds ReadMealPhotoEvent + ReadMealPhotoFailedEvent to drive the photo flow through the bloc. |
| lib/features/add_meal/presentation/bloc/bulk_add_bloc.dart | Wires photo events into bloc, maps use case results to UI states, and shares resolve/emit path for all sources. |
| lib/features/add_meal/domain/usecase/read_meal_text_usecase.dart | Switches to catching shared MealInterpreterException. |
| lib/features/add_meal/domain/usecase/read_meal_photo_usecase.dart | New sealed-result use case for photo reading with explicit failure semantics. |
| lib/features/add_meal/domain/meal_text_interpreter.dart | Re-exports shared exception type; removes text-only exception class. |
| lib/features/add_meal/domain/meal_photo_interpreter.dart | New photo interpreter interface + MealPhoto value object; re-exports shared exception type. |
| lib/features/add_meal/domain/meal_interpreter_exception.dart | New shared exception type with transient/auth helpers. |
| lib/features/add_meal/data/anthropic_meal_text_interpreter.dart | Refactors to delegate request/parse/schema to AnthropicMealItemsApi (prompt-only here). |
| lib/features/add_meal/data/anthropic_meal_photo_interpreter.dart | New vision interpreter using shared API + counts-only post-filtering. |
| lib/features/add_meal/data/anthropic_meal_items_api.dart | New shared Anthropic Messages API wrapper: tool schema, forced tool choice, parsing, validation, error mapping. |
| lib/core/utils/locator.dart | Registers ReadMealPhotoUseCase + updates BulkAddBloc factory signature. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| MealTextParseResult _countsOnly(MealTextParseResult result) => | ||
| MealTextParseResult( | ||
| items: [ | ||
| for (final item in result.items) | ||
| if (item.unit == null) | ||
| item | ||
| else | ||
| ParsedMealItem(query: item.query, quantity: null, unit: null), | ||
| ], | ||
| errors: result.errors, | ||
| ); |
There was a problem hiding this comment.
Fixed. You're right on the principle even though the corpus never produced one.
I ran 447 live calls over 149 photos; every quantity that came back was an integer:
1.0:22 2.0:44 3.0:23 4.0:6 5.0:4 6.0:6 9.0:1 12.0:2 32.0:1
Zero fractions. But "the model happens to behave" is exactly the argument I rejected when I kept _countsOnly at all — it has never fired in 447 calls either, and it stays because it is what remains true when a model version changes. Same reasoning applies here, so _countsOnly now drops any quantity that is not a whole number, along with its number rather than just its unit.
Note the check is on the value, not the JSON: 2.0 and 2 are indistinguishable on the wire, so a whole number written as a decimal still counts. Two tests cover both directions, and the fix is mutation-checked.
| static Future<Uint8List?> _rawBytes(String sourcePath) async { | ||
| try { | ||
| return await File(sourcePath).readAsBytes(); | ||
| } catch (_) { | ||
| return null; | ||
| } | ||
| } |
There was a problem hiding this comment.
Fixed — _rawBytes now checks file.length() before reading.
Worth being precise about what this changes: nothing observable. _fitting already rejected oversized bytes, so the outcome was correct before and after. The fix is that an eight-megabyte camera JPEG is no longer pulled into memory purely to be thrown away — and it only ever happens on the fallback path, which is the device with no WebP encoder, i.e. the one least able to absorb the allocation.
I mutation-checked it and the behaviour test passes with the pre-check removed, exactly because the outcome is unchanged. I'm leaving it at that rather than building a filesystem seam to assert "did not allocate"; the test pins the contract and the comment records why the check is where it is.
A 149-image corpus run against the live API turned up JPEGs the provider refuses outright: a photograph carrying Adobe APP14 markers came back 400 on every attempt, while the same picture re-encoded went through. Those landed on "couldn't read the photo, check your connection and try again" — advice that can never work, because nothing about the connection is wrong and the next attempt fails identically. They now land on "couldn't use that image, try another photo", which is the one thing that does work. MealPhotoFailed carried a bool, which had room for exactly two outcomes and this is a third. It now carries MealPhotoFailure, so the bloc switches exhaustively and a fourth outcome cannot be quietly folded into an existing one. Worth noting what this says about MealPhotoEncoder: re-encoding to WebP is not only a size optimisation, it normalises images the provider would otherwise reject. The raw-bytes fallback, taken when a device has no WebP encoder, is the path that can still hit this — and now it fails legibly. Refs #599
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (9)
lib/l10n/intl_de.arb:1164
- This disclosure implies the photo is “nie gespeichert — weder auf diesem Gerät…”. The user is choosing a photo that already exists on-device; the app’s actual guarantee is that it doesn’t store an extra copy (and it won’t be included in exports). Consider rephrasing to avoid claiming the photo is not on the device.
"bulkAddPhotoDisclosureLabel": "Das Foto wird an Anthropic gesendet, um die darauf abgebildeten Lebensmittel zu erkennen. Es wird nur für diese eine Anfrage gesendet und nie gespeichert — weder auf diesem Gerät noch von OpenNutriTracker.",
lib/l10n/intl_en.arb:1242
- This disclosure currently says the photo is “never saved — not on this device…”. However the user is explicitly picking/taking a photo that already exists on-device (camera roll / picker temp file). What the code guarantees is that OpenNutriTracker doesn’t persist an additional copy (so it won’t appear in exports or be shown again). Consider rephrasing to avoid implying the photo is not on the device at all.
"bulkAddPhotoDisclosureLabel": "The photo is sent to Anthropic to identify the foods in it. It is sent for that one request and never saved — not on this device, and not by OpenNutriTracker.",
lib/l10n/intl_cs.arb:1133
- This disclosure says the photo is never saved “ani v tomto zařízení…”. Since the user selects a photo that already exists on the device, the accurate guarantee is that OpenNutriTracker won’t persist an extra copy (and it won’t appear in exports). Consider rephrasing to avoid implying it isn’t on-device.
"bulkAddPhotoDisclosureLabel": "Fotka se odešle společnosti Anthropic, aby rozpoznala potraviny na ní. Odesílá se pouze pro tento jeden požadavek a nikdy se neukládá — ani v tomto zařízení, ani v OpenNutriTrackeru.",
lib/l10n/intl_it.arb:1133
- This disclosure claims the photo is never saved “né su questo dispositivo…”. The user is choosing an existing on-device photo; the app’s guarantee is that it doesn’t store an additional copy (so it won’t be included in exports). Consider rephrasing to avoid implying the photo isn’t on the device.
"bulkAddPhotoDisclosureLabel": "La foto viene inviata ad Anthropic per identificare gli alimenti che contiene. Viene inviata solo per quella richiesta e non viene mai salvata — né su questo dispositivo né da OpenNutriTracker.",
lib/l10n/intl_pl.arb:1133
- This disclosure says the photo is never saved “ani na tym urządzeniu…”. Since the user selects a photo that already exists on the device, the accurate statement is that OpenNutriTracker won’t persist an extra copy (and it won’t show up in exports). Consider rephrasing to avoid implying it isn’t on-device.
"bulkAddPhotoDisclosureLabel": "Zdjęcie jest wysyłane do Anthropic w celu rozpoznania produktów, które się na nim znajdują. Jest wysyłane tylko na potrzeby tego jednego zapytania i nigdy nie jest zapisywane — ani na tym urządzeniu, ani przez OpenNutriTracker.",
lib/l10n/intl_sk.arb:1164
- This disclosure says the photo is never saved “ani v tomto zariadení…”. The user is selecting a photo that already exists on the device; the app’s guarantee is only that it doesn’t store an extra copy (and exports won’t include it). Consider rephrasing to avoid implying the photo is not on-device.
"bulkAddPhotoDisclosureLabel": "Fotka sa odošle spoločnosti Anthropic, aby rozpoznala potraviny na nej. Odosiela sa iba pre túto jednu požiadavku a nikdy sa neukladá — ani v tomto zariadení, ani v OpenNutriTrackeri.",
lib/l10n/intl_tr.arb:1164
- This disclosure implies the photo is never saved “ne bu cihazda…”. Since the user is picking a photo that already exists on the device, the accurate guarantee is that OpenNutriTracker won’t persist an additional copy (and it won’t be included in exports). Consider rephrasing to avoid implying it isn’t on-device.
"bulkAddPhotoDisclosureLabel": "Fotoğraf, içindeki yiyecekleri tanımak için Anthropic'e gönderilir. Yalnızca bu istek için gönderilir ve asla kaydedilmez — ne bu cihazda ne de OpenNutriTracker tarafından.",
lib/l10n/intl_uk.arb:1133
- This disclosure says the photo is never saved “ані на цьому пристрої…”. The user is choosing a photo that already exists on-device; the code’s guarantee is that OpenNutriTracker doesn’t persist an extra copy (and it won’t appear in exports). Consider rephrasing to avoid implying the photo is not on the device.
"bulkAddPhotoDisclosureLabel": "Фото надсилається до Anthropic, щоб розпізнати продукти на ньому. Воно надсилається лише для цього одного запиту й ніколи не зберігається — ані на цьому пристрої, ані OpenNutriTracker.",
lib/l10n/intl_zh.arb:1134
- This disclosure says the photo is never saved “本设备不保存…”. Since the user is selecting a photo that already exists on the device, the accurate guarantee is that OpenNutriTracker won’t store an additional copy (and it won’t be included in exports). Consider rephrasing to avoid implying the photo is not on the device.
"bulkAddPhotoDisclosureLabel": "照片会发送给 Anthropic 以识别其中的食物。仅为该次请求发送,不会保存 — 本设备不保存,OpenNutriTracker 也不保存。",
Driving the flow on a Pixel 6 falsified the disclosure this feature ships with. `image_picker` does not hand back the user's original file: it copies the chosen photo into the app's cache directory and returns that path, and it never cleans the copy up. After one pick the full JPEG was still sitting in cache/, byte for byte. The settings text says the photo is "never saved — not on this device", and the README said the app "never writes it to disk". Both were false, in the one part of this project where a privacy claim is meant to be checkable. MealPhotoEncoder.encodeAndDiscardSource now removes that copy once the photo is encoded, in a finally so it goes whether or not the request succeeded — a photo the provider rejected is exactly as unwelcome on disk as one it read. Re-verified on the device: the picker's temp directory is empty after a read, and no image exists anywhere in the app's storage outside the food-thumbnail cache. The README now describes the mechanism rather than asserting the absence, since "the picker makes a copy and we delete it" is the checkable claim and "nothing is ever written" was not. The documents directory was never involved, so the "not in your exports" half of the claim was true throughout. Refs #599
| switch (reading) { | ||
| case MealPhotoUnavailable(): | ||
| emit(const BulkAddPhotoErrorState(BulkAddPhotoError.unavailable)); | ||
| case MealPhotoFailed(:final failure): | ||
| emit( | ||
| BulkAddPhotoErrorState(switch (failure) { | ||
| MealPhotoFailure.auth => BulkAddPhotoError.auth, | ||
| // The provider will refuse this picture every time, so it lands | ||
| // on "try another photo" rather than on "try again". | ||
| MealPhotoFailure.rejectedImage => BulkAddPhotoError.unreadable, | ||
| MealPhotoFailure.transient => BulkAddPhotoError.transient, | ||
| }), | ||
| ); | ||
| case MealPhotoRead(:final result): | ||
| // An empty list is an answer, not a failure: the model looked and | ||
| // found no food. It lands on the same "nothing to log" message the | ||
| // text path uses rather than on an error, because nothing went | ||
| // wrong — the photo just was not of a meal. | ||
| await _resolveAndEmit( | ||
| result, | ||
| emit, | ||
| usesImperialUnits: event.usesImperialUnits, | ||
| source: BulkAddReadSource.photo, | ||
| ); | ||
| } |
There was a problem hiding this comment.
I don't think this one is right, so I'm leaving the code as it is.
Dart 3 pattern switches do not fall through, and a non-empty case body no longer needs an explicit break. That requirement was Dart 2. Each case here ends implicitly, and the switch is exhaustive over the sealed MealPhotoReadResult — which is the reason it is a sealed hierarchy.
The evidence rather than the language-lawyering:
flutter analyzeis clean on this exact file (and repo-wide).- 1212 tests pass, including four that drive this switch through all three of its branches.
- I built this commit and ran it on a Pixel 6 — photographed a plate, got eight rows back. Code that does not compile does not do that.
Happy to be shown otherwise if there's a specific SDK version where this breaks, but I'd want the analyzer output before changing it.
A photo the model found nothing in showed "Nothing to log yet" — the same line an untouched screen shows. Confirmed on a Pixel 6: the user hands the app a photograph, waits, and gets back a message that reads as though nothing happened. It is worse than merely vague. The notice that says a model did the reading is only drawn when there are rows, so the empty case was the one screen where a machine had answered and nothing on it said so. Adds bulkAddPhotoNoFoodLabel across all nine locales and branches only when the read came from a photo, so the text path keeps its own wording. Parse errors still outrank both — a bad segment explains itself. Covered by two widget tests, one per direction: the photo line appears where it should and stays off the text path. Both mutation-checked. Refs #599
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (1)
lib/features/add_meal/presentation/bloc/bulk_add_bloc.dart:254
- The comment for
MealPhotoReadhandling says an empty photo result lands on the same "nothing to log" message as the text path, but the updatedBulkAddScreennow shows a photo-specific empty-state message (bulkAddPhotoNoFoodLabel) whensource == BulkAddReadSource.photo. Please update this comment to match the actual UI behavior so future changes don’t regress the intended wording split.
// An empty list is an answer, not a failure: the model looked and
// found no food. It lands on the same "nothing to log" message the
// text path uses rather than on an error, because nothing went
// wrong — the photo just was not of a meal.
…sure overclaiming Three findings from review. **A fractional quantity is not a count.** `_countsOnly` dropped amounts carrying a unit but let a bare `1.5` through, which becomes "1.5 serving" downstream — a proportion the model estimated, wearing the confidence of something it counted. A 447-call corpus over 149 photos returned 109 quantities and every one was an integer, so this has never happened. That is the same reason `_countsOnly` itself has never fired, and the same reason to keep it: the guarantee is what stays true when a model version changes. The check is on the value rather than the JSON, since 2 and 2.0 are indistinguishable on the wire. **The disclosure overclaimed.** It said the photo is "never saved — not on this device", but the user picked it out of their own gallery, so of course it is on the device. The claim the app can actually make is that it keeps no copy of its own and the photo never reaches an export. Reworded in all nine locales; the README already said it this way after the Pixel run. **`_rawBytes` loaded oversized files before discarding them.** Now the length is checked first. Nothing observable changes — `_fitting` already rejected them — but the fallback path is the device with no WebP encoder, handling the camera's raw output, and that is the worst place to allocate eight megabytes in order to throw it away. A fourth comment claimed the `switch (reading)` cases fail to compile without an explicit break. That is the Dart 2 rule; Dart 3 pattern switches do not fall through. Left as written — it analyzes clean, four tests drive all three branches, and this commit's build ran on a Pixel 6. Refs #599
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (2)
lib/features/add_meal/util/meal_photo_encoder.dart:41
mediaTypeForPathrejects.jfiffiles even though they are JPEGs. On devices that lack a WebP encoder (the_rawBytesfallback path), a picked.jfifimage would be treated as unreadable even though it could be sent asimage/jpeg. Consider acceptingjfifasimage/jpegto avoid false negatives.
static const _mediaTypes = {
'webp': 'image/webp',
'jpg': 'image/jpeg',
'jpeg': 'image/jpeg',
'png': 'image/png',
lib/features/add_meal/presentation/bloc/bulk_add_bloc.dart:254
- The comment for the
MealPhotoReadbranch says an empty photo result lands on the same "nothing to log" message as the text path, but the UI now shows a dedicated photo-specific empty-state string (bulkAddPhotoNoFoodLabel). Update the comment to match the current user-facing behavior.
// An empty list is an answer, not a failure: the model looked and
// found no food. It lands on the same "nothing to log" message the
// text path uses rather than on an error, because nothing went
// wrong — the photo just was not of a meal.
f44ed68
into
feature/ai-assisted-meal-logging
Tier 2 of #599, in the shape decision #5 leaves available: the model identifies foods, the food databases supply every number. Photograph a plate, get review rows.
Targets
feature/ai-assisted-meal-logging, notdevelop— #648 stays open and untouched.What it does
The camera action appears on the multi-item add screen only when an API key is enabled. Everything downstream of the read — resolving against Open Food Facts / USDA / BLS, the review rows,
amountNeedsCheck, the write path — is the pipeline tier 1b already uses. The reader changes; the row logic does not.A photo may count, never measure
This is the part worth reviewing carefully.
The text path can return
100 gbecause the user typed it. A photograph states nothing, so a gram figure read off a picture is estimation wearing the costume of a measurement — and it arrives in the review screen looking exactly as confident as a number the user typed. Counting discrete items is a different act: two eggs are two eggs, and the user can check the count against the picture.So the prompt asks for counts only, and
_countsOnlydiscards any amount that comes back carrying a unit.Dropping only the unit would be worse than useless:
200 grice200→BulkAddBlocreads a count → 200 servings2eggs22— kept, becomes servings as on the text pathEnforced in code, not requested in a prompt. Mutation-checked both ways — stripping only the unit, and skipping the filter entirely, each fail tests.
Failures are shown, not swallowed
ReadMealTextUseCasefalls back to the deterministic parser because text always has one underneath it. A photo does not — there is no offline way to turn a picture into food names. SoReadMealPhotoUseCasereturns a sealed result and the screen says what happened:An empty answer stays an answer, not a failure (#647): the model looked and found no food.
The photo is never stored
Encoded to WebP q80 / 1024 px in memory and sent for one request. Deliberately not
UserImageStorage, which this otherwise resembles — sharing that code would mean sharing its file write, and a meal photo landing in the documents directory is a photo the export zip picks up and the user never asked to keep.The README privacy table said "only the line you type is sent". That is the app's falsifiable claim, so it changed with the code, along with the settings disclosure, in all 9 locales.
Adds no dependency:
image_pickerandflutter_image_compressare already here for meal and recipe photos.First commit is a refactor, reviewable on its own
AnthropicMealItemsApinow owns the request, the response parsing and the tool schema for both paths. The schema is the provenance guarantee in executable form — no macro fields, so the model has nowhere to put a calorie count. That is only worth having if reviewing it is one question rather than one per caller, and two copies would drift. All 34 existing tests pass with only a mechanical rename applied.Verification
Corpus run: 149 photos, 3 passes, 447 live calls
Real Flickr photographs of meals (Openverse), labelled by search term, in four classes. Run against the real shipping interpreter with the counts-only filter temporarily disabled, so the model's raw output is visible; the filter is then re-applied in analysis.
The provenance rule holds without needing to be enforced
Zero units across 1,066 returned items. The model never once measured.
_countsOnlynever fired in 447 calls. Keep it anyway — it costs nothing and it is what stays true when a model version changes — but the prompt is carrying this on its own today.Where a quantity did appear it was always a genuine count: 2 pancakes, 3 apple slices, 4 blueberries in a yoghurt bowl. It counted discrete items even inside a scene I had labelled uncountable, and still assigned no mass.
The spike is settled
Image block + forced
tool_choicein one request works. #599 listed this as documented only by inference.Where the value actually is: composed plates
A full English breakfast returned 6.47 items per photo at 100% label recall — six searches saved in one shot. That is a materially better case than I gave this tier credit for when I argued it "barely beats typing".
But most rows still arrive with no amount
89.7% of review rows have no quantity (950 of 1,059). Even on the countable class, only 36% of photos produced any count at all. The friction this removes is finding the food, not stating how much — the user still types nearly every number.
Non-food is handled correctly
42 / 45 returned
[]. All three exceptions were one image: an office desk that genuinely has a tomato, bread rolls and drinks on it — the model was right and my label was wrong. A photo of a child's face returned[]and did not describe the person.Stability, measured properly
My first cut said "63% of foods varied between runs", which was wrong — it counted
bread/whole wheat breadandgrapes/green grapesas disagreement. Assumption-free metrics:bread/toastandkebab/grilled meat skewers, so 34% is an upper bound on hallucination, not a measurement of it.Naming drift still matters downstream, because
whole wheat breadandbreadresolve to different database records. It is a weaker failure than inventing a food, not a harmless one.A real bug, found and fixed here
Three calls failed with HTTP 400 — deterministically, always the same image. It is a JPEG carrying Adobe APP14 markers; re-encoded to either JPEG or WebP the identical picture succeeds.
Those were landing on "check your connection and try again" — advice that can never work. Now they land on "try another photo".
MealPhotoFailedcarried a bool with room for two outcomes and this was a third, so it now carries aMealPhotoFailureenum the bloc switches on exhaustively.It also shows
MealPhotoEncoderis doing more than shrinking bytes: re-encoding normalises images the provider would otherwise refuse. The raw-bytes fallback, used when a device has no WebP encoder, is the one path that can still hit this — and it now fails legibly.Latency
p50 1.3 s, p90 1.8 s, max 16.3 s. The max is the one to watch on a phone; the 20 s timeout covers it.
Caveats on this evaluation