Repository navigation
fix(recorder): record Gemini and Cohere embedding replies instead of error fixtures - #624
Open
abhiramshaji-cyber wants to merge 1 commit into
Conversation
…error fixtures
buildFixtureResponse only read OpenAI data[].embedding and a bare
embedding array, so a recorded Gemini embedContent reply
({ embedding: { values } }) or Cohere v2 embed reply
({ embeddings: { float: [[...]] } }) fell through to the proxy_error
fallback and replay served an error instead of the vector.
Read both shapes into the existing { embedding } fixture. Cohere prefers
float, else the first type holding number vectors; base64 only replies
still fall back since replay cannot serve them. Regenerate the A7 golden,
where the two Gemini record rows now save the vector.
commit: |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recording a Gemini or Cohere embedding call saved an error, so replays returned that error instead of the vector; now the vector itself is recorded and replayed.
Hi! Thank you so much for the quick and kind review on #619, it was a real pleasure. While reading
buildFixtureResponsefor that one I noticed the embedding replies from Gemini and Cohere take the same fallback path, so here is a small follow up in the same spirit.What this code is for
In
--recordmode,proxyAndRecordforwards an unmatched request upstream, relays the reply, and saves it as a fixture. For a JSON reply that was not streamed, it hands the parsed body tobuildFixtureResponse, which detects the provider shape. For embeddings it only knows OpenAIdata[].embedding(L1775) and a bareembedding: [...](L1973-L1976).buildFixtureResponsewith its one callerproxyAndRecord, Functions view in ArchitectBefore this PR
A test runs
aimock --recordand calls GeminiembedContentfor "hello".POST /v1beta/models/text-embedding-004:embedContent, no fixture matches, soproxyAndRecordforwards it.{"embedding":{"values":[0.1,0.2]}}.buildFixtureResponsechecksArray.isArray(obj.embedding). Hereembeddingis an object, so it does not match.Could not detect response format from upstreamand saves{ error: { type: "proxy_error" }, status: 200 }.Cohere
POST /v2/embeddoes the same: its reply is{"embeddings":{"float":[[0.1,0.2]]},"response_type":"embeddings_by_type"}, which no branch reads. The A7 golden snapshot pins the Gemini case today ingemini-embedding-record-offandgemini-embedding-record-strict-header-off.After this PR
Same test, same request.
buildFixtureResponsesees thatobj.embedding.valuesis an array and returns{ embedding: values }.gemini-embeddings.tswraps it back as{ embedding: { values } }, so the replayed body is byte for byte what upstream sent.obj.embeddings, preferringfloatand otherwise the first type whose first entry is an array (int8,uint8,binary,ubinary). The replay handler incohere.tsputs it back under every requested type.embeddingreplies still hit their branches first and come out unchanged.flowchart LR classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827 classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px subgraph BEFORE["BEFORE: embedding saved as proxy_error"] direction TB B1["POST embedContent or /v2/embed"] --> B2["proxyAndRecord"] B2 --> B3["buildFixtureResponse"] B3 --> B4{"data array or embedding array?"} B4 -->|"embedding.values or embeddings.float"| B5["proxy_error fixture, replay fails"] B4 -->|"OpenAI data"| B6["embedding fixture"] end subgraph AFTER["AFTER: the vector is recorded"] direction TB A1["POST embedContent or /v2/embed"] --> A2["proxyAndRecord"] A2 --> A3["buildFixtureResponse"] A3 --> A4{"THE CHANGE: embedding.values or embeddings by type?"} A4 -->|"Gemini or Cohere"| A5["embedding fixture, replay serves the vector"] A4 -->|"OpenAI data"| A6["same embedding fixture as before"] end B1 ~~~ A1 class B1,B2,B3,B6 box class A1,A2,A3,A6 box class B4,B5 bad class A4,A5 good linkStyle 3 stroke:#cc0000,stroke-width:5px linkStyle 8 stroke:#008800,stroke-width:5px style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827 style AFTER fill:#f3fff3,stroke:#008800,color:#111827The change
Two small branches in
buildFixtureResponse, right after the bareembeddingcheck: one forobj.embedding.values(Gemini) and one for theobj.embeddingsobject keyed by type (Cohere v2). Both return the existing{ embedding }fixture shape, so neither replay handler changes.The A7 golden is regenerated with
pnpm golden:capture src/__tests__/response-a7-equivalence.test.ts. Only two rows change:gemini-embedding-record-offandgemini-embedding-record-strict-header-offnow save{ "embedding": [0.1, 0.2] }instead of theproxy_errorfixture.gemini-embedding-record-strictis unchanged because strict mode answers 503 before anything is recorded.src/__tests__/recorder-embeddings.test.tsrecords through a stub upstream and replays from a fresh server for each shape: Gemini with values and with an empty array, Coherefloat,int8,int8plusfloat, an empty vector, a two text batch, and abase64only reply. Seven of the eight fail onmain.What changes for a user
embedContentand Cohere/v2/embedcalls now replay the real vector instead of an error./api/embed(which keeps its own json passthrough) and every other reply shape are untouched.embeddingbranches.base64strings still falls back to the error fixture, since the replay handler would serve numbers under thebase64key and could not round trip it.data[0]behaves the same way today, so I kept it consistent. Would you prefer multi vector support for batches? I'd be glad to add it here or in a follow up, whichever you like.Thanks again for all your work on aimock, and any feedback or tips are very welcome!