Skip to content

fix(recorder): record Gemini and Cohere embedding replies instead of error fixtures - #624

Open
abhiramshaji-cyber wants to merge 1 commit into
CopilotKit:mainfrom
abhiramshaji-cyber:fix/recorder-gemini-cohere-embeddings
Open

abhiramshaji-cyber wants to merge 1 commit into
CopilotKit:mainfrom
abhiramshaji-cyber:fix/recorder-gemini-cohere-embeddings

Conversation

@abhiramshaji-cyber

Copy link
Copy Markdown
Contributor

Recording a Gemini or Cohere embedding call saved an error, so replays returned that error instead of the vector; now the vector itself is recorded and replayed.

Hi! Thank you so much for the quick and kind review on #619, it was a real pleasure. While reading buildFixtureResponse for that one I noticed the embedding replies from Gemini and Cohere take the same fallback path, so here is a small follow up in the same spirit.

What this code is for

In --record mode, proxyAndRecord forwards an unmatched request upstream, relays the reply, and saves it as a fixture. For a JSON reply that was not streamed, it hands the parsed body to buildFixtureResponse, which detects the provider shape. For embeddings it only knows OpenAI data[].embedding (L1775) and a bare embedding: [...] (L1973-L1976).

buildFixtureResponse and its only caller proxyAndRecord
buildFixtureResponse with its one caller proxyAndRecord, Functions view in Architect

Before this PR

A test runs aimock --record and calls Gemini embedContent for "hello".

  1. The client sends POST /v1beta/models/text-embedding-004:embedContent, no fixture matches, so proxyAndRecord forwards it.
  2. Upstream answers 200 with {"embedding":{"values":[0.1,0.2]}}.
  3. buildFixtureResponse checks Array.isArray(obj.embedding). Here embedding is an object, so it does not match.
  4. No later branch matches, so the fallback (L2525-L2537) logs Could not detect response format from upstream and saves { error: { type: "proxy_error" }, status: 200 }.
  5. The next replay of "hello" serves that error instead of the vector.

Cohere POST /v2/embed does the same: its reply is {"embeddings":{"float":[[0.1,0.2]]},"response_type":"embeddings_by_type"}, which no branch reads. The A7 golden snapshot pins the Gemini case today in gemini-embedding-record-off and gemini-embedding-record-strict-header-off.

After this PR

Same test, same request.

  1. Same as above through step 2.
  2. buildFixtureResponse sees that obj.embedding.values is an array and returns { embedding: values }.
  3. The fixture holds the vector. The replay handler in gemini-embeddings.ts wraps it back as { embedding: { values } }, so the replayed body is byte for byte what upstream sent.
  4. For Cohere, the new branch takes the first vector from obj.embeddings, preferring float and otherwise the first type whose first entry is an array (int8, uint8, binary, ubinary). The replay handler in cohere.ts puts it back under every requested type.
  5. OpenAI and bare embedding replies still hit their branches first and come out unchanged.
flowchart LR

classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827
classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px
classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px

subgraph BEFORE["BEFORE: embedding saved as proxy_error"]
direction TB
  B1["POST embedContent or /v2/embed"] --> B2["proxyAndRecord"]
  B2 --> B3["buildFixtureResponse"]
  B3 --> B4{"data array or embedding array?"}
  B4 -->|"embedding.values or embeddings.float"| B5["proxy_error fixture, replay fails"]
  B4 -->|"OpenAI data"| B6["embedding fixture"]
end

subgraph AFTER["AFTER: the vector is recorded"]
direction TB
  A1["POST embedContent or /v2/embed"] --> A2["proxyAndRecord"]
  A2 --> A3["buildFixtureResponse"]
  A3 --> A4{"THE CHANGE: embedding.values or embeddings by type?"}
  A4 -->|"Gemini or Cohere"| A5["embedding fixture, replay serves the vector"]
  A4 -->|"OpenAI data"| A6["same embedding fixture as before"]
end

B1 ~~~ A1

class B1,B2,B3,B6 box
class A1,A2,A3,A6 box
class B4,B5 bad
class A4,A5 good

linkStyle 3 stroke:#cc0000,stroke-width:5px
linkStyle 8 stroke:#008800,stroke-width:5px

style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827
style AFTER fill:#f3fff3,stroke:#008800,color:#111827
Loading

The change

Two small branches in buildFixtureResponse, right after the bare embedding check: one for obj.embedding.values (Gemini) and one for the obj.embeddings object keyed by type (Cohere v2). Both return the existing { embedding } fixture shape, so neither replay handler changes.

The A7 golden is regenerated with pnpm golden:capture src/__tests__/response-a7-equivalence.test.ts. Only two rows change: gemini-embedding-record-off and gemini-embedding-record-strict-header-off now save { "embedding": [0.1, 0.2] } instead of the proxy_error fixture. gemini-embedding-record-strict is unchanged because strict mode answers 503 before anything is recorded.

src/__tests__/recorder-embeddings.test.ts records through a stub upstream and replays from a fresh server for each shape: Gemini with values and with an empty array, Cohere float, int8, int8 plus float, an empty vector, a two text batch, and a base64 only reply. Seven of the eight fail on main.

What changes for a user

  • Recorded Gemini embedContent and Cohere /v2/embed calls now replay the real vector instead of an error.
  • OpenAI embeddings, Ollama /api/embed (which keeps its own json passthrough) and every other reply shape are untouched.
  • An empty vector is recorded as an empty vector, matching the existing OpenAI and bare embedding branches.
  • A Cohere reply that only has base64 strings still falls back to the error fixture, since the replay handler would serve numbers under the base64 key and could not round trip it.
  • A batched Cohere request keeps only the first vector, and replay repeats it for every text. OpenAI data[0] behaves the same way today, so I kept it consistent. Would you prefer multi vector support for batches? I'd be glad to add it here or in a follow up, whichever you like.

Thanks again for all your work on aimock, and any feedback or tips are very welcome!

…error fixtures

buildFixtureResponse only read OpenAI data[].embedding and a bare
embedding array, so a recorded Gemini embedContent reply
({ embedding: { values } }) or Cohere v2 embed reply
({ embeddings: { float: [[...]] } }) fell through to the proxy_error
fallback and replay served an error instead of the vector.

Read both shapes into the existing { embedding } fixture. Cohere prefers
float, else the first type holding number vectors; base64 only replies
still fall back since replay cannot serve them. Regenerate the A7 golden,
where the two Gemini record rows now save the vector.
@pkg-pr-new

pkg-pr-new Bot commented Nov 17, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@624

commit: ea05eb8

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant