feat(apis): OpenAI Chat Completions → Vertex AI Gemini translation

[noalimoy]

Summary

Add an openai_chat_completions_to_vertexai_gemini HttpFilter under
apis/src/vertex/gemini/ that lets standard Chat Completions clients
talk to Vertex AI Gemini models through Praxis. Gemini uses a completely
different JSON schema — the filter does full structural translation in
both directions.

• Request (request.rs): converts messages → Gemini contents
  with systemInstruction extraction, maps all OpenAI roles to Gemini
  equivalents, translates tools → functionDeclarations /
  tool_choice → toolConfig, builds generationConfig, and
  resolves tool_call_id → function name for functionResponse parts.
  Rewrites path to the Vertex AI generateContent (or
  streamGenerateContent?alt=sse) endpoint.
• Response (response.rs): translates Gemini candidates → Chat
  Completions choices, maps finish reasons, and extracts
  usageMetadata → usage. Handles both non-streaming (full JSON)
  and streaming (per-SSE-frame chat.completion.chunk delta).
• SSE streaming (mod.rs): reuses the shared SseFrameParser for
  cross-chunk reassembly, translates each Gemini frame inline, and
  emits data: [DONE] on stream close (Gemini does not send one;
  OpenAI SDK clients require it).
• Error normalization (wire.rs): maps Vertex/gRPC error
  envelopes to the Chat Completions error shape, falling back to HTTP
  status when the body is unparseable.
• Config (config.rs): project (required), region (default
  us-central1), max_body_bytes (default 4 MiB).

Credential injection is handled by the existing gcp_adc filter.
Integration test fixtures are deferred (TODO(#114)) until a live
Vertex environment is available for recording.

Related issue

Part of #114
Closes #997

Validation

• Unit tests — Vertex-specific tests across 5 modules (config,
  request with all role types and tool resolution, response for
  streaming and non-streaming, error normalization, filter lifecycle)
• Integration or functional tests — example config lint passes;
  functional test deferred (TODO(#114))
• make lint, make test, make doc (all exit 0)

Checklist

• I reviewed every changed line and can explain the change.
• New capabilities include an example config and functional example test.
• User-facing behavior and generated documentation are updated.
• Performance-sensitive changes include appropriate benchmark or load-test evidence.
• Commits are signed and include a Signed-off-by trailer.

Breaking changes

None. New filter only; no existing behavior or API is changed.

[leseb]

1000!

[praxis-bot]

praxis-bot review: feat(apis): OpenAI Chat Completions -> Vertex AI Gemini translation

Solid filter implementation with thorough test coverage across config, request, response, streaming, and error normalization modules. CI passes (21/21). The code follows project conventions well: deny_unknown_fields, full-width separators, assertion messages in tests, proper SPDX headers, and correct filter registration. The request transformation handles all major OpenAI roles, tool calls, multipart content, and response format mapping.

Four items below, all Medium severity.

[praxis-bot]

[Medium] project and region are interpolated directly into the Vertex AI URL path in vertex_path() (format string on mod.rs:110). build_config validates non-empty but does not reject characters that would produce malformed or traversable URLs (/, .., ?, #, whitespace, control characters).

A misconfigured project like my-project/../../other would alter the request target.

Add character validation, e.g.:

fn validate_path_segment(filter: &str, field: &str, value: &str) -> Result<(), FilterError> {
    if value.contains('/') || value.contains('?') || value.contains('#') || value.contains("..") {
        return Err(FilterError::from(format!(
            "{filter}: {field} must not contain '/', '?', '#', or '..'"
        )));
    }
    Ok(())
}

Then call it for both project and region in build_config.

[praxis-bot]

[Medium] The SseFrameParser buffer size is hardcoded to 4 * 1024 * 1024 (4 MiB), ignoring the user-configured max_body_bytes. If an operator sets max_body_bytes: 524288 (512 KiB), the SSE parser still accepts up to 4 MiB per frame, bypassing the configured limit. Conversely, a higher max_body_bytes doesn't raise the SSE cap.

Pass self.config.max_body_bytes through to the SSE parser. Since translate_sse_chunk doesn't have access to self, you could store it in StreamState at construction:

StreamState {
    parser: SseFrameParser::new(self.config.max_body_bytes),
    is_first_chunk: true,
}

This requires threading the config value to translate_sse_chunk. One option is adding a max_bytes parameter, or making translate_sse_chunk a method on the filter.

[praxis-bot]

[Medium] build_generation_config maps most OpenAI parameters but omits n (number of completions), which Gemini supports as candidateCount. An OpenAI client sending "n": 3 will silently get a single candidate back.

Add the mapping:

if let Some(v) = obj.get("n") {
    config.insert("candidateCount".to_owned(), v.clone());
}

Also consider mapping logprobs / top_logprobs to Gemini's responseLogprobs / logprobs if you want fuller parameter parity, though those are less commonly used.

[praxis-bot]

[Medium] created_timestamp() is called separately for each SSE frame during streaming (via transform_stream_chunk), producing a different created value on every chunk of the same response. OpenAI's streaming sends a consistent created timestamp across all chunks in a single completion.

Compute the timestamp once in StreamState at construction and pass it through:

struct StreamState {
    parser: SseFrameParser,
    is_first_chunk: bool,
    created: u64,
}

Then use state.created in transform_stream_chunk instead of calling created_timestamp() per frame.

[leseb]

A bunch of AI review:

P1 — Gemini thought signatures are discarded. Both translation directions omit extra_content.google.thought_signature (request.rs:L298, response.rs:L124). Gemini 3 requires these signatures verbatim for tool continuations; otherwise Vertex returns 400. Google documentation

P1 — Tool-call IDs collide across rounds. IDs restart at call_vertex_0 for every response (response.rs:L147). The history-wide ID/name map can consequently associate an earlier tool result with a later function.

P1 — SSE errors become successful truncated streams. Parser and per-frame translation errors are logged and dropped, after which [DONE] is still emitted (mod.rs:L323, mod.rs:L368). Incomplete EOF frames are likewise not rejected.

P1 — Streamed tool calls omit index. The non-streaming tool-call helper is reused for deltas without adding the required index (response.rs:L139). OpenAI SDKs and this repository’s existing stream consumer require it. OpenAI Python schema

P2 — Streaming silently drops every candidate except the first. n is mapped to candidateCount, but streaming uses .first() and hardcodes choice index zero (response.rs:L232).

P2 — Prompt safety blocks appear as successful empty completions. A response containing only promptFeedback becomes empty choices—or an empty delta followed by [DONE] (response.rs:L36). Google documents this no-candidate shape for content violations. Vertex response documentation

P2 — The client-controlled model is interpolated into the URI without validation. /, ?, #, or traversal-like values can corrupt routing instead of producing an OpenAI-shaped 400 (request.rs:L49, mod.rs:L106).

P2 — Empty bodies bypass translation and path rewriting. They are forwarded to Vertex at the original OpenAI path rather than rejected (mod.rs:L161).

P2 — Streaming usage is missing. stream_options.include_usage is discarded and usageMetadata is never emitted in the stream.

P2 — Logprobs are requested but never returned. The request enables Vertex logprobs (request.rs:L430), while both response translators discard logprobsResult.

P2 — Required functional and inference-fixture coverage is absent. The example is explicitly exempted with a TODO instead of receiving the integration test required by the repository instructions (lint_example_tests.rs:L45).

Also, could we add an integration test using the official OpenAI Python SDK, backed by a strict local fake Vertex/Gemini server? It should cover non-streaming, streaming, tool calls, and errors while asserting the translated Vertex path and request body. No live Vertex endpoint is required. Since client compatibility is this filter’s core contract, I consider this required before merging.

Thanks!

[praxis-bot]

Re-review

All four findings from the previous review have been addressed in the updated commit:

1. Path traversal in project/region — Fixed. validate_path_segment() now rejects /, ?, #, .., and ASCII control characters, with tests.
2. SSE parser ignoring max_body_bytes — Fixed. Config value threaded through to SseFrameParser::new().
3. Missing n → candidateCount — Fixed. Also added logprobs / top_logprobs mapping.
4. created_timestamp() per SSE frame — Fixed. Timestamp stored once in StreamState at stream start.

One new finding below.

[praxis-bot]

[Medium] model is interpolated into the Vertex AI URL path here, but unlike project and region (now validated via validate_path_segment in build_config), model comes from the client request body at runtime and is never checked for path-traversal characters.

A request with "model": "gemini-2.0-flash/../../admin" would produce a traversed URL targeting an unintended upstream endpoint. This is the same vulnerability class that the previous review identified for project/region, applied to request-time data.

Apply the existing validation before path construction:

// In on_request_body, after transform_request:
if result.model.contains('/')
    || result.model.contains('?')
    || result.model.contains('#')
    || result.model.contains("..")
    || result.model.bytes().any(|b| b.is_ascii_control())
{
    return Ok(FilterAction::Reject(Rejection {
        status: 400,
        message: "model name contains invalid path characters".to_owned(),
    }));
}

Alternatively, reuse config::validate_path_segment by making it pub(crate) and calling it on the model name.

[praxis-bot]

Re-review

Previous finding (model path traversal, re-review #2) remains unaddressed. One new finding below.

[praxis-bot]

[Medium] convert_function_call_to_tool_call generates IDs as call_vertex_{index} where index resets to 0 for every response. In multi-turn tool-use conversations, successive Gemini responses produce duplicate IDs (both yield call_vertex_0). This collides with build_tool_call_name_map in request.rs, which builds a flat HashMap from ALL assistant messages in the conversation history -- the last writer wins.

Concrete failure scenario:

1. Turn 1: Gemini returns functionCall(get_weather) -> proxy emits id: "call_vertex_0"
2. Client sends tool result tool_call_id: "call_vertex_0" + new prompt
3. Turn 2: Gemini returns functionCall(search) -> proxy emits id: "call_vertex_0" (duplicate)
4. Client sends full history including both turns' tool results
5. build_tool_call_name_map maps call_vertex_0 -> search (last wins), corrupting turn 1's resolution
6. Turn 1's functionResponse is sent with name: "search" instead of name: "get_weather"

Gemini rejects the mismatched name or produces wrong behavior.

Fix: generate unique IDs. Options:

• Use the filter context's id_generator (already available via ctx): ctx.id_generator.next_id("call_vertex")
• Use a monotonic counter stored in StreamState that persists across responses
• Append a random suffix: format!("call_vertex_{random_hex}_{index}")