Skip to content

Update AI Backend examples for other paths #465

@artberger

Description

@artberger

In answer to this question from a user:

In the Agentgateway docs for OpenAI LLM consumption, it states that you can configure an "ai-route" to either map to the "completions" endpoint or to "passthrough": https://agentgateway.dev/docs/llm/providers/openai/#multiple-endpoints

Does someone know how you can configure that on kgateway? Does that need to go on a Backend? On the HTTPRoute? On a Policy?
I'm trying to access the OpenAI /v1/embeddings endpoint via Agentgateway controlled by GGv2, and it seems to treat all my requests as "completions" requests (i.e., it barfs on my embeddings request stating my request lacks a "messages" field).

Docs followup for: kgateway-dev/kgateway#12590

Example config: https://github.com/apexlnc/kgateway/blob/main/examples/ai-backend-with-routes.yaml

apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  name: unified-ai-backend
spec:
  type: AI
  ai:
    llm:
      anthropic:
        model: claude-sonnet-4-20250514
        authToken:
          kind: SecretRef
          secretRef:
            name: anthropic-secret
      routes:
        "/v1/chat/completions": "completions"
        "/v1/messages": "messages"
        "/v1/models": "models"
        "*": "passthrough"

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions