-
Notifications
You must be signed in to change notification settings - Fork 39
Description
In answer to this question from a user:
In the Agentgateway docs for OpenAI LLM consumption, it states that you can configure an "ai-route" to either map to the "completions" endpoint or to "passthrough": https://agentgateway.dev/docs/llm/providers/openai/#multiple-endpoints
Does someone know how you can configure that on kgateway? Does that need to go on a Backend? On the HTTPRoute? On a Policy?
I'm trying to access the OpenAI /v1/embeddings endpoint via Agentgateway controlled by GGv2, and it seems to treat all my requests as "completions" requests (i.e., it barfs on my embeddings request stating my request lacks a "messages" field).
Docs followup for: kgateway-dev/kgateway#12590
Example config: https://github.com/apexlnc/kgateway/blob/main/examples/ai-backend-with-routes.yaml
apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
name: unified-ai-backend
spec:
type: AI
ai:
llm:
anthropic:
model: claude-sonnet-4-20250514
authToken:
kind: SecretRef
secretRef:
name: anthropic-secret
routes:
"/v1/chat/completions": "completions"
"/v1/messages": "messages"
"/v1/models": "models"
"*": "passthrough"