Skip to content

feat: install an unreleased chart (platform.apsPath) and route agent inference through agentgateway (platform.llmRouting) - #52

Open
paurosello wants to merge 1 commit into
mainfrom
llm-routing-local-chart
Open

feat: install an unreleased chart (platform.apsPath) and route agent inference through agentgateway (platform.llmRouting)#52
paurosello wants to merge 1 commit into
mainfrom
llm-routing-local-chart

Conversation

@paurosello

Copy link
Copy Markdown
Contributor

The lab could not test an unreleased chart change at all. Every hop of the
delivery chain consumes a released artifact: agent-platform-connectivity is
published to gsoci, agent-platform-standalone generates its whole chart from
that release, and this lab vendors agent-platform-standalone from git at
platform.apsRef. So the platform change most worth testing — the one not yet
shipped — was the one change the lab could not install.

platform.apsPath

Installs a local agent-platform-standalone checkout instead of the pinned
commit: ensurePlatformChart skips the fetch and only builds the chart's
dependencies. The checkout is read, never written (helm fills its gitignored
charts/). apsRepo/apsRef are ignored while it is set and the boot says
which chart it used.

$ agentlab configure --defaults --aps-path ~/workspace/agent-platform-standalone
$ agentlab platform
==> Using the local agent-platform-standalone at … (chart dependencies only)

--aps-path "" clears it. The agent-platform-standalone side of the same
gap is hack/curate.sh -fleet-dir
(giantswarm/agent-platform-standalone#93); together they are the loop: curate
there, install here.

platform.llmRouting

The first thing tested through that loop. Agent inference stops leaving the
agent pods for api.anthropic.com and goes through the platform's own
agentgateway, so the data plane emits GenAI metrics for every model call.

One toggle renders both halves — the cluster-internal llm listener on the
edge Gateway, and the base URL kagent's default ModelConfig dials — because a
listener nothing uses and agents pointed at a closed port are both broken
labs. The port is one constant, written into the listener and into that URL,
so the pair cannot drift. It has nothing to do with platform.agentsPort, the
HOST port for the kagent UI.

Off by default: the chart change is unreleased
(giantswarm/agent-platform#254), and the released umbrella's
values.schema.json rejects an undeclared top-level key outright, so a block
emitted by default would fail every install (HACKS.md U13). It needs
platform.agents, since the cutover is a kagent ModelConfig.

Verified on the lab

The listener answers a real completion, and after one agent turn the lab
Prometheus serves, through the edge:

agentgateway_gen_ai_client_token_usage_sum{agent="llmprobe",agent_namespace="kagent",gen_ai_token_type="input"}  712
agentgateway_gen_ai_client_cost_usd_total{agent="llmprobe",gen_ai_response_model="claude-sonnet-4-6"}            0.002226
agentgateway_cost_catalog_lookups_total{status="Exact"}                                                          4

agentlab platform-test passes with routing on. The same run caught a routing
bug in giantswarm/agent-platform#254 (the agent-platform-mcps catch-all
HTTPRoute outranked the LLM route on its own listener, so every inference call
answered mcp: client must accept both application/json and text/event-stream), fixed on that PR — which is what this loop is for.

@paurosello
paurosello requested a review from a team as a code owner September 3, 2026 08:14
…inference through agentgateway (platform.llmRouting)

The lab could not test an unreleased chart change at all. Every hop of the
delivery chain consumes a released artifact: agent-platform-connectivity is
published to gsoci, agent-platform-standalone generates its whole chart from
that release, and this lab vendors agent-platform-standalone from git at
platform.apsRef. So the platform change most worth testing — the one not yet
shipped — was the one change the lab could not install.

platform.apsPath closes the last hop: it installs a local
agent-platform-standalone checkout instead of the pinned commit, skipping the
fetch and only building the chart's dependencies. The checkout is read, never
written (helm fills its gitignored charts/). apsRepo/apsRef are ignored while
it is set and the boot says which chart it used.
`agentlab configure --defaults --aps-path <dir>` sets it, `--aps-path ""`
clears it. The standalone side of the same gap is `hack/curate.sh -fleet-dir`
(giantswarm/agent-platform-standalone#93); together they are the loop:
curate there, install here.

The first thing tested through that loop is platform.llmRouting: agent
inference stops leaving the agent pods for api.anthropic.com and goes through
the platform's own agentgateway, so the data plane emits GenAI metrics for
every model call. One toggle renders both halves — the cluster-internal `llm`
listener on the edge Gateway, and the base URL kagent's default ModelConfig
dials — because a listener nothing uses and agents pointed at a closed port
are both broken labs. The port is one constant, written into the listener and
into that URL, so the pair cannot drift; it has nothing to do with
platform.agentsPort, the HOST port for the kagent UI.

Off by default: the chart change is unreleased, and the released umbrella's
values.schema.json rejects an undeclared top-level key outright, so a block
emitted by default would fail every install (HACKS.md U13). It needs
platform.agents, since the cutover is a kagent ModelConfig.

Verified on the lab: the listener answers a real completion, and after one
agent turn the lab Prometheus serves
agentgateway_gen_ai_client_token_usage_sum and
agentgateway_gen_ai_client_cost_usd_total keyed by
agent="<serviceaccount>", agent_namespace="kagent", with
agentgateway_cost_catalog_lookups_total{status="Exact"} growing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant