Skip to content

feat(sdk/go/ai): support the Infron gateway - #874

Open
meridah7 wants to merge 1 commit into
Agent-Field:mainfrom
meridah7:feat/infron-provider
Open

feat(sdk/go/ai): support the Infron gateway#874
meridah7 wants to merge 1 commit into
Agent-Field:mainfrom
meridah7:feat/infron-provider

Conversation

@meridah7

@meridah7 meridah7 commented Aug 4, 2026

Copy link
Copy Markdown

Summary

Adds the Infron gateway to sdk/go/ai, in the shape this package already uses to describe a gateway, so selecting it is an env-var change rather than a fork.

Infron is an OpenAI-compatible gateway serving the standard <provider>/<model> ids, so nothing about a model's identity changes when it runs there: infron/moonshotai/kimi-k2.6 routes the same model the bare id names. I followed the attribution/config helpers already in the package rather than introducing a second way to describe a provider.

Disclosure: I work on Infron. Everything below is checkable from the diff and the commands in the test plan.

File Change
sdk/go/ai/infron_attribution.go New. Mirrors the existing attribution helper. Infron accepts the same HTTP-Referer / X-Title pair, and the attribution env vars already configured in a deployment are honored as fallbacks, so a deployment that already declares itself as "AgentField AI" keeps that identity after switching gateways.
sdk/go/ai/config.go IsInfron(); DefaultConfig() reads INFRON_API_KEY and points at https://llm.onerouter.pro/v1.
sdk/go/ai/client.go Attaches attribution on both the sync and the streaming path, at the two call sites this package already uses for it.
sdk/go/ai/model_params.go Opts Infron into native usage accounting, and strips the routing-only infron/ prefix before the request goes out.
sdk/go/ai/response.go Top-level cost field + normalizeNativeCost() on Response and StreamChunk. See below.
docs/ENVIRONMENT_VARIABLES.md, sdk/go/ai/README.md The env vars and the prefix swap.
sdk/go/ai/infron_attribution_test.go New. 17 cases: detection, precedence, attribution fallback, prefix stripping, cost normalization.

The one real difference, handled rather than papered over

This package reads native cost from usage.cost. Infron reports it at the top level of the body, and of the final stream chunk:

{ "model": "deepseek/deepseek-v4-flash", "cost": 0.000002,
  "usage": { "prompt_tokens": 12, "completion_tokens": 9 } }   // no usage.cost

Parsed as-is that leaves Usage.Cost == nil, which recordLLMUsage reads as price unknown: the call is still recorded, but with a nil cost and an empty cost_source instead of "provider". Nothing errors, and it only shows up later as a hole in the cost data.

Response and StreamChunk now carry the top-level field, and normalizeNativeCost() folds it into Usage.Cost on parse, so every existing consumer keeps reading one place. An explicit usage.cost always wins; the fold only fills a gap.

The model prefix is stripped before the wire

infron/ is a routing marker for callers that select a gateway by model string, but the gateway serves the bare id, so leaving the prefix on returns No available providers for model infron/moonshotai/kimi-k2.6. stripInfronPrefix removes it in marshalRequest, mirroring the prefix handling this package already does on the media path. Only a copy of the Request is rewritten, so the caller's Request and config.Model are untouched and IsInfron() still reports the truth.

Worth flagging either way: the chat path does not strip the pre-existing gateway prefix today (only the media path does), so a model string carrying that prefix hits the same wall against its own gateway. I left that alone rather than change existing behavior inside a PR about a new provider, but happy to send it separately if you want the two symmetrical.

Backwards compatibility

A gateway key that was already honored before Infron existed keeps precedence. Adding INFRON_API_KEY to an existing environment never reroutes it; TestDefaultConfigExistingGatewayWinsOverInfron pins that. IsInfron() also does not match a bare shared model id (moonshotai/kimi-k2.6), only the explicit infron/ prefix or the Infron host, so gateways cannot be confused by model alone.

Type of change

  • New feature
  • Bug fix — one behavior change comes along for the ride: a gateway reporting cost at the top level now populates Usage.Cost instead of being dropped
  • Refactor / cleanup
  • Docs only
  • Tests only
  • CI / tooling
  • Breaking change

Test plan

Rebased on main at 4bc8ce7 and re-run today.

  • cd sdk/go && go test ./...agent, ai, client, did, inputs, types all ok
  • cd sdk/go && go test -race ./ai/... — clean
  • gofmt -l clean on every touched file; go vet ./ai/... clean
  • End-to-end against the live gateway, driving the real SDK — 29/29. Not hand-rolled HTTP: it builds ai.Config, calls ai.NewClient, and exercises Complete() and StreamComplete(). The attribution headers are asserted by putting a capturing reverse proxy in front of the real gateway, so what is checked is what actually went on the wire:
1. DefaultConfig picks up INFRON_API_KEY .................... 7/7
2. An existing gateway key still wins when both are set ..... 3/3
3. Attribution headers on the wire (proxy capture) .......... 7/7
     HTTP-Referer: https://agentfield.ai
     X-Title:      AgentField AI
     body:         {"model":"moonshotai/kimi-k2.6", ... "usage":{"include":true}}
4. LIVE sync call ........................................... 6/6
     text "hello", in=10 out=2, Usage.Cost $0.00000100
5. LIVE streaming call ...................................... 3/3
     final chunk Usage.Cost populated
6. Live calls on kimi-k2.6 / minimax-m2.5 / glm-5.2 ......... 3/3
  • A second live pass over the paths the first one does not touch — 31/31.
D1.  Agent cost tracker records cost_source=provider ........ 4/4
D2.  AGENTFIELD_INFRON_ATTRIBUTION=false suppresses headers . 3/3
D3.  AI_BASE_URL still overrides the Infron default ......... 3/3
D4.  Pre-existing gateway path untouched (mock upstream) .... 5/5
       attribution header still sent, usage.cost still read,
       its routing prefix still NOT stripped (unchanged)
D5.  Explicit usage.cost never clobbered by top-level cost ... 2/2
D6.  8 concurrent calls; caller config left unmutated ....... 3/3
D7.  Structured JSON output ................................. 3/3
D8.  Tool calling (kimi-k2.6, get_weather) .................. 5/5
D9.  Error paths (bad model, bad key) ....................... 2/2
D10. Prompt-cache token accounting (2008 cached tokens) ..... 1/1
  • Before/after proof that the cost gap was real. Same live call, same model, only the SDK revision differs:
main @ 4bc8ce7        Usage.Cost = <nil>          -> cost_source ""
feat/infron-provider  Usage.Cost = $0.00000100    -> cost_source "provider"

One pre-existing failure, unrelated to this PR: TestOpenCodeConcurrencyLimit_RealSubprocess in sdk/go/harness fails identically on a clean checkout of main on macOS — same test, same parse error, verified side by side before opening this. It shells out to date +%s%N, which BSD date does not support, so the test parses a literal N. It passes in CI on Linux. Happy to send that as a separate fix if useful.

Test coverage

  • I ran tests for the surface I changed locally (sdk-go).
  • New code paths are covered by tests in this PR (no bare additions).
  • No coverage-baseline.json change needed, and here is why:
main this PR
sdk/go/ai statement coverage 93.3% 93.2%

Measured back to back with -count=1; the number moves about 0.1 pp between runs on its own. That is against a max_surface_drop of 1.0 and a min_surface of 84.0, on the smaller of the two numbers feeding the sdk-go surface. Patch coverage on the non-test lines this PR adds is 92.7% (101/109 coverable added lines), against min_patch = 80.0; the new file itself is at 95.7%, IsInfron and the prefix strip at 100%.

Notes

  • Python SDK parity is the obvious follow-up (the litellm-side attribution module). Kept out to keep this reviewable. Happy to send it right after, or fold it in here if you would rather review once.
  • llm.onerouter.pro is deliberately not added to vouchedRewriteDomains. I probed both max_tokens and max_completion_tokens and they behaved identically, with neither demonstrably enforced, so I left the conservative legacy max_tokens path in place per the reasoning already in that comment. Easy to add if you have better information.
  • Companion PR on the SWE-AF side (Infron as an open_code provider + the INFRON_API_KEY auto-select path): feat: add Infron as an open_code gateway provider SWE-AF#126. The two are independent; either can land alone.

@meridah7
meridah7 requested review from a team and AbirAbbas as code owners August 4, 2026 21:48
@CLAassistant

CLAassistant commented Aug 4, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@meridah7 meridah7 changed the title feat(sdk/go/ai): support the Infron gateway alongside OpenRouter feat(sdk/go/ai): support the Infron gateway Aug 4, 2026
Infron is an OpenAI-compatible inference gateway that serves the standard
<provider>/<model> ids, so a model moves across by prefix alone:
infron/moonshotai/kimi-k2.6 routes the same model the bare id names.

Follows the provider shape already in this package rather than inventing
a new one:

- infron_attribution.go mirrors the existing attribution helper. Infron
  accepts the same HTTP-Referer / X-Title pair, and the attribution env
  vars already configured for the existing gateway are honored as
  fallbacks, so a deployment that already declares itself as
  'AgentField AI' keeps that identity after switching gateways.
- Config gains IsInfron(); DefaultConfig() reads INFRON_API_KEY and
  points at https://llm.onerouter.pro/v1.
- client.go attaches attribution on both the sync and streaming paths.
- marshalRequest opts Infron into native usage accounting and strips the
  routing-only 'infron/' model prefix before the request goes out
  (stripInfronPrefix, mirroring the prefix handling on the media path).
  The gateway serves the bare id, so leaving the prefix on returns 'No
  available providers for model infron/...'. Only a copy of the Request
  is rewritten; the caller's Request is untouched.

One real difference is handled rather than papered over: Infron returns
the native cost at the top level of the body and of the final stream
chunk, rather than nested under usage. Parsed naively that leaves
Usage.Cost nil, which the cost tracker reads as 'price unknown' -- usage
still recorded, but with no cost and an empty cost_source instead of
'provider'. Response/StreamChunk now carry the top-level field and
normalizeNativeCost folds it into Usage.Cost, so every existing consumer
keeps reading one place. An explicit usage.cost always wins.

A gateway key that was already honored before Infron existed keeps
precedence, so adding an Infron key never reroutes an existing
deployment.

llm.onerouter.pro is deliberately NOT added to vouchedRewriteDomains:
max_tokens and max_completion_tokens behaved identically in probing and
neither could be shown to be enforced, so the conservative legacy
max_tokens path stays, per the reasoning already in that comment.
@meridah7
meridah7 force-pushed the feat/infron-provider branch from 8574fd0 to 6323eaa Compare August 4, 2026 22:03
@santoshkumarradha

Copy link
Copy Markdown
Member

Thanks for putting this together. I did an initial pass on the diff, but I can’t take it to merge yet because the required repo checks never showed up on this PR. At the moment I only see , so branch protection is still blocking on missing checks like . Please rebase on current or otherwise retrigger the normal PR workflows, and I’ll do a final mergeability pass once the required checks are actually running.

@santoshkumarradha

Copy link
Copy Markdown
Member

Thanks for putting this together. I did an initial pass on the diff, but I can’t take it to merge yet because the required repo checks never showed up on this PR. At the moment I only see license/cla, so branch protection is still blocking on missing checks like coverage-summary. Please rebase on current main or otherwise retrigger the normal PR workflows, and I’ll do a final mergeability pass once the required checks are actually running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants