Skip to content

feat(harness): add Grok Build CLI provider - #861

Open
storyicon wants to merge 1 commit into
Agent-Field:mainfrom
storyicon:feat/grok-cli-harness-provider
Open

feat(harness): add Grok Build CLI provider#861
storyicon wants to merge 1 commit into
Agent-Field:mainfrom
storyicon:feat/grok-cli-harness-provider

Conversation

@storyicon

Copy link
Copy Markdown

Summary

This PR adds a first-class Grok Build CLI harness provider so AgentField can orchestrate local grok headless sessions with the same provider surface as Codex, OpenCode, Gemini, and Claude Code.

The goal is to make Grok a peer runtime for multi-turn coding-agent work (read/write/edit/bash tool loops) without requiring callers to shell out ad hoc or invent a parallel execution path.

Motivation

AgentField already standardizes coding-agent execution behind HarnessProvider + build_provider(). Operators who use Grok Build CLI locally currently cannot plug that binary into the harness stack:

  • No provider="grok" registration
  • No binary resolution / doctor metadata
  • No metrics parsing that tolerates Grok’s incomplete usage payloads

Without this, multi-agent frameworks built on AgentField (for example SWE-AF) cannot select Grok as a first-class runtime the way they select Codex or Claude Code.

Design considerations

1. Provider shape matches existing CLI providers

GrokProvider follows the same contract as CodexProvider / OpenCodeProvider:

Concern Approach
Discovery ensure_cli_available("grok", bin_path)
Invocation async run_cli with optional env / cwd
Errors RawResult(is_error=True, failure_type=…)
Success payload final agent text in RawResult.result

This keeps the runner, schema pipeline, and retry logic provider-agnostic.

2. PTY wrapping is required for Grok today

Plain pipe attachment causes Grok to fail with Device not configured (os error 6). When script(1) is available, the provider runs:

script -q /dev/null grok … --output-format json --prompt-file …

script allocates a PTY while still delivering agent JSON on stdout. If script is missing, the provider falls back to a direct grok invocation (same flags) so environments without script are not hard-broken, but PTY-capable hosts remain the recommended path.

3. Prompt delivery via --prompt-file

Long AgentField prompts exceed practical argv limits. The provider always writes the prompt to a temporary file and passes --prompt-file, then unlinks the file in finally.

4. Permission mode mapping for unattended runs

Harness executions are unattended. The provider maps common AgentField permission modes onto Grok’s --permission-mode values and defaults to non-interactive approvals (bypassPermissions / acceptEdits / --always-approve) so coding loops do not block on TTY prompts.

5. Metrics robustness (token fields may be absent)

Grok’s JSON envelope frequently omits usage.input_tokens / output_tokens. The previous pattern of storing None into Metrics integer fields breaks aggregation in the shared runner:

tokens["input_tokens"] += raw.metrics.input_tokens  # int += None → TypeError

This PR:

  • Normalizes missing token counts to 0 inside _usage_metrics
  • Also coerces None → 0 in _accumulate_metrics as a defensive invariant for all providers

This is intentional fail-soft for accounting only. It does not invent business outcomes or hide failed CLI runs.

6. Localhost / proxy hygiene

The provider sets default NO_PROXY / no_proxy for 127.0.0.1,localhost,::1 so control-plane traffic is not accidentally routed through local HTTP proxies during harness sessions.

7. What this PR deliberately does not do

  • Does not implement Grok --json-schema constrained decoding (can be a follow-up)
  • Does not change the universal Write-to-.agentfield_output.json schema strategy in the runner
  • Does not add network/API-based Grok chat completions (this is CLI-only, consistent with Codex/OpenCode)

API / config surface

  • SUPPORTED_PROVIDERS includes "grok"
  • HarnessConfig.provider documents "grok"
  • HarnessConfig.grok_bin (default "grok")
  • Doctor / availability metadata for binary install + optional XAI_API_KEY

Example:

from agentfield.types import HarnessConfig
from agentfield.harness.providers._factory import build_provider

provider = build_provider(HarnessConfig(provider="grok", model="grok-4.5"))

Files

Path Change
sdk/python/agentfield/harness/providers/grok.py New Grok CLI provider
sdk/python/agentfield/harness/providers/_factory.py Register grok
sdk/python/agentfield/harness/_availability.py ProviderSpec for grok
sdk/python/agentfield/harness/_runner.py Safe metrics accumulation
sdk/python/agentfield/types.py grok_bin + provider docs
sdk/python/tests/test_harness_factory.py Factory + protocol coverage
sdk/python/tests/test_harness_grok_provider.py Unit tests for provider helpers

Test plan

  • build_provider(HarnessConfig(provider="grok")) returns GrokProvider
  • Custom grok_bin is honored
  • Missing usage tokens become 0 (not None)
  • JSON payload extraction tolerates script control-character noise
  • Factory supported-provider list includes grok
  • CI: sdk/python harness unit tests on this branch
  • Manual (optional): with local grok installed and logged in, run a short harness task with provider="grok" and confirm non-error RawResult

Compatibility

  • Additive for existing providers (Claude / Codex / Gemini / OpenCode paths unchanged aside from defensive metrics coercion)
  • Requires a local Grok Build CLI on PATH (or grok_bin) for live runs
  • Consumers that pin older AgentField versions are unaffected until they upgrade

Follow-ups (out of scope)

  1. Optional native --json-schema structured decoding for Grok
  2. Streaming / idle-watchdog tuning for long non-streaming JSON sessions
  3. Version-pin guidance once Grok CLI flags stabilize across releases

Introduce a first-class harness provider for the local Grok Build CLI so
AgentField can drive Grok headless sessions the same way it drives Codex,
OpenCode, Gemini, and Claude Code.

The provider wraps the Grok CLI with a PTY (via script) because plain pipes
currently fail with "Device not configured", feeds prompts through
--prompt-file to avoid argv limits, and normalizes missing token usage to
zero so metrics aggregation does not crash on incomplete usage objects.
@storyicon
storyicon requested review from a team and AbirAbbas as code owners August 2, 2026 06:09
@CLAassistant

CLAassistant commented Aug 2, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants