Skip to content

Add CLI (ep) and Claude Code skills#30

Merged
samcm merged 19 commits intomasterfrom
qu0b/cli-and-skills
Mar 9, 2026
Merged

Add CLI (ep) and Claude Code skills#30
samcm merged 19 commits intomasterfrom
qu0b/cli-and-skills

Conversation

@qu0b
Copy link
Member

@qu0b qu0b commented Mar 6, 2026

Summary

  • Extract shared app kernel (pkg/app/) from pkg/server/builder.go — transport-agnostic initialization (plugins, proxy, sandbox, search indices) reused by both MCP server and CLI
  • Add ep CLI binary (cmd/cli/, pkg/cli/) with subcommands: datasources, schema, docs, execute, session, search, version — same capabilities as the MCP server without the protocol layer
  • Add/update Claude Code skillsinvestigate (devnet debugging workflow), updated query (CLI + MCP references), updated create-plugin (references pkg/app/)
  • Symlink libllama_go.so in download-models so binaries work without LD_LIBRARY_PATH

Non-breaking and additive — MCP server code is unchanged in behavior, just delegates initialization to pkg/app/.

Commits

  1. refactor: extract pkg/app from server builder — shared kernel
  2. feat: scaffold cmd/cli and pkg/cli — binary + root command + Makefile
  3. feat(cli): add datasources command
  4. feat(cli): add schema command
  5. feat(cli): add docs command
  6. feat(cli): add execute command
  7. feat(cli): add session command
  8. feat(cli): add search command
  9. docs: update CLAUDE.md with CLI reference
  10. feat(skills): update query skill with CLI commands
  11. feat(skills): add investigate skill, update create-plugin references
  12. fix: symlink libllama_go.so to repo root in download-models

Test plan

  • make build — MCP server still builds
  • make build-cli — CLI binary builds
  • make test — existing tests pass
  • ./ep version — prints version info
  • ./ep datasources --help — shows usage
  • ./ep docs — lists Python API modules (no proxy needed)
  • MCP server behavior unchanged (make run)

qu0b added 17 commits March 6, 2026 15:29
Move shared initialization logic (plugin registry, sandbox, proxy,
cartographoor, semantic search indices) into a transport-agnostic
pkg/app package. The MCP server builder now delegates to app.Build()
and layers on MCP-specific concerns (auth, tool registry, resource
registry).

This enables the upcoming CLI to reuse the same core without depending
on the MCP protocol layer.
Add a separate CLI binary entry point (cmd/cli) with Cobra root command,
version subcommand, and build-cli Makefile target. The CLI binary is
named 'ep' and will host analytics commands alongside the existing MCP
server.
List available datasources (ClickHouse, Prometheus, Loki) discovered
from the credential proxy. Supports --type filter and --json output.
Show ClickHouse table schemas via 'ep schema [table-name]'. Uses the
lightweight app build (proxy + plugins only, no sandbox/embedding) for
fast startup. Adds BuildLight() to pkg/app and SchemaClient() accessor
to the ClickHouse plugin.
Show Python API documentation for ethpandaops library modules. Uses
static embedded data from plugins — no proxy or config needed.
Execute Python code in a sandbox via 'ep execute'. Supports --code,
--file, or stdin for input. Stdout goes to stdout, metadata (session
ID, output files) goes to stderr for clean piping. Adds
BuildWithSandbox() and SandboxEnv() to pkg/app.
Manage sandbox sessions via 'ep session {list,create,destroy}'.
Sessions keep containers alive between executions, preserving
workspace files.
Semantic search over examples and runbooks via 'ep search examples'
and 'ep search runbooks'. Supports --category/--tag filters, --limit,
and --json output. Uses the full app build for embedding model and
search indices.
Document the ep CLI binary, its subcommands, and the shared app kernel.
Update references from pkg/server/builder.go to pkg/app/app.go where
plugin registration now lives. Add ethnode plugin to the list.
Add ep CLI usage alongside MCP server references so the skill works
in both contexts. Document all CLI subcommands for discovery, search,
execution, and session management.
The prebuilt libllama_go.so from kelindar/search uses AVX-VNNI
instructions (avx_vnni) which are not available on all x86_64 CPUs,
causing SIGILL crashes. Build from source with -DGGML_NATIVE=OFF
for portable binaries that still enable AVX2 SIMD.
Add investigate skill for CLI-driven devnet debugging with multi-phase
workflow (discovery, Dora collection, Loki logs, root cause analysis).
Update create-plugin skill to reference pkg/app/app.go for registration.
The kelindar/search library searches CWD for libllama_go.so at runtime.
Creating a symlink from the repo root to models/libllama_go.so removes
the need for LD_LIBRARY_PATH when running binaries locally.
Add WaitForReady(ctx) to ClickHouseSchemaClient so CLI waits for the
initial async schema fetch before reading results. Fix fmt.Println
redundant newline vet warning in search.go.
…g prebuilt

The prebuilt binary from kelindar/search uses AVX-VNNI instructions
not available on all x86_64 CPUs. Build from source with
-DGGML_NATIVE=OFF for portable binaries (still enables AVX2).
@qu0b
Copy link
Member Author

qu0b commented Mar 6, 2026

Manual Testing Results

Tested against a local Docker Compose stack (proxy on localhost:18081, Docker sandbox).

Command Result Notes
ep version Prints version/commit/build info
ep --help All subcommands listed
ep docs Lists all 5 modules (clickhouse, dora, ethnode, loki, prometheus)
ep docs clickhouse Shows function signatures, params, returns
ep docs --json Valid JSON output
ep datasources Lists 6 datasources from proxy
ep datasources --type clickhouse Filters to 3 ClickHouse clusters
ep datasources --json Valid JSON with metadata
ep schema Lists tables with column counts (waits for async fetch)
ep schema --json Full column details in JSON
ep schema <table> Shows columns, networks, comments
ep execute --code 'print("hello")' Prints hello from sandbox, creates session
ep session list Shows active sessions (or "No active sessions")
ep search examples "block count" Returns 3 results with similarity scores
ep search runbooks "finality delay" Returns finality delay runbook (score: 0.43)
make build (MCP server) MCP server still builds
make test All tests pass

Bugs found and fixed during testing

  1. Schema race conditionep schema returned empty results because the CLI read GetAllTables() before the async background fetch completed. Fixed by adding WaitForReady(ctx) to ClickHouseSchemaClient interface.
  2. go vet warningfmt.Println with redundant newline in search.go. Fixed by switching to fmt.Print.
  3. libllama_go.so SIGILL — The pre-built shared library from GitHub crashed with illegal instruction. Fixed by building from source in the Makefile (mirrors the Dockerfile approach).

qu0b added 2 commits March 6, 2026 17:27
Add .githooks/pre-commit that runs golangci-lint on changed files.
Install with `make setup-hooks`. Fix Makefile to install golangci-lint
v2 (matching .golangci.yml config version). Remove unused buildApp().
Fix .gitignore `cli` pattern that incorrectly matched `pkg/cli/`.
Add direct CLI access to plugin datasources without requiring the
Python sandbox. Commands make authenticated HTTP calls through the
credential proxy (or directly for Dora), providing instant results.

New commands:
- ep ethnode {syncing,version,health,peers,finality,header,block-number,beacon-get,exec-rpc}
- ep dora {networks,overview,validator,slot,epoch}
- ep clickhouse query
- ep prometheus {query,query-range,labels,label-values}
- ep loki {query,labels,label-values}

All commands support --json for machine-readable output.
@qu0b
Copy link
Member Author

qu0b commented Mar 6, 2026

Updated Manual CLI Testing Results

Re-tested all plugin CLI commands including ethnode (previously blocked). Used bal-devnet-2 devnet nodes for ethnode since mainnet/hoodi/sepolia are behind Cloudflare mTLS (not accessible from local dev).

Dora (direct HTTP via cartographoor)

Command Status
ep dora networks ✅ human + JSON
ep dora overview hoodi ✅ human + JSON
ep dora validator hoodi 12345
ep dora slot hoodi 1000000
ep dora epoch hoodi 100

ClickHouse (via proxy)

Command Status
ep clickhouse query xatu "SELECT count() ..." ✅ TSV human output + JSON with metadata/statistics

Prometheus (via proxy)

Command Status
ep prometheus query platform "up" ✅ human {labels} => value + JSON
ep prometheus query-range platform "up{job='grafana'}" --start now-1h --end now --step 5m ✅ human range output + JSON
ep prometheus labels platform ✅ one per line + JSON
ep prometheus label-values platform job ✅ one per line + JSON

Loki (via proxy)

Command Status
ep loki query primary '{app="dora"}' --limit 5 ✅ raw log lines + JSON
ep loki labels primary ✅ one per line + JSON
ep loki label-values primary app ✅ one per line + JSON

Ethnode (via proxy → bal-devnet-2 / hc-lighthouse-besu-super-1)

Command Output Status
ep ethnode syncing Head slot: 180665, Sync distance: 22009, Is syncing: true ✅ human + JSON
ep ethnode version CL: Lighthouse/v8.0.1-65bb283/x86_64-linux
ep ethnode health Not initialized (503)
ep ethnode peers Connected: 20, Disconnected: 936
ep ethnode finality Finalized: epoch 5642, Current justified: epoch 5644
ep ethnode header Slot, proposer index, root, parent/state/body roots
ep ethnode beacon-get ... /eth/v1/node/identity Full peer ID, ENR, P2P addresses, metadata
ep ethnode block-number 504 timeout (EL offline on this node) ✅ error handling correct
ep ethnode exec-rpc ... eth_chainId 504 timeout (EL offline) ✅ error handling correct

Summary

  • 22/22 commands tested ✅
  • All --json variants produce valid pretty-printed JSON
  • Human-readable output properly formatted for all commands
  • Error handling works correctly (proxy errors, timeouts, 503s propagated with meaningful messages)
  • Ethnode exec-rpc/block-number timed out as expected (EL was offline on the test node — el_offline: true confirmed by syncing endpoint)

Note on mainnet/hoodi/sepolia ethnode access

Mainnet, hoodi, and sepolia bn-*.srv.{network}.ethpandaops.io endpoints are behind Cloudflare mTLS. TLS handshake fails from local dev (alert 40 — no client certificate). This is by design: the proxy is meant to run inside the K8s cluster where these resolve to internal IPs. Devnet nodes (e.g. bal-devnet-2) work fine over public HTTPS with basic auth.

@samcm
Copy link
Member

samcm commented Mar 9, 2026

LGTM! 🚀

@samcm samcm merged commit c1a57d0 into master Mar 9, 2026
5 checks passed
parithosh pushed a commit that referenced this pull request Mar 10, 2026
* refactor: extract pkg/app from server builder

Move shared initialization logic (plugin registry, sandbox, proxy,
cartographoor, semantic search indices) into a transport-agnostic
pkg/app package. The MCP server builder now delegates to app.Build()
and layers on MCP-specific concerns (auth, tool registry, resource
registry).

This enables the upcoming CLI to reuse the same core without depending
on the MCP protocol layer.

* feat: scaffold cmd/cli and pkg/cli

Add a separate CLI binary entry point (cmd/cli) with Cobra root command,
version subcommand, and build-cli Makefile target. The CLI binary is
named 'ep' and will host analytics commands alongside the existing MCP
server.

* feat(cli): add datasources command

List available datasources (ClickHouse, Prometheus, Loki) discovered
from the credential proxy. Supports --type filter and --json output.

* feat(cli): add schema command

Show ClickHouse table schemas via 'ep schema [table-name]'. Uses the
lightweight app build (proxy + plugins only, no sandbox/embedding) for
fast startup. Adds BuildLight() to pkg/app and SchemaClient() accessor
to the ClickHouse plugin.

* feat(cli): add docs command

Show Python API documentation for ethpandaops library modules. Uses
static embedded data from plugins — no proxy or config needed.

* feat(cli): add execute command

Execute Python code in a sandbox via 'ep execute'. Supports --code,
--file, or stdin for input. Stdout goes to stdout, metadata (session
ID, output files) goes to stderr for clean piping. Adds
BuildWithSandbox() and SandboxEnv() to pkg/app.

* feat(cli): add session command

Manage sandbox sessions via 'ep session {list,create,destroy}'.
Sessions keep containers alive between executions, preserving
workspace files.

* feat(cli): add search command

Semantic search over examples and runbooks via 'ep search examples'
and 'ep search runbooks'. Supports --category/--tag filters, --limit,
and --json output. Uses the full app build for embedding model and
search indices.

* docs: update CLAUDE.md with CLI reference and pkg/app architecture

Document the ep CLI binary, its subcommands, and the shared app kernel.
Update references from pkg/server/builder.go to pkg/app/app.go where
plugin registration now lives. Add ethnode plugin to the list.

* feat(skills): update query skill with CLI commands

Add ep CLI usage alongside MCP server references so the skill works
in both contexts. Document all CLI subcommands for discovery, search,
execution, and session management.

* fix(docker): build libllama_go.so from source for all architectures

The prebuilt libllama_go.so from kelindar/search uses AVX-VNNI
instructions (avx_vnni) which are not available on all x86_64 CPUs,
causing SIGILL crashes. Build from source with -DGGML_NATIVE=OFF
for portable binaries that still enable AVX2 SIMD.

* feat(skills): add investigate skill, update create-plugin references

Add investigate skill for CLI-driven devnet debugging with multi-phase
workflow (discovery, Dora collection, Loki logs, root cause analysis).
Update create-plugin skill to reference pkg/app/app.go for registration.

* fix: symlink libllama_go.so to repo root in download-models

The kelindar/search library searches CWD for libllama_go.so at runtime.
Creating a symlink from the repo root to models/libllama_go.so removes
the need for LD_LIBRARY_PATH when running binaries locally.

* add env file

* update gitignore

* fix: schema WaitForReady and vet warning

Add WaitForReady(ctx) to ClickHouseSchemaClient so CLI waits for the
initial async schema fetch before reading results. Fix fmt.Println
redundant newline vet warning in search.go.

* fix(makefile): build libllama_go.so from source instead of downloading prebuilt

The prebuilt binary from kelindar/search uses AVX-VNNI instructions
not available on all x86_64 CPUs. Build from source with
-DGGML_NATIVE=OFF for portable binaries (still enables AVX2).

* feat: add golangci-lint pre-commit hook

Add .githooks/pre-commit that runs golangci-lint on changed files.
Install with `make setup-hooks`. Fix Makefile to install golangci-lint
v2 (matching .golangci.yml config version). Remove unused buildApp().
Fix .gitignore `cli` pattern that incorrectly matched `pkg/cli/`.

* feat(cli): add native CLI commands for all plugins

Add direct CLI access to plugin datasources without requiring the
Python sandbox. Commands make authenticated HTTP calls through the
credential proxy (or directly for Dora), providing instant results.

New commands:
- ep ethnode {syncing,version,health,peers,finality,header,block-number,beacon-get,exec-rpc}
- ep dora {networks,overview,validator,slot,epoch}
- ep clickhouse query
- ep prometheus {query,query-range,labels,label-values}
- ep loki {query,labels,label-values}

All commands support --json for machine-readable output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants