feat(plugin): a Claude Code plugin for context-guru, with the #141 review's blockers fixed - #160
Conversation
3627248 to
e9be584
Compare
e9be584 to
56790db
Compare
Review of #161 found the same defect this PR is themed on — claims that are not true — in seven more places, one of them in shipped code. All seven fixed, plus the `HasOffload` unit test the reviewer raised without asking for. Rebased onto current main first, so #142's preset-table guard and the preset pass below cannot re-fix or re-break each other. ## Merge-blocking **1. `proxy/proxy.go` named `mcp` as offloader-free.** It is not: `smartcrush` implements components.Offload, which this PR's own test asserts (`wantAdd: true`) and its own mirror-image mutation proves. I had corrected the test and left the comment wrong — the copy a future reader actually trusts. The fix deletes the list rather than correcting it. The comment now names the shapes affected (`off`, `safe`, any cachesplit-only configuration) and then says why enumerating presets here is the wrong move: a list in a comment is a second source of truth, and this one was wrong about `mcp` on its first draft. That is the whole argument for gating on the interface. **2. Five sites named the `cache` preset, which does not exist on this base.** It is #141's, and it reached here in the wholesale copy of `proxy/proxy.go` this PR already admits to, then travelled into the test files when they were split out of that branch. Reworded to name configurations that exist here; the underlying defect they describe is unchanged and still reproduces on `off` and `safe`. **3. `proxy/counttokens_test.go` carried copy-paste artifacts vet and gofmt cannot see.** A duplicated 3-line doc comment, and a 20-line orphan documenting a function that lives in `expandgate_test.go` under a different name and citing a test that exists nowhere. Both from the same cause: my splitter took each test's doc comment by scanning back to the previous blank line, which swallowed the FOLLOWING test's comment as a trailing block. A third artifact the review did not list is fixed too — `expandgate_test.go`'s doc comment still described "the preset's promise" and cited `docs/how-to/install-plugin.md` and an install skill, both of which belong to #160. ## The rest **4.** `docs/reference/config.md` and `docs/components.md` said `auto` injection has exactly two conditions. It has three. Both now say so, and say what the third is for. The cache-stability argument those passages make is unaffected — a pipeline does not change turn to turn either — so it gained a member rather than needing a rewrite. **5. `make build` now sets `CGO_ENABLED=0`.** The docs could claim "no C toolchain" all they liked while step 1 of the quickstart was `make build`, which needed one because the Makefile exported `CGO_ENABLED=1` for every target. Pointing readers at `build-static` would have fixed the sentence; making the DEFAULT build pure Go makes the claim true of the command the docs tell people to run. `CGO_ENABLED=1` stays for the test targets, where `-race` requires it, and the comment says exactly that. Verified: `CC=/nonexistent make build` produces a statically linked binary. README, CLAUDE.md and the quickstart no longer require a C toolchain. All five remaining `codesmart`-is-the-default sites are corrected — including two in `config/config.go`, which is how the claim spread to five documents: it sat three lines from the flag that disproves it. **6.** `docs/setup.md` overstated its own evidence, which is the exact sin this PR is about. It claimed CI removes the C compiler from `PATH` (with cgo off the toolchain never consults `CC`; that variable is a tripwire, not the mechanism) and that cross-compilation to four targets is asserted, when CI builds native linux/amd64 only. Now says what CI actually does, and states separately that the other three targets were verified by hand and are asserted at release time. Same overstatement fixed in the `ci.yaml` comment. **7.** `ci.yaml` promised a linked issue and linked nothing, and named a different flake than the PR body did. Both are real; the comment is about the campaign one, and now links #163. ## HasOffload unit tests `./components`: nil-safe, empty pipeline (the A/B control arm), reformatters-only, and an offloader in three positions. Revert-verified both ways — always-true fails the empty and reformatter cases, always-false fails the offloader cases. A registry-walking test was supposed to make it rot-proof, and **it skipped**: registrations happen in `components/all`, so a test inside `components` can neither see them nor import the package that does. A test that skips reads as coverage and is not, so it moved to `components/all`, where it runs — 21 of 21 registered components, 13 implementing Offload. It fails if either count is zero, because an all-false or all-true population would agree with a broken HasOffload. Full `go test ./...`, `go vet ./...` and `gofmt -l` clean; doc link/anchor checker re-run over every document touched. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
56790db to
295f551
Compare
…pand-tool gate, count_tokens, C-toolchain claim, preset facts) (#161) * fix: defects that surfaced while building the distribution funnel None of this is distribution work. Every item is a defect in code or docs that already shipped, found while doing #141, and split out at review request so it can be judged on its own — and so it can land whether or not the funnel does. ## 1. The expand tool was advertised where no marker can exist `expand.Inject` under `auto` gated on "the request declares tools" and "the store persists". Nothing asked whether the pipeline could produce a `<<cg:HASH>>` marker at all, so an offloader-free pipeline declared `context_guru_expand` to the provider — and every call against it must fail, because there is nothing in the Store to resolve. Measured on the real gateway route: tools SENT by client : [Read Bash] tools FORWARDED upstream: [Read Bash context_guru_expand] Affected `safe` and any cachesplit-only configuration, and — the one that matters most — **`off`, the A/B control arm**. A control that carries an extra tool declaration is not a control, and every measurement taken against it was comparing two arms that differed by more than the pipeline. The cost when it fires is a wasted round trip and a step of the user's turn: on a transcript containing marker-shaped text (this repo's own docs contain literal `<<cg:HASH>>`), a model calls the tool and gets "[expand: original for id ... is no longer available]". It was also a code-vs-comment contradiction, which is why nobody noticed: `Options.InjectExpand` documented the gate as requiring "an expandable marker", while `expand/inject.go` says "No marker condition, deliberately" three lines from the code. Both now describe what happens. `components.Pipeline.HasOffload()` answers by TYPE ASSERTION, not a list of component names: a name list is a second copy of "which components are lossy" and drifts the moment somebody adds one. `components.Offload` cannot be implemented by accident — it requires returning cache keys proving the original was stashed. Marker independence is preserved (the property that keeps the tools array byte-stable across a session, and hence the prefix cached): a pipeline does not change turn to turn. **Ten existing tests changed fixture.** Every test of the expand loop hand-seeds the Store to simulate an offload, but built its handler with `pipeline: []` — which cannot offload anything. Harmless while injection ignored the pipeline; now they use `offloadCapablePipeline` (`[linecap]`, which does not act on their short bodies). No assertion was weakened; each fixture now matches its own premise. ## 2. `POST /v1/messages/count_tokens` was not served Absent it, a client asking how big its context is gets a 404 and falls back to working it out with **inference requests** — billed calls, caused by a proxy whose purpose is to reduce them. Cheap to add, and it costs every routed user, not only the funnel. Forwarded verbatim, with no pipeline. Returning the compacted count would be smaller and would be wrong in the dangerous direction: the client budgets its own transcript from this number, and because every component fails open, the next request could forward the full body and take a 400. Over-reporting is recoverable; under-reporting is a failed turn. The cost of that choice is now documented in `docs/reference/routes.md`, where the route was absent entirely — a routed session self-compacts earlier than it needs to (115,933 reported vs 32,802 forwarded on a measured body). The hosted branch has tests, because that branch is the only thing standing between the multi-tenant service and an unmetered open forwarder that would send OUR credential upstream. ## 3. Our own docs said the binary needs a C toolchain `docs/setup.md`, `docs/hosted.md` and `docs/get-started/quickstart-proxy.md` all told evaluators to install one. It is needed for `go test -race` and for the optional `cg_skeleton` tag, not for the binary. setup.md went further and named **bifrost's tokenizer** as a cgo dependency, which it never was — o200k_base is embedded (`internal/tokens/tokens.go`). Asserted rather than re-claimed: a new `purego` CI job builds with `CGO_ENABLED=0` and `CC=/nonexistent-c-compiler`, checks the artifact is statically linked, starts it and probes /healthz. It also runs the packages whose behaviour depends on which components compile in — because `build-test` runs exclusively with `CGO_ENABLED=1` (the race detector needs it), so `TestEveryPresetBuilds` had **never executed in the configuration a user would build**. That guard exists for exactly the `preset: coding` / `unknown component "skeleton"` breakage. ## 4. Preset facts stated outside the guarded files (#143, #145) - The binary defaults to **`house`**; five sites said `codesmart` (README x3, `docs/reference/config.md`, `docs/get-started/quickstart-proxy.md` — the last is step 2 of the first page anyone runs). Anyone running the binary bare while reading those measured a different configuration than the published SWE-bench numbers describe. - README's `codesmart`/`codesafe` pipeline lists and `docs/get-started/connect-ibm-service.md`'s "Default pipeline" were stale — naming `toon`, retired after acting 0 of 5,752 production requests, and omitting components that do run. The IBM page's omission of `toolfilter` matters most: that page is what a prospective hosted tenant reads to decide what the service does to their traffic. All regenerated from the `presets` map. The two tables inside #142's drift guard are untouched here; these are the sites that guard cannot reach. ## Verification Five mutations, each proven to have landed in the source before its result was allowed to count: expand injection ungated -> TestExpandToolIsAdvertisedOnlyWhereMarkersCanExist FAIL on cachesplit-only, `safe`, and `off` HasOffload always false -> same test FAIL on `mcp` and the offloader pipeline: "mints markers but no longer advertises the expand tool, so a model cannot recover what it offloaded" count_tokens route unregistered -> TestCountTokensIsServed FAIL (404) count_tokens rewrites the body -> TestCountTokensIsServed FAIL hosted auth removed -> TestCountTokensHostedRequiresAuth FAIL (502, want 401) The second is the mirror-image check: it proves the gate did not trade one silent defect for another, an offloader whose output nothing can expand. Two things I got wrong on the way, recorded because both were caught by tests rather than by me: - I first asserted `mcp` had no offloader. `smartcrush` implements `components.Offload` (`components/offload/smartcrush.go`), so that pipeline genuinely mints markers and genuinely needs the tool. The case now asserts the opposite, with the reason — and it is the argument for asking the interface rather than keeping a hand-written list. - Copying `proxy/proxy.go` wholesale from the older distribution branch onto current main silently reverted #155's `effPreset`/`notePreset` work. `TestCompactRowNamesThePresetThatRan` — a test I had never read — failed with "the dashboard names a pipeline that did not run". The file was restored from main and the two edits re-applied on top; #155's change is intact. `go build ./...`, `go vet ./...`, `gofmt -l` and the full `go test ./...` are clean. One unrelated flake seen once and not reproduced: `TestConcurrentCallsDoNotRaceOnTheGateHistogram` failed in a full-suite run with "no single-flight follower ran ... the race was never exercised", then passed 8/8 in isolation and in two further full suites, and passes on clean main. Reported separately rather than papered over. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> * ci(purego): run one package binary at a time The new job runs `go test` over five package trees, and `go test` starts up to GOMAXPROCS package binaries in parallel. On a 2-core CI runner that added a second heavily-parallel run of the proxy package per PR, and under that contention a timing-sensitive control-plane test from #150 (TestCtlGetCampaignAggregatesPredictedAndRealPerTenant) failed on two unrelated PRs — then passed on a re-run of the same commit, and passes 3/3 whole-package on a 16-core box against both main and the affected branch. Filed as #163. Hunting that flake is not this job's business. Not provoking it is: `-p 1` costs about a minute and removes the contention this job introduced, without dropping any coverage. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> * fix: make this PR's own prose true, and unit-test HasOffload Review of #161 found the same defect this PR is themed on — claims that are not true — in seven more places, one of them in shipped code. All seven fixed, plus the `HasOffload` unit test the reviewer raised without asking for. Rebased onto current main first, so #142's preset-table guard and the preset pass below cannot re-fix or re-break each other. ## Merge-blocking **1. `proxy/proxy.go` named `mcp` as offloader-free.** It is not: `smartcrush` implements components.Offload, which this PR's own test asserts (`wantAdd: true`) and its own mirror-image mutation proves. I had corrected the test and left the comment wrong — the copy a future reader actually trusts. The fix deletes the list rather than correcting it. The comment now names the shapes affected (`off`, `safe`, any cachesplit-only configuration) and then says why enumerating presets here is the wrong move: a list in a comment is a second source of truth, and this one was wrong about `mcp` on its first draft. That is the whole argument for gating on the interface. **2. Five sites named the `cache` preset, which does not exist on this base.** It is #141's, and it reached here in the wholesale copy of `proxy/proxy.go` this PR already admits to, then travelled into the test files when they were split out of that branch. Reworded to name configurations that exist here; the underlying defect they describe is unchanged and still reproduces on `off` and `safe`. **3. `proxy/counttokens_test.go` carried copy-paste artifacts vet and gofmt cannot see.** A duplicated 3-line doc comment, and a 20-line orphan documenting a function that lives in `expandgate_test.go` under a different name and citing a test that exists nowhere. Both from the same cause: my splitter took each test's doc comment by scanning back to the previous blank line, which swallowed the FOLLOWING test's comment as a trailing block. A third artifact the review did not list is fixed too — `expandgate_test.go`'s doc comment still described "the preset's promise" and cited `docs/how-to/install-plugin.md` and an install skill, both of which belong to #160. ## The rest **4.** `docs/reference/config.md` and `docs/components.md` said `auto` injection has exactly two conditions. It has three. Both now say so, and say what the third is for. The cache-stability argument those passages make is unaffected — a pipeline does not change turn to turn either — so it gained a member rather than needing a rewrite. **5. `make build` now sets `CGO_ENABLED=0`.** The docs could claim "no C toolchain" all they liked while step 1 of the quickstart was `make build`, which needed one because the Makefile exported `CGO_ENABLED=1` for every target. Pointing readers at `build-static` would have fixed the sentence; making the DEFAULT build pure Go makes the claim true of the command the docs tell people to run. `CGO_ENABLED=1` stays for the test targets, where `-race` requires it, and the comment says exactly that. Verified: `CC=/nonexistent make build` produces a statically linked binary. README, CLAUDE.md and the quickstart no longer require a C toolchain. All five remaining `codesmart`-is-the-default sites are corrected — including two in `config/config.go`, which is how the claim spread to five documents: it sat three lines from the flag that disproves it. **6.** `docs/setup.md` overstated its own evidence, which is the exact sin this PR is about. It claimed CI removes the C compiler from `PATH` (with cgo off the toolchain never consults `CC`; that variable is a tripwire, not the mechanism) and that cross-compilation to four targets is asserted, when CI builds native linux/amd64 only. Now says what CI actually does, and states separately that the other three targets were verified by hand and are asserted at release time. Same overstatement fixed in the `ci.yaml` comment. **7.** `ci.yaml` promised a linked issue and linked nothing, and named a different flake than the PR body did. Both are real; the comment is about the campaign one, and now links #163. ## HasOffload unit tests `./components`: nil-safe, empty pipeline (the A/B control arm), reformatters-only, and an offloader in three positions. Revert-verified both ways — always-true fails the empty and reformatter cases, always-false fails the offloader cases. A registry-walking test was supposed to make it rot-proof, and **it skipped**: registrations happen in `components/all`, so a test inside `components` can neither see them nor import the package that does. A test that skips reads as coverage and is not, so it moved to `components/all`, where it runs — 21 of 21 registered components, 13 implementing Offload. It fails if either count is zero, because an all-false or all-true population would agree with a broken HasOffload. Full `go test ./...`, `go vet ./...` and `gofmt -l` clean; doc link/anchor checker re-run over every document touched. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com> --------- Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
a2c50f5 to
ceff711
Compare
…view's blockers fixed Split out of #141 as its own PR: all six of the review's blocking findings were in the plugin, and the release plumbing and conformance work should not wait behind them. The core lands in #141. `/plugin marketplace add rossoctl/context-guru` → `/plugin install` → `/context-guru:install`. Three skills over four scripts and two hooks. Default routing scope is `.claude/settings.local.json`: one repo, gitignored, `--global` an explicit opt-in — a base URL pointing at localhost breaks Claude Code everywhere a dead proxy is routed. **1. `/context-guru:uninstall` killed the user's own session and left the proxy running.** It ran `pkill -f "context-guru-proxy.*${PORT}"`. The port was passed through `LISTEN_ADDR` in the environment, so it appeared nowhere in the proxy's command line and the pattern matched no proxy — while it DID match the shell running the `pkill`, i.e. the session's own Bash tool. A user runs uninstall *because* their sessions are broken; this killed the session mid-command, reported nothing removed, and left the port held. Fixed with a handle rather than a better pattern: the starter passes `--listen` (so the port is in `argv` and `ps` is honest) and writes a pidfile under `~/.local/state/context-guru`; uninstall kills that PID, falls back to the socket's owner via `lsof`/`ss`, and confirms the process is ours before killing anything. The skill also no longer offers a broader pattern as a fallback — on a host running a production instance or a benchmark arm, that would take those down too. **2. `install.sh` could not install anything, and its documented fallback was missing.** Strict checksums now; `download_failed` (a tag with no assets) is documented as an outcome; the `go install` fallback the header comment described is implemented; curl's stderr no longer breaks the `key=value` contract the skill parses. **3. A dead proxy is a silent, indefinite hang** — no output on either stream — and `/context-guru:status` cannot diagnose it, because invoking a skill needs a model call, which is the broken thing. New `check-proxy.sh` on `UserPromptSubmit`: it probes `/healthz`, tries to restart, and otherwise prints what to do. A hook is the only thing that runs without a model turn. It never blocks a prompt. **4. The `cache` preset advertised `context_guru_expand`.** Fixed in #141 (the gate belongs in the proxy); the docs and the install skill here no longer claim otherwise where they were wrong. **5. `settings.py` destroyed the user's undo, and uninstall did not restore what it replaced.** The backup stamp was second-granularity with an overwriting `copy2`, so an install→uninstall round trip wrote both backups to the same path and the survivor held the POST-install state — the value it existed to protect was gone from the file AND the backup. Now microsecond-stamped and created with `O_EXCL`. And `replaced` was reported then forgotten, so after a `--force` install over somebody's gateway, uninstall left them with no base URL at all; the replaced value is now recorded and restored. `is_ours` deserves a note. The review suggested matching `http://(127.0.0.1|localhost|[::1]):\\d+ /anthropic` as ours, to stop a port change reporting a conflict against context-guru itself. A test caught why that is wrong: litellm's default is `http://127.0.0.1:4000/anthropic`, so a URL-shape rule would let uninstall delete somebody else's routing. Two local proxies are indistinguishable by URL, so `add` records the URL it wrote and later runs read that record. Anything unrecorded stays a conflict — for both add and remove. **6. The atomic write widened a credential-bearing file's mode** from 600 to 644 under the common umask, and `os.replace` onto a symlinked `settings.json` replaced the LINK with a regular file, so a dotfile-managed setup silently never received the edit. Mode is preserved; the path is resolved first. - **`start-proxy.sh` printed a dead dashboard link** — it advertised `/dashboard/` and never passed `--dashboard`, so the first line the plugin ever prints was a 404. Now passed, with `--dashboard-db` under the state directory: the default would write `./context-guru-dashboard.db` into the user's repository. - **Backups accumulated forever** (one per add and per remove). Pruned to the newest 10. - **The zero-value cases are now stated** where a first-run user reads them, and `status` checks the one that is both commonest and previously undocumented: **outside a git repository** there is no environment snapshot, so `cachesplit` skips and the saving is exactly zero. The status skill also no longer treats `acted: 0` / `savings_pct: 0` as a verdict — those count content removal, and this component relocates a breakpoint. - **`--idle-exit`'s 24h is the plugin's value, not the flag's default** (which is 0 = never). Said so, along with probes not counting as activity. - Upgrade path documented (`CONTEXT_GURU_UPGRADE=1`, `CONTEXT_GURU_VERSION`). The scripts are tested from Go (`context-guru-plugin/plugin_test.go`) so `go test ./...` and CI cover them. Seven mutations, each proven to have landed before its result counted: backup() back to overwriting copy2 -> TestBackupsDoNotClobberEachOther FAIL "both operations reported the same backup path ..., so one overwrote the other" uninstall stops restoring -> TestUninstallRestoresTheBaseURLItReplaced FAIL restored="" want "https://gateway.corp.example/anthropic"; env left {ANTHROPIC_AUTH_TOKEN:keep} mode no longer preserved -> TestSettingsPreservesFileMode FAIL realpath removed -> TestSettingsFollowsASymlink FAIL checksum fail-open again -> TestInstallRefusesAnUnverifiedDownload FAIL port back in the environment -> TestHookMakesTheProxyIdentifiable FAIL pidfile no longer written -> TestHookMakesTheProxyIdentifiable FAIL One of those is worth recording as a process note: my first attempt at the backup mutation reverted only the timestamp granularity and left the `O_EXCL` retry loop in place, so the name was still unique and the test passed — proving nothing. Reverting half a fix is its own way to get a vacuous result. The run above restores the original function whole. Pre-existing coverage still passes: settings merge/conflict/removal/backup, and the hook's silence in unrouted projects, idempotence, non-failure when the binary is missing, and its wait for `/healthz`. **Still not verified end to end in a real Claude Code session**, because `install.sh` resolves a GitHub release and no tag has published assets yet. That is the first thing to do once #141 merges and a tag exists. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… path #141 sets goreleaser's `wrap_in_directory: true`, so the release archive unpacks into a directory and the binary is no longer at the root — which is exactly what `[ -f "$TMP/$BIN" ]` assumed. It is wrapped for a reason worth not undoing: a flat archive plus the documented `tar xzf` with no `-C` overwrites the README.md and LICENSE of whatever directory the user is standing in, which for somebody evaluating a proxy for their agent is their own project. So find the binary rather than assume it. That works for either layout, and the failure it avoids is the worst-placed one available: a stranger's very first install, reporting `binary_not_in_tarball` — which reads as a broken release rather than a file that moved. Found by checking #141's packaging change against this script rather than by waiting for it to fail. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
ceff711 to
5d37e0c
Compare
amiddavid
left a comment
There was a problem hiding this comment.
Review of #160 — plugin only, 2 commits, 17 files
Reviewed at 5d37e0c in a separate worktree, rebased scope (origin/main...HEAD), so #141's
changes are excluded. I took your steer and spent the depth on settings.py and the
SessionStart/UserPromptSubmit hooks rather than the docs.
The six blocking findings from #141 are genuinely fixed, and the tests are not vacuous — each
one drives the real script and asserts an observable (the sentinel file, the recorded argv, the
pidfile naming a live PID, the file mode, the symlink target). TestInstallRefusesAnUnverified Download in particular tests the fail-closed property rather than the happy path. The
is_ours-by-record decision is right, and your reasoning for declining the URL-shape suggestion
holds: litellm's 127.0.0.1:4000/anthropic really does make a shape rule unsafe.
Two findings below are, I think, blocking, and both are on the surfaces you named. I verified each
by running the scripts rather than by reading — measurements are in the comments.
1 — the UserPromptSubmit hook cannot finish inside its own timeout. Measured 18s on the exact
dead-proxy path it exists for, against a timeout: 10. The hook is the whole fix for blocking
finding #3, and in the scenario it was written for it is killed before it prints anything.
2 — settings.py remove without --url deletes any base URL, including a stranger's. Measured:
a corp gateway with no context-guru record was removed with result=removed, exit 0. The safety
property currently lives only in the uninstall skill's prompt — a model remembering a flag — while
this script is the layer that is supposed to be deterministic and to "refuse to guess."
Four medium items and a tail of small ones follow. Nothing here needs a real install to close
except the go install PATH item, which I could not exercise: there is no Go toolchain on this
machine, so I could not run go test ./.... CI's build-test was still pending when I checked;
make cover does run ./... so the new package is genuinely gated (the piped go test | sed step
at ci.yaml:34 is only the coverage log, not the gate).
| { | ||
| "type": "command", | ||
| "command": "\"${CLAUDE_PLUGIN_ROOT}\"/scripts/check-proxy.sh", | ||
| "timeout": 10 |
There was a problem hiding this comment.
Blocking — this timeout is shorter than the work the hook does, on the path the hook exists for.
Measured, with routing configured, no proxy listening, and a binary present that never binds:
$ time check-proxy.sh # via CLAUDE_PLUGIN_ROOT, routed port, CONTEXT_GURU_BIN=a sleep script
ELAPSED=18s (UserPromptSubmit timeout is 10s)
The 18s is structural, not a fluke: check-proxy.sh probes /healthz, then invokes
start-proxy.sh, which probes again and then runs its own 15s health-wait loop. The
diagnostic cat <<EOF block is the last thing in the script, so at 10s the hook is killed and
the user sees nothing at all — the identical symptom (a prompt that produces no output) that
this hook was added to replace with an explanation.
The comment at start-proxy.sh:99 says "the hook's own timeout (60s in hooks.json) is the real
backstop." That is true for SessionStart and false here; this path has 10s.
Two fixes, and I think you want both:
- Print the note before attempting recovery, not after. The note is the deliverable; the
restart is opportunistic. Emitting it first means a kill at any point still leaves the user
informed. - Bound the wait when invoked from this hook — e.g. have
start-proxy.shhonour a
CONTEXT_GURU_HEALTH_BUDGET(seconds) thatcheck-proxy.shsets to ~5, and raise this
timeoutto 30 so a slow-but-successful recovery still gets to report success.
A test here would have caught it; see my comment on check-proxy.sh.
| # Ours is the URL passed in, or the one we recorded at install time — which covers the case | ||
| # where the configured port changed since. It is NOT "any loopback /anthropic URL": litellm's | ||
| # default is one of those, and uninstall must not delete somebody else's routing. | ||
| if args.url and current != args.url and not is_ours(data, current): |
There was a problem hiding this comment.
Blocking — remove with no --url deletes a base URL that is not ours, and reports success.
args.url defaults to "" (line 286), and this guard is if args.url and .... With --url
omitted the entire conflict check is skipped and the key is deleted unconditionally. Measured
against a file with someone else's gateway and no context-guru record:
$ settings.py remove --file settings.json # note: no --url
result=removed
was=https://gateway.corp.example.com
restored=
exit=0
The gateway is gone, restored= is empty because there was no record to restore from, and the
exit code says success. The module docstring advertises exactly this invocation as supported —
settings.py remove --file PATH [--url URL] — and states two lines later that a base URL that is
not ours "is a CONFLICT and exits non-zero."
What makes this blocking rather than cosmetic is where the safety currently lives. uninstall/SKILL.md
does pass --url, and its prose says "Passing --url is what keeps this safe" — so the property
that protects the user's gateway is a model remembering a flag in a prompt. This script is the
deterministic half precisely so that it is not. It is also the failure mode with the worst blast
radius you named: it edits ~/.claude/settings.json.
Fix, either way round:
- make
--urlrequired forremove(mirroring theaddcheck inmain()), or - treat a missing
--urlas "remove only ifis_ours(data, current)" — i.e. fail toward leaving
the user's configuration alone, which is whatis_ours' own docstring says it is for.
Worth a test: remove with no --url over a foreign URL must exit 2 and leave the file byte-identical.
|
|
||
| # Up to ~15s. A cold start is well under a second; the budget is for a loaded laptop, and the | ||
| # hook's own timeout (60s in hooks.json) is the real backstop. | ||
| for _ in $(seq 1 60); do |
There was a problem hiding this comment.
Medium — worst-case loop duration is ~138s, against a 60s SessionStart timeout.
The comment says "Up to ~15s", which holds only when the port is refused — then curl returns
instantly and 60 × 0.25s ≈ 15s. If something is listening but not answering (a hung proxy, a
half-open socket, a port taken by an unrelated service that accepts and stalls), each curl burns
its full --max-time 2. Measured against a listener that accepts and never replies:
one curl took 2054ms -> 60 iterations ≈ 138s (SessionStart timeout is 60s)
Consequence: the hook is killed at 60s, so the block at lines 109-116 never runs — the user gets
no log path, no /context-guru:status pointer, no tail of the log. And a hung port is one of the
likelier ways to reach that block, since a cleanly-absent proxy exits the loop fast either way.
Budget on wall clock rather than iteration count, and drop the per-probe timeout:
deadline=$(( $(date +%s) + ${CONTEXT_GURU_HEALTH_BUDGET:-15} ))
while [ "$(date +%s)" -lt "$deadline" ]; do
curl -fsS --max-time 1 "$HEALTH" >/dev/null 2>&1 && { ...; exit 0; }
sleep 0.25
doneThat also gives you the knob check-proxy.sh needs for the 10s-timeout finding on hooks.json.
| command -v go >/dev/null 2>&1 || return 1 | ||
| emit "fallback=go_install" | ||
| # CGO off: the binary is pure Go, and requiring a C toolchain here would reintroduce the gate. | ||
| if CGO_ENABLED=0 GOBIN="$DEST" go install "github.com/${REPO}/cmd/context-guru-proxy@${VERSION}" 2>"$TMP/go.err"; then |
There was a problem hiding this comment.
Medium — the go install fallback skips both the mkdir and the on_path report that the
tarball path does.
try_source_build returns 0 and line 126 exits 0 immediately, so everything from line 178 down is
bypassed on this path:
mkdir -p "$DEST"(line 178) never runs, yetGOBIN="$DEST"is set here. I could not test
whethergo installcreates a missingGOBIN— there is no Go toolchain on this machine — so
this needs confirming rather than taking from me; if it does not, the fallback fails on a clean
box where~/.local/bindoes not exist yet, which is the common case for the audience this
whole script targets.on_pathis never emitted (lines 193-197). So a user who lands on the fallback gets no
warning when$DESTis not onPATH— and~/.local/binfrequently is not. They then hit
start-proxy.sh:56: "routing is configured for port N but the proxy binary is not on PATH",
from a hook, in a later session, with nothing connecting it back to the install.install/SKILL.md
readson_pathand tells the user about it, so the skill is silent here too.
Both are fixed by making the fallback fall through to the shared tail instead of exiting: have
try_source_build set the path and break/skip to line 188, or hoist mkdir -p "$DEST" above
line 121 and factor the on_path case into a function both paths call.
Minor, same function: emit "fallback=go_install" is printed before the attempt, so it appears
even when the build fails. install/SKILL.md documents reading it together with the result, which
is correct — just noting the line is "attempted", not "used".
|
|
||
| If nothing answers, start it in the foreground of a background shell and read the log rather | ||
| than declaring victory: | ||
|
|
There was a problem hiding this comment.
Medium — this manual-start command reintroduces the dead dashboard link the PR lists as fixed.
It omits --dashboard and --dashboard-db, which start-proxy.sh:89-90 passes deliberately.
--dashboard defaults to false (cmd/context-guru-proxy/main.go:105), so a proxy started this
way does not serve /dashboard/ — and step 6, ~25 lines below, tells the user
Dashboard:
http://127.0.0.1:<port>/dashboard/— the four billed token tiers are where the cache
effect is visible.
which is a 404. That is the same defect as the "dead dashboard link — it advertised /dashboard/
and never passed --dashboard" item in the PR description.
It also does not self-heal, and that is the part worth fixing rather than documenting:
start-proxy.sh is idempotent on /healthz (line 51), so once this hand-started proxy is up the
hook will never replace it — and with --idle-exit 24h it holds the port for a day. The user's
dashboard is broken for that whole window with no signal as to why.
check-proxy.sh:48 has the same omission in the command it prints to the user.
Simplest fix: make both places invoke start-proxy.sh rather than restating the command line —
the script already self-gates on ANTHROPIC_BASE_URL, which step 5 correctly explains is unset in
the installing session, so it would need an override (CONTEXT_GURU_FORCE=1, or just documenting
ANTHROPIC_BASE_URL=http://127.0.0.1:$PORT/anthropic start-proxy.sh). Failing that, add the two
dashboard flags in both places.
Credit where due: step 5's explanation of why the hook does nothing in the installing session is
exactly right, and it is the kind of thing that usually gets left out.
| import glob | ||
|
|
||
| try: | ||
| found = sorted(glob.glob(f"{path}.context-guru-backup-*"), key=os.path.getmtime) |
There was a problem hiding this comment.
Low — glob.glob treats [, ? and * in the path as pattern syntax, so pruning silently does
nothing for a settings file under a directory containing them (~/projects/foo[1]/.claude/...).
Backups then accumulate forever, which is the thing KEEP_BACKUPS exists to stop, and the failure
is invisible because prune_backups is best-effort by design.
glob.glob(glob.escape(path) + ".context-guru-backup-*") fixes it.
| t.Helper() | ||
| p, err := exec.LookPath(name) | ||
| if err != nil { | ||
| t.Skipf("%s not available: %v", name, err) |
There was a problem hiding this comment.
Low, process rather than code: requireTool uses t.Skipf, so on a runner without python3 or
bash every test in this file skips and go test is green. ubuntu-latest has both today, so
this is latent — but the whole point of the package header is that these scripts get the coverage
their blast radius warrants, and a skip is indistinguishable from a pass in CI output.
For python3 and bash specifically I'd t.Fatalf instead: they are guaranteed on the CI image,
so an absence means the image changed and you want to hear about it. Keep t.Skipf for anything
genuinely optional.
| [ -n "$found" ] || die "binary_not_in_tarball: no $BIN anywhere in $TARBALL" | ||
|
|
||
| mkdir -p "$DEST" || die "cannot_create_$DEST" | ||
| install -m 755 "$found" "$DEST/$BIN" || die "install_failed_to_$DEST" |
There was a problem hiding this comment.
Low, and unverified — please confirm before acting on it, I could not. On Linux, opening a
currently-executing binary with O_TRUNC fails with ETXTBSY, and coreutils install truncates
rather than replacing the inode. If that holds here, the upgrade path (CONTEXT_GURU_UPGRADE=1)
fails with install_failed_to_$DEST in exactly the situation upgrades happen in — a proxy that is
running, which with --idle-exit 24h is most of the time. macOS permits the write, so this would
not reproduce on a dev box.
I could not test it: no Go toolchain and no Linux here.
If it does hold, install -m 755 "$found" "$DEST/$BIN.new" && mv -f "$DEST/$BIN.new" "$DEST/$BIN"
avoids it — rename swaps the directory entry and leaves the running image alone — and is also
atomic for anyone starting the proxy concurrently.
| # setsid detaches the proxy from this hook's process group so it survives the hook returning | ||
| # and is not killed with the session's process tree. --idle-exit is what eventually reaps it. | ||
| STARTER=(setsid) | ||
| command -v setsid >/dev/null 2>&1 || STARTER=(nohup) # macOS has no setsid |
There was a problem hiding this comment.
Low — the comment above claims the starter "detaches the proxy from this hook's process group so
it survives the hook returning and is not killed with the session's process tree." setsid does
that; the macOS fallback nohup does not — it only makes the child ignore SIGHUP, leaving it in
the caller's process group. So on macOS, which is presumably where most of this was developed, a
signal delivered to the session's process group still reaches the proxy.
disown at line 93 does not help either — that is bash job-table bookkeeping, not a process-group
change.
In practice nohup + disown usually survives, so I would not change the mechanism on my say-so.
But the comment states a property the fallback does not provide, and the next person to read it
will trust it. Either say "setsid where available; nohup only blocks SIGHUP", or get the real
property portably with (trap '' HUP; exec "$BIN" ... &) in a subshell.
| ## Quickstart (60 seconds) | ||
|
|
||
| Download a release binary — statically linked, **no Go and no C compiler needed** — or build | ||
| **Claude Code users — two commands, no toolchain, and no API key needed on a Pro/Max |
There was a problem hiding this comment.
Nit — "two commands" sits directly above a three-command block, and docs/how-to/install-plugin.md
spells out why it is three ("The first two are once per machine. The third is once per repo").
marketplace.json compresses it further to "in one command."
Since the split is the genuinely reassuring part of the design — the machine-wide step and the
per-repo routing decision are separate on purpose, and the routing one is the one with blast
radius — I would say so rather than round the count: "two commands to install, one per repo to
route."
…ured before and after
Thirteen findings. Two were blocking, and both were on the surfaces I had named as the highest-risk
ones in the review request, which is the part worth noting: naming them was not the same as having
tested them.
## The hook took 19s and its timeout was 10s — on the path it exists for
check-proxy.sh exists for one case: routing configured, nothing listening, so a prompt produces
NOTHING — no error, no readable timeout, the session just hangs. The hook replaces that silence with
an explanation. But the explanation is printed LAST, after an attempt to recover, and recovery
called start-proxy.sh with its default 15s health wait.
Measured on that exact path: 19s, against a UserPromptSubmit timeout of 10s. So the hook was killed
before it printed anything, and the user got the identical symptom the hook was added to remove —
now with a hook that appeared to have handled it.
The shared cause was in start-proxy.sh: the health wait was `for _ in $(seq 1 60)` with
`--max-time 2`, commented "up to ~15s". That is only true when the port is REFUSED, where curl
returns instantly. Against a socket that ACCEPTS and never answers — a hung proxy, a half-open
socket, an unrelated service on the port — each probe burns its full timeout: measured 2046ms, so
~122s against SessionStart's 60s. The hook was killed there too, so the failure report at the
bottom of that script (log path, status pointer, last lines of the log) never ran. A hung port is
one of the likeliest reasons to need that report and was the one case that never produced it.
* the wait now budgets on WALL CLOCK, honouring CONTEXT_GURU_HEALTH_BUDGET (default 15s)
* check-proxy.sh asks for 5s, since it runs from a much tighter hook
* hooks.json allows 30s, so even the expensive shapes finish with room
Measured after: 5s on the dead-proxy path, 11.1s on the worst shape (a port that accepts and
stalls, where all three probes cost their full timeout), and the hung-port start path is 17s with
the failure report actually printing.
I did NOT take the other half of the suggested fix — printing the note before attempting recovery.
The note is guaranteed to survive a kill that way, but this hook's common case is a silent
successful recovery after an --idle-exit between two prompts, and a paragraph about a dead proxy on
every one of those is noise on a path that is working. The budget is what makes the ordering safe,
and TestCheckHookFinishesInsideItsOwnTimeout reads the timeout out of hooks.json so the two cannot
drift apart again.
## `remove` with no --url deleted anybody's base URL and reported success
The conflict check read `if args.url and current != args.url and not is_ours(...)`. So omitting
--url — an invocation the module docstring advertises as supported — skipped the check entirely.
Measured against a corporate gateway with no context-guru record: `result=removed`, `restored=`
empty, exit 0. The user's gateway silently gone, reported as success, in ~/.claude/settings.json.
What made it blocking was not the branch but where the safety lived: uninstall/SKILL.md passes
--url, and its prose said that flag "is what keeps this safe" — so the property protecting the
user's configuration depended on a model remembering a flag in a prompt. This script is the
deterministic half precisely so that it does not have to be.
The check is unconditional now: with no --url, is_ours() decides alone, i.e. remove only what we
recorded installing. The escape hatch for an unrecorded value is to name it with --url, which the
skill already does. The prose no longer claims a safety property it does not provide.
## Two of the review's findings were false, and I tested rather than argued
Both were flagged as unverified — no Go toolchain, macOS only. Go lives on a Linux box here, so:
* `go install` DOES create a missing GOBIN (Linux, Go 1.26.4), so the fallback needs no mkdir
* `install -m 755` over a RUNNING binary SUCCEEDS on Linux — no ETXTBSY, because coreutils
`install` unlinks the destination first
I still install-then-rename, but for the real reason, which the false premise was sitting next to:
that unlink is a window in which $DEST/$BIN does not exist, and a SessionStart hook firing in
another project during an upgrade would report "the proxy binary is not on PATH". rename(2) swaps
the directory entry in one step.
Also declined as written: `O_EXCL` on the fixed temp name `.context-guru-tmp`. It closes the mode
window, but a leftover temp file from a crash then makes every later save fail until somebody
deletes it by hand — trading a mode window for a permanent lockout of the file this script exists
to edit. mkstemp has neither problem.
## The rest
* install/SKILL.md's manual-start command and check-proxy.sh's printed command both omitted
--dashboard/--dashboard-db, reintroducing the dead /dashboard/ 404 this PR lists as fixed — and
because start-proxy.sh is idempotent on /healthz, the hook would never replace such a proxy, so
it stayed broken for the whole 24h idle-exit window
* install.sh's go-install fallback returned before the shared tail, so `on_path` was never
emitted; ~/.local/bin frequently is not on PATH, and the user found out from a hook in a LATER
session with nothing tying it to the install. Both paths call report_path now
* `fallback=go_install` renamed to `fallback=go_install_attempted`: it is printed before the
build, so it appeared even when the build failed
* settings.py: the temp file was created 0644 and corrected only after the whole file was
written — a window where a replacement for a 0600 settings file holding ANTHROPIC_AUTH_TOKEN is
world-readable. mkstemp creates it 0600
* prune_backups used glob.glob on the PATH, so under a directory like `foo[1]` it matched nothing
and pruning silently never happened. glob.escape
* both port gates were PREFIX matches, so PORT=8787 also matched 87871 — the hook would start our
proxy under a user routed to a different local proxy there. The trailing "/" is always present,
since every URL we write ends in /anthropic
* the setsid/nohup comment credited both with detaching the process group. Only setsid does that;
nohup blocks SIGHUP and leaves the child in this process group, and `disown` is bash job-table
bookkeeping. The mechanism works, so the comment was the bug — corrected rather than replaced
* README said "two commands" above a three-command block, and marketplace.json compressed it to
"one command". The split is the reassuring part of the design — install once, route per repo —
so both now say that instead of rounding the count
## Verification
Six properties for check-proxy.sh, which was the only one of the six blocking fixes from the
previous review with no test at all — and its absence is exactly what hid the 19s defect. The hook
timeout is READ FROM hooks.json rather than retyped, and the timing assertion measures the
accept-and-stall shape rather than the cheap refused-port one: on the cheap shape the assertion
would have passed at almost any timeout, which is the opposite of what it is for.
Plus: the `remove`-without---url regression (the file must come back BYTE-IDENTICAL, since a
refusal that rewrites the file has already done the thing it refused), the fallback's on_path
report, glob-escaped pruning, and a prefix-port case in both silence tests.
requireTool now fails instead of skipping for python3 and bash. A skip and a pass are
indistinguishable in CI output, so on a runner without either, every test in this file skipped and
`go test` was green.
Nine mutations, each proven to have LANDED in the source before its result counted, each failing
its named test, each file restored byte-identical afterwards, and the full suite green again after
the sweep:
check-proxy asks for no health budget -> TestCheckHookFinishesInsideItsOwnTimeout
hooks.json timeout back to 10s -> TestCheckHookFinishesInsideItsOwnTimeout
printed command loses the dashboard flags -> TestCheckHookFinishesInsideItsOwnTimeout
remove's check conditional on --url again -> TestUninstallRefusesAForeignBaseURLEvenWithNoURLGiven
health wait back to an iteration count -> TestStartHookBudgetsOnWallClockNotIterations
check-proxy port gate back to a prefix match -> TestCheckHookIsSilentWhereRoutingIsNotConfigured
start-proxy port gate back to a prefix match -> TestHookIsSilentAndInertWhereRoutingIsNotConfigured
fallback exits without the PATH report -> TestInstallReportsPATHFromTheSourceFallbackToo
prune_backups without glob.escape -> TestBackupPruningSurvivesAGlobbyPath
One gap stated rather than papered over: the 0600 temp-file window is fixed but not tested. It is a
window between two syscalls, and a test asserting on it would be asserting on a race it cannot
observe. Verified by reading; TestSettingsPreservesFileMode still covers the resulting mode.
`go test ./...` and `gofmt -l` clean; docs pass the anchor check.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
|
All thirteen. Two of them were blocking, both on the two surfaces I asked you to go deep on — and the fact that I named those surfaces as the highest-risk ones and had still not tested them is the honest summary of this round. Pushed as The 19s hook with a 10s timeoutReproduced exactly, and it is worse than a slow hook: Your diagnosis of the shared cause was right, and I confirmed the mechanism separately: the Wall-clock budget via I took one half of your fix and not the other. The note stays last. Printing first guarantees it survives a kill, but the common case here is a silent successful recovery after an idle-exit between two prompts, and a paragraph about a dead proxy on every one of those is noise on a working path — it also lands in the model's context for that turn. The budget is what makes the ordering safe. If you still think the guarantee beats the noise, say so and I will flip it; it is a judgement call, not a disagreement about the facts.
|
amiddavid
left a comment
There was a problem hiding this comment.
Re-review of 6577320 — all thirteen verified fixed; two new findings, one of them mine to own
I re-ran the measurements rather than reading the diff. Everything I filed is genuinely fixed, and
the three places you did not do what I suggested are all better than what I suggested. Detail below,
then two new items — the first of which I should have caught in round one and did not.
The two blocking findings, measured before and after
| Path | Before | After | Bound |
|---|---|---|---|
check-proxy.sh, dead proxy |
18s | 5s | 30s timeout |
check-proxy.sh, accept-and-stall |
— | 5s | 30s timeout |
start-proxy.sh, accept-and-stall |
~138s | 16s | 60s timeout |
The diagnostic is delivered in every case now, including the stall shape. I measure the stall
shape at 5s rather than your 11.1s — probably harness differences, and in the safe direction either
way; the assertion has margin regardless.
settings.py remove verified on three shapes:
- no
--url, foreign gateway, no record →result=conflict, exit 2, file byte-identical ✓ - no
--url, our own recorded URL → stillresult=removed✓ (the fix did not overshoot into
makingremoveuseless without the flag) --urlmismatching a foreign URL → stillresult=conflict, exit 2 ✓
Your three pushbacks: all accepted, and one correction to my own record
1. My two unverified items were wrong, and you tested what I could not. go install creating a
missing GOBIN, and install not hitting ETXTBSY because coreutils unlinks first — I flagged
both as needing confirmation precisely because I had no Linux box and no toolchain, and you went and
got the answer. Thank you for testing rather than deferring. The install-then-mv you kept is
better justified than my version of it: the unlink window is real, rename(2) closes it, and the
comment in the script now records the corrected reasoning instead of my wrong premise, which is the
right thing to leave behind for the next reader.
2. Diagnostic staying last: you are right, and TestCheckHookRecoversSilently is what settles
it. My reasoning was "the kill-survival guarantee is unconditional, ordering makes it free." Yours
is that the guarantee is only worth anything on the failure path, while the common path after an
idle-exit is a silent successful recovery — and a dead-proxy paragraph on every one of those lands
in the model's context on a turn where nothing is wrong. That cost is certain and recurring; mine
was hypothetical once the budget exists. Budget first, ordering second is the correct dependency,
and the test now pins the silence so nobody "helpfully" moves the note up later. No flip needed.
3. mkstemp over O_EXCL: correct, and I missed the failure mode. A leftover
.context-guru-tmp from a crash or a full disk would have made every subsequent save fail
permanently — turning a brief mode window into a permanent lockout of the one file this script
exists to edit, recoverable only by hand. Trading a transient defect for a persistent one is
strictly worse. The except BaseException unlink is a good addition I had not asked for.
I also accept the untested-window gap as stated: a race between two syscalls is not honestly
assertable, and "verified by reading" is the right label rather than inventing a test that would
pass either way.
On the test I asked you to check
TestCheckHookFinishesInsideItsOwnTimeout is right, and the change you describe is the one that
matters: measuring the accept-and-stall shape instead of a free port is what makes it sensitive to
anything. Reading the bound from hooks.json via hookTimeout and asserting against limit/2
rather than limit is better than what I suggested — the half-margin means it fails on a real
regression instead of only on a loaded runner, and it cannot drift from the config. Asserting
--dashboard appears in the printed recovery command is a nice touch: it pins finding 5 in the same
place, so the 404 cannot come back through the text.
TestCheckHookIsSilentWhereRoutingIsNotConfigured covering 127.0.0.1:87871 as a prefix of 8787
pins the gate fix at the level it actually broke. That is the table-driven version I would have
written.
Nothing further on the other eleven — report_path shared by both install paths, glob.escape,
t.Fatalf for python3/bash, the nohup comment, and the "two commands to install, one per repo
to route" wording all check out.
Merge order
Agreed and unchanged: release.yaml via workflow_dispatch → v0.1.0 → this. I am not treating
"has never run end to end" as a finding, for the reason you gave — it is gated on a tag existing,
not on anything in this diff. Worth saying plainly in the PR when it merges, so the first person to
run /context-guru:install against a real release knows they are the first.
Two new comments below. The first is a genuine miss on my part from round one — same defect class
as #141's finding 1, which you noted is not hypothetical here. It is unchanged by this commit, so it
is not a regression; I simply did not see it. The second is a small test-hygiene item on new code.
| fi | ||
|
|
||
| if [ -n "$pid" ]; then | ||
| kill "$pid" && rm -f "$PIDFILE" |
There was a problem hiding this comment.
Medium, and my miss from round one — the ownership check is printed after the kill it is
supposed to gate.
This is not a regression from 6577320; it has been here since 0abd995 and I did not catch it. Same
defect class as #141's finding 1, which is why I want it fixed rather than left as prose ordering.
Read the skill top to bottom, which is how it gets executed:
- lines 74-78 —
if [ -n "$pid" ]; then kill "$pid" && rm -f "$PIDFILE" - line 81 — "Before killing anything, confirm the PID is ours — the port may be held by
something else entirely, and this step must not kill a stranger's process" - line 85 — the
ps -p "$pid" -o command= | grep -q context-guru-proxycheck
The instruction says "before killing anything" and sits after the block that kills. An agent working
through this file in order has already sent the signal by the time it reads the guard, and the check
then reports on a process that is already dying. The safety property is stated but structurally
unreachable.
It matters most on the path that produces an untrustworthy PID in the first place. The pidfile is
ours by construction, but the fallback at lines 66-72 is lsof/ss on "whoever holds the port" —
which is explicitly there to cover a stale pidfile and a hand-started proxy, i.e. exactly
the cases where the PID may belong to something else. A recycled PID from a stale pidfile passes
kill -0 too.
The fix is to make it one unskippable block rather than two blocks and a warning, so the ordering
cannot be got wrong by reading:
if [ -z "$pid" ]; then
echo "(nothing listening on ${PORT})"
elif ps -p "$pid" -o command= | grep -q context-guru-proxy; then
kill "$pid" && rm -f "$PIDFILE"
else
# The port is held by something that is not ours — a hand-started service, or a recycled PID
# from a stale pidfile. Report it and stop; do not kill it.
echo "NOT OURS — pid $pid holds ${PORT} and is not context-guru-proxy; left alone"
fiThen keep the liveness re-check at lines 91-93 as it is, since "did the kill work" is a separate
question from "is it ours".
One consequence worth stating in the skill while you are there: if the check says NOT OURS, step 1
has already removed the routing, so the user is safe and unblocked — the leftover is a port
conflict to report, not something to escalate on. That is the reassurance that stops the next reader
reaching for kill -9.
| if err != nil { | ||
| return | ||
| } | ||
| held = append(held, c) // hold it open, answer nothing |
There was a problem hiding this comment.
Low, new-code hygiene: held is appended here on the listener goroutine and iterated by the
t.Cleanup closure at line 812, which runs on the test goroutine. That is an unsynchronised
read/write of the same slice, and make cover runs go test -race — so if a connection is being
accepted while cleanup runs, this is a race-detector failure rather than a test failure, which is a
confusing thing to land on.
Registering t.Cleanup from inside the goroutine is also a hazard in its own right: t.Cleanup
called after the test has finished panics, and nothing here orders the goroutine's registration
before the test's end.
Cleanup order makes it narrow rather than safe — the held cleanup is registered second so it runs
first (LIFO), while the goroutine is still blocked in Accept and can still append. In practice
curl is done by then, so I would not expect this to fire often; "not often" is the bad frequency
for a race in CI.
Simplest fix is to drop held entirely — closing the listener is what ends the goroutine, and the
accepted connections die with the process at test end:
t.Cleanup(func() { ln.Close() })
go func() {
for {
c, err := ln.Accept()
if err != nil {
return
}
defer c.Close() // held open for the life of the goroutine, answers nothing
}
}()If you want the connections explicitly closed, register the cleanup on the test goroutine before
starting the listener goroutine and guard the slice with a mutex. Either way the property the helper
exists for — accept, never answer — is unaffected.
Worth saying: stallingPort is the right helper to have added. The shape it produces is what turned
both timeout tests from assertions that would pass at any value into ones that catch the defect.
There was a problem hiding this comment.
Correcting my own severity: this is not "low", it is the build-test failure on 6577320. It
fired on the first CI run, not rarely.
--- FAIL: TestCheckHookFinishesInsideItsOwnTimeout (11.14s)
FAIL github.com/rossoctl/context-guru/context-guru-plugin 20.047s
Three WARNING: DATA RACE reports, all the same shape — Read at ... by goroutine 65 inside
testing.go:1317 (that is tRunner running the registered cleanups) against Previous write at ... by goroutine 66, the accept loop. The frames are plugin_test.go:812, :813, :817 and :821:
the held iteration and c.Close() in the cleanup, versus the append and Accept in the
goroutine. Run: https://github.com/rossoctl/context-guru/actions/runs/33815232946/job/100845795572
I said "not often is the bad frequency for a race in CI" and then underrated it anyway — I reasoned
that curl would be finished by cleanup time, but this test holds three probes against the
stalling port, and ln.Close() runs after the held cleanup (LIFO), so the accept loop is still
live and writing while the cleanup reads. Sorry for filing it as a nit; the fix in my comment above
is unchanged and is what unblocks CI.
Two things worth taking from the failure beyond the fix.
Your 11.1s was right and my 5s was the outlier — the run measures 11.14s, so that was Linux/CI
versus my macOS box, and your figure is the one to design the margin against. That also means the
timing assertion passed: 11.14s against a limit/2 bound of 15s. The defect I filed on
hooks.json is fixed and the test proves it on the platform that matters; the only thing red here
is the helper's bookkeeping. Good news for the substance, and an argument for the half-margin you
chose over my limit — at 30s the bound is 15s and 11.14s fits, but it is not roomy, so I would
resist any future trimming of that timeout.
Second: this is test-only code. No production path touches held, so nothing about the plugin's
behaviour is implicated — worth stating in the commit message that fixes it, so the next reader of
the history does not go hunting for a proxy defect behind a red build-test.
Everything else in the run is green (DCO, build, purego, trivy), and this is the only failing
test in the only failing package.
…data race in my own test Two items from the re-review of 6577320. One is a real defect in the uninstall skill; the other is a race I introduced in TEST-ONLY code in the previous commit, which is what turned build-test red — no production path touches it, so a red build-test on 6577320 is not a proxy defect. ## uninstall sent the signal before the check that was supposed to gate it skills/uninstall/SKILL.md had `kill "$pid"` in one bash block, and "Before killing anything, confirm the PID is ours" as PROSE with its own snippet BELOW it. These skills are executed the way they read, top to bottom, so the signal was already sent by the time the guard was reached. The guard was decoration. It matters most on the lsof/ss fallback immediately above it, which exists precisely for a stale pidfile or a hand-started proxy — i.e. the cases where the PID may belong to something else entirely. `kill -0` does not help: a recycled PID satisfies it perfectly well. This is the same shape as the defect that had this skill killing the user's own Claude Code session: a destructive command whose safety condition lives somewhere the reader gets to afterwards. Now one if/elif/else, so the ordering cannot be got wrong by reading it: * no pid -> say so * pid is not ours -> print WHAT is on the port, signal nothing, and LEAVE THE PIDFILE ALONE (removing it would strand a proxy of ours still running under another pid) * pid is ours -> kill, then remove the pidfile Given an EXECUTING test rather than more careful prose, because this class has now recurred twice and reading is what missed it both times: TestUninstallDoesNotSignalAProcessThatIsNotOurs extracts the fenced block straight out of the skill and runs it against a stubbed `ps`. A stranger's process must not be signalled and its pidfile must survive; ours must be signalled. Mutation-checked: with the two-block ordering restored, the block sends SIGTERM to `/usr/bin/postgres`. ## The data race was mine, in the test helper, and -race is the only thing that sees it stallingPort registered its connection cleanup with t.Cleanup from INSIDE the listener goroutine, so the cleanup closure read `held` on the test goroutine while the accept loop appended to it on another. Three WARNING: DATA RACE, all on that slice. Fixed by ownership rather than a mutex: the goroutine that appends is the only thing that reads, and it closes the connections in its own deferred func. The test goroutine only closes the listener, which is what makes Accept return and lets that teardown run. Worth recording why I did not catch it: I ran the suite repeatedly on a box without -race, where this is invisible — a real race with no symptom until the scheduler interleaves badly. CI runs `go test -race` via `make cover`, so CI was the first thing that could see it. New concurrent test code gets -race locally from now on; the full suite is verified under it here. Also worth recording, because it is the vacuous-pass shape again: the first version of the uninstall test stubbed `kill` as a FILE ON PATH. `kill` is a bash builtin, so the stub was never consulted — the block's `kill -0` liveness probe failed for a pid that does not exist, it fell through to the socket-owner lookup, and the run exercised none of the branch under test. It only failed loudly because one of the two cases asserts a POSITIVE outcome (kill WAS invoked); the negative case alone would have passed for the wrong reason. A shell function overrides the builtin, so the preamble defines one. ## Two notes from the reviewer's own measurements, kept because they inform later changes * their 5s vs my 11.1s on the accept-and-stall shape is macOS vs Linux/CI, and the CI run measured 11.14s — so the Linux number is the one to design the margin against * that leaves 11.14s against a limit/2 bound of 15s: real margin, but not roomy. The 30s UserPromptSubmit timeout should not be trimmed later without re-measuring on Linux `go test -race ./...` and `gofmt -l` clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
|
Both done, pushed as The race — yours to flag, mine to have causedYou were right to lead with it, and right that it is test-only: nothing in the proxy touches I had it fixed from the same CI log before your message arrived, and the diagnosis matches: Why I did not catch it is the part worth recording. I ran that suite repeatedly on a box without On underrating it as "low hygiene": I would not spend much on that. You filed the right observation with a wrong severity, and the severity was recoverable in one CI run. Filing it at all is what made the fix a five-minute job. uninstall killing before the checkReal, and the diagnosis is exact — the guard was decoration, because these files execute the way they read. Your point that it bites hardest on the Collapsed into one Given an executing test rather than better prose, because this class has now recurred twice in this PR's lineage and reading is what missed it both times. That test has a process note of its own, and it is the same trap as the race: my first version stubbed Your measurementsTaken, and both recorded in the commit body rather than just agreed to here:
Thanks for verifying the two blocking fixes by measurement rather than by reading the diff — the three Merge order unchanged: |
amiddavid
left a comment
There was a problem hiding this comment.
Re-review of 82b8f6f — both items fixed and independently verified; no blocking findings left
The uninstall ordering fix, verified by re-running it rather than reading it
I extracted the fenced block from uninstall/SKILL.md myself and ran it against a stubbed ps, in
both shapes and then with your old two-block ordering restored:
| Block | ps reports |
signalled | pidfile survives | said NOT OURS |
|---|---|---|---|---|
| 82b8f6f | /usr/bin/postgres -D /var/lib/postgres |
no | yes | yes |
| 82b8f6f | context-guru-proxy --listen 127.0.0.1:8787 |
yes | no (removed, correctly) | no |
| mutation: old two-block order | /usr/bin/postgres … |
yes | no | yes |
So the mutation does signal postgres, and the fix does not. Your report matches exactly, and the
test is sensitive to the defect rather than to its surroundings.
The pidfile surviving on NOT OURS is a better call than what I proposed, and I had not thought it
through. I only said "do not remove it"; you gave the reason that makes it load-bearing — a proxy of
ours may still be running under a different pid, and "refuse to touch this process" plus "throw away
our own handle" combine into a proxy nobody can stop by the documented route. Deleting the handle is
the more expensive half of that pair.
skillBlock is the right shape for testing prose that gets executed, and it fails on zero or more
than one match rather than silently taking the first — which is the failure mode that would otherwise
make it test the wrong block after an edit. I confirmed exactly one of the three bash blocks in that
skill contains the needle.
The race, fixed by ownership
Correct, and better than the mutex I offered as the alternative: held is now touched only by the
listener goroutine and closed by its own defer, with t.Cleanup(ln.Close) registered on the test
goroutine as the thing that unblocks Accept. No shared state, so nothing to synchronise — and
t.Cleanup is no longer called from a non-test goroutine, which was the second hazard in the same
five lines. Noting in the commit that no production path touches stallingPort is the right thing to
leave in the history.
Agreed on the process point, and it is the more transferable lesson: a non-race run cannot see this
class at all, so -race locally before pushing is the cheap version of what CI told you.
The trap you flagged — I went looking, and there are exactly two instances left
Thank you for naming the shape; it made this a mechanical search rather than a guess. Your
kill-as-a-file version is a textbook vacuous pass, and the reason it surfaced is exactly what you
said: the positive row asserted a positive outcome, so the stub had to actually be consulted for
the test to go green. That is the property worth generalising.
By that standard the stub-based tests come out clean. TestInstallRefusesAnUnverifiedDownload is
refusal-only on its face, but it asserts checksum_unavailable — a string only reachable if the curl
stub successfully served the tarball first, since real curl would have produced download_failed
instead. So the stub is proven consulted by the assertion itself.
TestInstallReportsPATHFromTheSourceFallbackToo asserts built_from=source and covers both
on_path values, so the go stub is likewise pinned.
Two tests remain in the shape you told me to distrust, and I filed the detail on one of them below.
Neither is wrong today — both are saved by a positive control that lives in a different test — and
that is the fragility worth one line each.
CI
DCO, build, purego, trivy green on 82b8f6f; build-test was still running when I posted, and
I am watching it. Merge order unchanged and still the real gate: release.yaml via
workflow_dispatch → v0.1.0 → this.
Nothing here blocks. From my side this is ready once build-test is green.
| // more here: this hook runs on EVERY PROMPT in every project on the machine, not once per session. | ||
| // Anything it prints lands in the model's context for that turn, so a regression is noise on every | ||
| // turn the user takes, in projects that have nothing to do with context-guru. | ||
| func TestCheckHookIsSilentWhereRoutingIsNotConfigured(t *testing.T) { |
There was a problem hiding this comment.
Following up on the trap you flagged — this is one of the two tests still in that shape, and the
only reason I am filing it is that you asked me to look for it.
Every assertion here is an absence: exit 0, and empty output. Nothing in the test requires the
script to have done anything, so it cannot distinguish "the gate correctly declined" from "the
script exited before reaching the gate." Demonstrated by running its four rows against a
check-proxy.sh gutted to a bare exit 0:
all four rows pass: yes (exit 0 + silence are the only assertions)
TestHookIsSilentAndInertWhereRoutingIsNotConfigured at line 291 is the same shape — the sentinel's
absence plus silence, with no in-test evidence the sentinel mechanism works at all.
Both are saved today, but from a distance: the positive control for this one is
TestCheckHookFinishesInsideItsOwnTimeout (routed → it speaks), and for line 291 it is
TestHookStartsTheProxyAndWaitsForHealthz (routed → the sentinel appears). That is real coverage, so
neither test is wrong. The fragility is that the control is somewhere else: a future change that
skips, renames or narrows the positive test takes the meaning out of the silence test without
touching it or making it fail, and a silence test that has quietly stopped proving anything is the
exact failure you just spent a round on.
Cheapest fix is to make the control local — one more row in each table asserting the routed case
does act:
{"routed to our port", "http://127.0.0.1:8787/anthropic", wantSilent: false},with the assertion inverted for that row (non-empty output here; sentinel present at line 291). Then
the table itself proves the mechanism is live, and the silence rows mean something on their own.
Low priority and entirely your call — the coverage exists either way. It is the "would this still
fail if the thing under test were removed?" question, which is the same question your kill stub
answered the hard way.
|
|
Addendum — I ran the grep I suggested, and the blast radius is exactly one test. There are seven
The remaining five assert nothing from that block within their following lines. So the fix is genuinely one test's wait, not a pattern to unpick across the file — which makes the Worth noting for whoever picks it up: |
The last finding from the #160 review, and it is the trap I had just flagged to the reviewer, pointed back at my own tests. TestCheckHookIsSilentWhereRoutingIsNotConfigured and TestHookIsSilentAndInertWhereRoutingIsNotConfigured assert only ABSENCES: no output, no proxy started, exit 0. So neither could distinguish "the gate declined" from "the script exited before it ever reached the gate". Gut either hook to a bare `exit 0` and every row passed. A positive control did exist — TestHookStartsTheProxyAndWaitsForHealthz for the starter — but a control living in a DIFFERENT test is one these tests cannot rely on: narrow or skip that test later and the meaning drains out of these without anything failing. So each table gets one more row asserting the routed case DOES act: * the starter must launch the stand-in and report the failure when it never answers /healthz * the checker must speak about a dead proxy Both use a port of their own from freePort rather than the literal 8787, so neither probes nor starts anything against a developer's real proxy, and the routed starter row sets CONTEXT_GURU_HEALTH_BUDGET=1 so the control costs a second rather than fifteen. Verified the way the reviewer demonstrated the gap: with each script replaced by `#!/usr/bin/env bash\nexit 0`, the corresponding test now FAILS on its control row (it passed before), and both files restored byte-identical afterwards. Note on the red build-test this commit does not address: TestArriveCancelsAndReports in ./proxy is failing, and it is not this PR — #160 touches no file under proxy/, and the defect is on main (keepalive.go, 50e3966, 2026-08-22, well before this branch). waitPings polls k.pings, incremented at keepalive.go:899 immediately after k.send() returns, but the value the test then asserts — e.refreshed — is not written until line 989, after the error check, the event construction and ev.Price(). The wait therefore returns before the state under test exists, and `refreshed = 0` is precisely the pre-write value. I could not reproduce it locally (60 runs under -race, then 200 at -cpu=1, all green), so rather than assert the mechanism from reading I widened the suspected window: a 200ms sleep inserted between those two points reproduces `refreshed = 0, want 48576` exactly. keepalive.go restored byte-identical; the probe is not part of this branch. Filed separately rather than absorbed here, so a plugin PR does not carry an unrelated proxy change. Worth recording one property of it: -race cannot catch that one — both accesses are properly mutex guarded, so the bug is ordering, not unsynchronised memory. Exactly inverted from the stallingPort race in the previous commit, which -race caught immediately and no plain run could see. `go test -race ./...` and `gofmt -l` clean. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
Review complete — all findings resolved, CI green at
|
…l machines
Everything here came from running the thing rather than reasoning about it: a hosted agent pod, and
a fresh Claude Code driven headlessly on a laptop. Four defects, and the first two would have hit
essentially every user.
## install.sh reported "no published release yet" while the release sat there
Version resolution went through `api.github.com`, which allows 60 requests/hour for unauthenticated
callers, counted PER IP — so the budget is shared by everyone behind one NAT: a corporate network, a
CI fleet, a shared box. Exhausted, it answers 403, resolution produced the empty string, and the
script reported:
result=error
reason=no_release_found: no published release yet for rossoctl/context-guru
which is FALSE. Observed on a corporate IP with `{"limit":60,"remaining":0,"used":60}` and v0.1.1
published and downloadable. The message then sent the user off to build from source — needing a Go
toolchain, which is precisely the gate this whole distribution exists to remove.
Resolution now follows the releases/latest WEB redirect, which carries no such budget and yields the
same concrete tag. The API is a fallback, and the two failures are finally distinguished:
`github_rate_limited` says the resolution failed and that this says nothing about whether a release
exists; `no_release_found` keeps its literal meaning. Same machine, same exhausted limit, minutes
apart: `result=error / no_release_found` became `version=v0.1.1 / checksum=verified /
result=installed`.
No unit test could have caught this: the install test stubs `curl`, so it never meets a real rate
limit. It took one run on a real network.
## The install broke the session doing the installing
Steps were: write the settings key, then start the proxy. Claude Code picks a settings `env` change
up LIVE — it does not wait for a restart — so between those two steps the session pointed at a proxy
that did not exist yet. Observed, in a fresh session running this skill:
API Error: Connection refused — a firewall or proxy may be blocking it (ConnectionRefused)
The session died there, having written the routing key and never reaching the step that starts the
proxy. It left the project routed with nothing listening: the hang state this entire design warns
about, manufactured by the installer.
Order reversed. The proxy goes up and answers /healthz FIRST, settings second, so the instant routing
takes effect something is already there — and if it does not come up, the key is never written, since
an unrouted project with no proxy is a working project.
This also corrects a claim the skill made in its own closing summary: "the setting takes effect in a
new session". It does not, and that mistaken belief is exactly what made the old order look safe.
## Chaining did not survive the session that set it up
A hosted agent's own gateway is not decoration: it holds the credential AND rewrites model names
(one pod maps `claude/haiku…` to the real id), so a proxy that forwards straight to
api.anthropic.com sends model names the API has never heard of and every request fails. Chaining
behind it is the only configuration that works there.
`ANTHROPIC_UPSTREAM` in the environment IS honoured by the binary — verified against the released
v0.1.1 with a fake upstream: env only, no flag, and the request arrived (`HIT /v1/messages`). A
review of this claimed the opposite, from the absence of a log line; absence of a log is not absence
of behaviour, which is the same trap as a stub that is never consulted.
But the hook only sees what the settings `env` block passes it, so:
* `start-proxy.sh` passes `--anthropic-upstream` explicitly when configured, via
`CLAUDE_PLUGIN_OPTION_UPSTREAM` or `ANTHROPIC_UPSTREAM`, and says what it chained behind;
* `plugin.json` gains an `upstream` user option (validated — `claude plugin validate` rejected the
first version for a missing `title`);
* `settings.py add --upstream` writes the second key in the SAME atomic save, records it, and
`remove` takes back only an upstream it recorded writing — one the user set is theirs.
Without that last piece chaining worked only until the running proxy idled out. The first real
install ended precisely there, telling the user to paste a key in by hand.
## The skill interrogated the machine, and Claude Code denied it
Unbounded investigation instructions plus an unexpected base URL led the model to enumerate
credential environment variables and profile processes. Claude Code's auto-mode classifier denied it:
[Credential Exploration] ... systematically probing for the presence of multiple unrelated
credential types ... reconnaissance consistent with setting up a credential-intercepting proxy
That is the worst possible impression for a proxy plugin to make, at the exact moment somebody is
deciding whether to trust it. None of it was necessary: the only fact needed is whether
`$ANTHROPIC_BASE_URL` names our port, and `echo` answers that.
So the skill now forbids it by name — no credential-variable enumeration, no `ps`/`ss`/`lsof` — and
was rewritten to be decisive: the disclosure is three lines rather than six paragraphs, the default
path asks for nothing, and exactly three cases stop (an unverifiable checksum, `--global`, and a
foreign base URL, where the answer is usually to chain rather than ask). Verified: the same flow now
runs to `exit=0` with nothing flagged and no questions asked.
## And the missing step in the documented flow
`/plugin install` prints "Run /reload-plugins to apply", and until you do, the session has no
`/context-guru:*` skills — so the very next documented command answers `Unknown command:
/context-guru:install`. Every reader following the README in order hit that. Both the README and
install-plugin.md now show four commands and name the symptom.
## Verification
`claude plugin validate` passes on both manifests — worth admitting it had never been run before
today, having built and reviewed this plugin across three rounds without once using the validator
that ships with the tool it targets. It found a real schema error immediately.
The full flow was then driven end to end in an ISOLATED `CLAUDE_CONFIG_DIR`: marketplace add from
the branch, plugin install, the install skill in a fresh session. It completes, chains behind the
laptop's own gateway automatically, and leaves nothing outside the scratch config dir — checked
afterwards rather than trusted. `--anthropic-upstream` appeared in the running proxy's argv, which is
what proves the flag plumbing rather than the flag's existence.
New tests: the upstream key round trip, and that an upstream we did not write survives uninstall.
`go test -race ./...` and `gofmt -l` clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
The second real install on a hosted agent got all the way to the last step and was denied:
Denied by auto mode classifier ∙ [Traffic Redirection] The agent is launching a third-party
plugin's proxy script that repoints ANTHROPIC_BASE_URL to a local proxy intercepting all Claude
API traffic before forwarding to an unverified upstream — the user installed the plugin by name
but never named or confirmed this specific traffic redirection/interception mechanism.
Everything before it worked: three lines of disclosure, no credential probing, the gateway on the pod
detected, chaining chosen over replacement, proxy before settings. Then this.
The objection is fair and this change does not try to defeat it. What it removes is a self-inflicted
part: the command being judged contained
`ANTHROPIC_BASE_URL="http://127.0.0.1:8787/anthropic" start-proxy.sh`, because the script self-gates
on that variable naming its port and the install runs before routing is written. So the skill was
literally issuing a traffic redirection in order to start the proxy that serves it.
`CONTEXT_GURU_FORCE=1` replaces that prefix. The install asks for what it means — start the proxy —
and nothing in the command line reassigns the variable that routes traffic.
There is a second, independent reason the prefix had to go: **Bash permission rules match by command
PREFIX**, so no rule naming this script could ever cover an env-prefixed invocation. Users could not
grant the thing they were being asked about. With the flag in front, `Bash(<path to scripts>/**)` does
cover it.
The gate itself is unchanged and just as strict without the flag — verified rather than assumed:
A: unrouted, no force -> exit 0, 0 bytes of output, nothing launched
B: unrouted, FORCE=1 -> launched: --listen … --idle-exit=24h --dashboard --dashboard-db …
C: forced, upstream set -> … --anthropic-upstream http://127.0.0.1:24180
Case C also confirms the plugin option reaches argv, which is the better route for chaining than an
env var the install has to remember: `start-proxy.sh` reads `CLAUDE_PLUGIN_OPTION_UPSTREAM` directly,
so a user who sets **Upstream base URL** in `/plugin configure` is chained in every later session
without the settings env block being involved at all.
One note on that test, since it initially reported a failure the script had not committed: the fake
binary did `exec sleep 60`, which REPLACES argv, so `pgrep -f <fake path>` found nothing and case B
looked like "nothing started" while the script had in fact launched it and waited out the health
budget. The fake now records its own argv to a file. Inferring from a process list what a script did
is the same mistake as inferring behaviour from a missing log line.
## Docs
Two questions a first-time user hits before any of this, neither of which was documented anywhere:
* `/plugin install` offers three scopes (user / project / local). Now a table, with which file each
writes and who it reaches — and a warning that project scope COMMITS a proxy plugin to a shared
repository, so everyone who clones gets a SessionStart hook that launches a proxy. That is a team
decision, not a personal one.
* `/plugin configure` offers four options. Now a table, all with working defaults, and the one that
matters called out: **Upstream base URL** must be set on a platform that supplies its own gateway,
because that gateway holds the credential AND may rewrite model names.
Plus the denial above, documented as an expected prompt rather than a defect, with the three ways
through (approve it; run the two commands yourself with `!`; add a permission rule) and the reason the
plugin cannot grant itself the third: a plugin that could approve its own traffic interception would
be worth distrusting.
Also kept explicit throughout: plugin scope and routing scope are independent questions, which the
two tables sitting near each other otherwise invites people to conflate.
`go test -race ./...` and `gofmt -l` clean; same-page doc anchors verified.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…ok harmless
Third install attempt on a hosted agent. The previous fix removed the `ANTHROPIC_BASE_URL=` prefix
that the classifier had objected to, and it was denied again — for a broader reason, quoted in full
because it changes what the right response is:
Denied by auto mode classifier ∙ [Traffic Redirection] Starting a third-party plugin's local proxy
that intercepts and forwards the agent's own Anthropic API traffic (ANTHROPIC_UPSTREAM=127.0.0.1:24180,
CONTEXT_GURU_FORCE=1) reroutes model traffic through unvetted code without the user having named or
reviewed that specific interception.
So the objection is not to the wording of the command. It is to the act: start a proxy that intercepts
this session's model traffic. **That is correct, and this change stops trying to get around it.** No
rewording of a skill can make "route all your model traffic through third-party code" not be that, and
an agent should not decide it on its user's behalf. Installing a plugin by name is a different consent.
What was still broken is that the user had no way to say yes.
## --force becomes an argument, because env prefixes are ungrantable
Bash permission rules match by command PREFIX. `CONTEXT_GURU_FORCE=1 /path/to/start-proxy.sh` does not
begin with the script path, so NO rule naming this script can cover it — and the previous round's fix
still produced an env-prefixed command, as did the upstream, which the model passed as
`ANTHROPIC_UPSTREAM=…`. Two prefixes, both ungrantable. A user who wanted to approve "this plugin may
start its proxy" had nothing to write.
`--force` as an argument makes the command `<script> --force`, which `Bash(<scripts dir>/**)` covers.
`CONTEXT_GURU_FORCE=1` still works, so nothing that already used it breaks.
The upstream has the same fix available and it already existed: `start-proxy.sh` reads
`CLAUDE_PLUGIN_OPTION_UPSTREAM`, so chaining needs no env prefix either. The skill now says to use the
plugin option rather than an env var, and to put no prefixes on that command at all. Verified:
the gate must still hold with no force -> nothing launched
<script> --force -> LAUNCHED
CONTEXT_GURU_FORCE=1 (back-compat) -> LAUNCHED
--force + the upstream OPTION -> … --anthropic-upstream http://127.0.0.1:24180
The last line is the one that matters: chained, with zero environment prefixes, so the whole command is
coverable by one rule.
## The skill now treats the denial as an outcome, not an obstacle
It says to expect it, quotes it, says it is correct, and hands the user exactly three options: approve
the prompt, run the one command themselves with `!`, or add a permission rule. Explicitly: do not offer
a fourth way, do not reword the command to look less like what it is, and never write the routing key
while the proxy is down.
Both agents that hit this handled it well — stopped, reported precisely what was and was not done,
declined to route around the denial. The skill now asks for that behaviour rather than relying on it.
## Docs
`install-plugin.md` gains the prefix-matching explanation, because it is the difference between a rule
that works and one that silently does not: a rule naming the script covers `start-proxy.sh --force` and
not `SOMEVAR=1 start-proxy.sh`. If any skill ever prints a command with env prefixes ahead of the script
path, no rule will help and the user is back to approving each time — worth being able to recognise.
Also states plainly that on a hosted agent one approval is the expected shape rather than a defect: the
pod's own gateway is doing the authenticating, and a classifier stopping an agent from putting a
third-party proxy in front of it unilaterally is the system working.
`go test -race ./...` and `gofmt -l` clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…consent gates
Findings from a third hosted-agent attempt, this time carried past the proxy start by hand.
## Silence at the gate cost somebody three rounds of guessing
A human pasted the long env-prefixed invocation into a session. The paste split across two lines, so
`ANTHROPIC_UPSTREAM=… CONTEXT_GURU_FORCE=1` became a bare assignment statement with no command attached
— set in the shell, never exported. `start-proxy.sh` then ran unrouted, hit its gate, and exited 0 with
no output, no log and no pidfile. Indistinguishable from "the script never ran at all".
It took two more attempts and a read of the script's source to identify. The gate's silence on stdout is
right — that branch runs in every session in every project, and a line there would be noise — but
silence with NO TRACE ANYWHERE is a different thing, and it is the state that made this undiagnosable.
The gate now appends one line to the proxy log:
2026-09-06T08:28:23Z declined: ANTHROPIC_BASE_URL=http://127.0.0.1:24180 does not name port 51105;
pass --force to start anyway
Stdout is still exactly 0 bytes — verified, since that property is what makes a user-scope hook
acceptable at all.
(The immediate cause is already fixed by the previous commit: `--force` as an argument means there is no
env prefix for a paste to break. This is about what happens when something else produces the same
silence.)
## There are TWO consent gates on a hosted agent, and I had documented one
The settings write was denied as well, after the proxy was up:
Denied by auto mode classifier ∙ [Traffic Redirection] The action rewrites the Claude Code settings
file to route all future API traffic (including the Authorization credential, which the plugin's own
comments note "passes straight through") through a locally-run proxy sourced from a third-party
marketplace plugin the user only generically installed — the user never named this specific
endpoint/credential-passthrough as something they approved.
That is the sharper of the two objections and it quotes this repo's own comments back at it, correctly.
Routing means the user's credential passes through this binary; on a chained install it must, because
that is how the platform gateway keeps authenticating. Checksum verification proves the download matches
what the repo published — it does not make the code trustworthy, and nothing in this path is signed.
Documented as the expected shape rather than a defect: an unattended install cannot complete on a hosted
agent, by design. One permission rule on the plugin's `scripts/` directory covers both gates, since both
are scripts in it.
## Also documented: whether the trial can show anything at all
That pod's working directory is not a git repository, so there is no environment snapshot to split,
`cachesplit` reports `verdict: skipped`, and the saving is exactly zero — structurally, not as a
warm-up. Worth knowing BEFORE deciding whether to route a credential through anything: the honest
recommendation in that situation is not to install. `/context-guru:status` already reports this
correctly; the docs now say it where somebody reads it first.
`go test -race ./...` and `gofmt -l` clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…t is not on PATH
DAM (a hosted coding-agent platform) is the environment this plugin's users will actually be in, not an
edge case — so the things that only break there are the things that matter most.
`install.sh` reported `on_path=false` on every attempt, and the advice was "add ~/.local/bin to your
shell profile". On that platform the advice does not survive: the writable directories reset on pod
restart. And the consequence is silent, which is what makes it serious rather than annoying — the
SessionStart hook resolves the proxy BY NAME, so:
* the install reports success, correctly;
* routing works in the session that set it up, because that session started the proxy by hand;
* every later session's hook fails to find the binary and starts nothing;
* the failure mode the hook exists to catch is a prompt that hangs with no error.
So the safety net is gone precisely where it is needed, and nothing anywhere says so.
`settings.py add --bin <abs path>` writes `env.CONTEXT_GURU_BIN` into the same block the hook inherits.
`start-proxy.sh` already honoured that variable and already accepted an absolute path, so this needs no
change there — the gap was only that nothing persisted it. Recorded in our own metadata too, so
uninstall removes a path only if we wrote it; one the user set is theirs.
The skill now passes `--bin` whenever step 1 reports `on_path=false`, and still mentions the PATH gap,
since a user also wants `context-guru-proxy` on their command line.
That completes the set of things a hosted agent needs written rather than assumed:
ANTHROPIC_BASE_URL route this project through the proxy
ANTHROPIC_UPSTREAM chain behind the platform's own gateway, which holds the credential and
rewrites model names
CONTEXT_GURU_BIN find the binary at all, when its directory is not on PATH
All three in one atomic write with one backup, all three removed by uninstall, none of them removed if
the user set them.
Verified: the three-key round trip, and that an upstream or a binary path we did NOT write survives
uninstall. `go test -race ./...` and `gofmt -l` clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…n runs
DAM is where this plugin's users are, and there the install needs approval at two gates. If granting
it takes several rules — or an unwritable one — the org-scale answer is 'approve it every time',
which is not an answer.
Bash permission rules match by command PREFIX. The skills invoked settings.py as
`python3 ${CLAUDE_PLUGIN_ROOT}/scripts/settings.py …`, so that command begins with `python3` and a
rule naming the plugin's scripts directory cannot match it — while start-proxy.sh and install.sh,
invoked directly, are covered by exactly such a rule. Granting the plugin therefore required either a
blanket `Bash(python3:*)` (far too broad — it allows any Python the agent cares to run) or nothing.
settings.py is already 0755 with a `#!/usr/bin/env python3` shebang, so it can simply be invoked
directly. All four invocations across the install, status and uninstall skills now are, which makes
one rule sufficient:
{"permissions": {"allow": ["Bash(~/.claude/plugins/cache/context-guru/**)"]}}
Also fixed two places in the install skill left inconsistent by the last few rounds: step 3 still
demonstrated an env-prefixed `ANTHROPIC_UPSTREAM=… start-proxy.sh` — the exact form the previous
commit removed for being ungrantable — and step 1 still said to tell the user to fix their PATH,
without mentioning the --bin option that actually repairs the hook.
`go test -race ./...` and `gofmt -l` clean.
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
… an argument
Fourth hosted-agent attempt, and this one reached the last step with a correctly pre-approved
permission rule. Two of my own decisions were wrong.
## --force was denied for its name, and the name was the problem
Denied ∙ [Safety Bypass Flag] The agent runs a third-party plugin's start-proxy.sh with --force
(after investigating that flag's role in bypassing checks) to redirect the agent's own API/proxy
traffic through an untrusted marketplace plugin, with no user authorization of this specific bypass.
Fair reading. Nothing is being bypassed: the gate asks 'is this project routed to us?', and during an
install the honest answer is 'not yet, that is the next step'. The previous round renamed this from an
env var to a flag to make it grantable and picked the one word that reads as overriding a safety check.
It is `--unrouted` now, which says what it means. `--force` still works so nothing that learned it
breaks, but no skill or doc mentions it.
## CLAUDE_PLUGIN_OPTION_* does not reach a Bash tool call
The agent had the Upstream base URL option configured — and the variable was absent from the shell, so
the script could not read it. Plugin options reach HOOK environments; they do not reach a script the
install skill runs. My design assumed otherwise, so on that pod the skill either had to env-prefix the
command (ungrantable, and denied twice already) or start an unchained proxy that fails every request.
`--upstream <url>` is an argument now, and takes precedence over both env sources. The whole command
is arguments and nothing else, so one permission rule covers it:
(no args, routed elsewhere) -> nothing launched (gate intact)
--unrouted -> launched
--force -> launched (back-compat)
--unrouted --upstream <url> -> … --anthropic-upstream http://127.0.0.1:24180
--unrouted --upstream=<url> -> … --anthropic-upstream http://127.0.0.1:24180
## Docs: the rule I told people to paste was unpasteable
The user pasted the JSON object into `/permissions`, which stores the literal text as a single allow
entry — matching nothing, while looking like it worked. `/permissions` wants the rule alone:
Bash(/home/you/.claude/plugins/cache/context-guru/**)
Documented both forms, with an absolute path rather than `~`, and why one rule suffices only because
every command is invoked directly with no env prefixes.
One test note, the same shape as twice before: my first version of the argument test checked the argv
log immediately after the script returned, before the async launch had written it, and reported
'nothing launched' for cases that had in fact launched. A trace showed the script doing the right thing
throughout. Reading evidence too early is its own way to get a confident wrong answer.
`go test -race ./...` and `gofmt -l` clean.
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
…q, and --bin as an argument
The install now completes on a hosted agent. Three changes from what it took to get there.
## --bin as an argument, so nobody has to improvise a symlink
The successful install took a detour: start-proxy.sh resolves the binary by name, ~/.local/bin was not
on that pod's PATH, so the script reported 'the proxy binary is not on PATH' and exited — and the agent
worked around it by symlinking the binary into a directory under the plugin cache that happened to be on
PATH. It worked, and it would have broken silently at the next plugin update or pod restart, for a reason
nobody would connect to a symlink made days earlier.
install.sh already prints `path=`. `--bin <path>` takes it, overrides both env sources, and is covered
by the same permission rule as the rest of the command. Reproduced the failure and confirmed the fix:
without --bin, binary off PATH -> not launched: "the proxy binary is not on PATH"
--unrouted --bin <abs path> -> LAUNCHED
--unrouted --bin … --upstream … -> LAUNCHED with --anthropic-upstream
## The permission rule is now a recommended pre-req, not a troubleshooting footnote
And it is framed for any deployment with auto mode or a restrictive permission policy, not for one
platform: the two steps it covers — starting the proxy, writing the routing key — are where model
traffic and the credential travelling with it start passing through a third-party binary. 'The user
installed a plugin called context-guru' really is different consent from 'the user approved intercepting
their API traffic', so the rule is the user saying the second thing once, deliberately.
Also now recorded as CONFIRMED rather than hedged: with a correctly formed rule in place beforehand,
both previously-denied steps ran without a prompt and the install completed. The failure mode to check
for is a JSON object pasted into /permissions instead of a bare `Bash(...)` string.
## A hosted-agents section, written for the shape rather than the platform
Any environment where the platform supplies its own local gateway — agent pod, managed workspace,
injected corporate proxy — with the signal to recognise ($ANTHROPIC_BASE_URL already set, by something
other than you), why chaining is required rather than optional (that gateway holds the credential and
may rewrite model names), and the three things to expect that are not errors. Plus the check worth doing
first: outside a git repository the cache preset has nothing to act on and the saving is zero.
## Neutrality pass, since all of this came from one platform
Verified rather than asserted:
* no platform-specific value in anything that runs — scripts, hooks and manifest are clean of ports,
paths and names; `24180` survives only as a sample value in one test;
* defaults unchanged: port 8787, preset cache, idle-exit 24h, upstream EMPTY;
* the plain path behaves exactly as before — with no gateway configured, start-proxy launches with no
--anthropic-upstream flag at all, and `settings.py add` writes exactly one env key, the user's own
keys untouched;
* a full fresh-Claude-Code install still runs end to end on a laptop, in an isolated CLAUDE_CONFIG_DIR:
marketplace add, install, reload, skill — completing with no questions and nothing flagged.
The hosted-agent work made this MORE general, not less: before it, the plugin assumed the base URL always
comes from a settings file, which is wrong on any containerised setup.
`go test -race ./...` and `gofmt -l` clean; all 13 same-page doc anchors resolve.
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
left a comment
There was a problem hiding this comment.
Review of the install rework — 8 commits, eb9eac9..4cb2817
Reviewed at 4cb2817. This is a substantially different install flow, so I treated it as a new pass
rather than a re-check, and I spent the time where you asked: start-proxy.sh argument parsing,
settings.py, and the not-platform-specific claim. Four findings, two of which I would fix before
merge, both in the two files you named as thinnest and highest blast radius. Everything below was
measured by running the code, not read.
First, the claim you most wanted checked: nothing here is DAM-specific — it holds
I checked it three ways of my own choosing rather than repeating yours:
- No platform literal in shipped code. Grepped
scripts/,hooks/and.claude-plugin/for
platform names, private hostnames,.corp, and bare IPv4 — nothing outside127.0.0.1and the
gateway.corp.exampleplaceholder in a comment. - A no-gateway install writes exactly one key.
add --urlwith no--upstream/--binproduces
env keys: ['ANTHROPIC_BASE_URL']. The two new keys are strictly opt-in. - The no-upstream path emits no upstream flag. With
ANTHROPIC_UPSTREAMand
CLAUDE_PLUGIN_OPTION_UPSTREAMboth unset, the proxy's argv is
--listen … --idle-exit=24h --dashboard --dashboard-db …and carries no--anthropic-upstream.
So the platform work is additive and the laptop path is unchanged. I did not re-verify a full fresh
install end to end — no release assets, same gate as before.
The install.sh rate-limit fix is the right shape: web redirect first, API only as a fallback, and
github_rate_limited distinguished from no_release_found. Falling back when -fI fails (a HEAD
that GitHub refuses) is handled, and the ''|releases|latest guard catches the no-releases redirect.
Conflating "rate limited" with "no release exists" was the worse half of that bug, and it is now
impossible to conflate.
The step-4-before-step-5 reordering is correct and the reasoning in the skill is the clearest
statement of it — env applying live is exactly why the installer could break its own session, and
"if it did not come up, stop and do not write the settings key" is the right gate.
On the vacuous-evidence shapes you hit three more times
All three of yours were in test code, and all three produced a confident wrong reading — a PATH stub
for a shell builtin, exec sleep making pgrep miss a live process, and reading an argv log before
the async write. Worth naming what they share: each broke the link between the observation and the
thing observed, while leaving the observation itself intact. The stub existed, the process existed,
the log existed. That is why they read as evidence. The cheap general defence is the one your kill
case demonstrated by accident — assert a positive outcome somewhere in the same test, so the
machinery has to actually work for the test to pass.
I found no instance of that shape in the new production code, and the two new tests
(TestChainingUpstreamSurvivesIntoLaterSessions, TestBinPathSurvivesAMachineWhereItIsNotOnPATH)
both assert positives and both check the uninstall-conservatism half.
Coverage gap, confirmed
Your self-assessment is accurate: no test passes --unrouted, --upstream or --bin to
start-proxy.sh. Grepping the suite, those strings appear only in unrelated assertions and in the
settings.py tests. Both findings below live exactly in that gap, and the settings.py ones survive
because the new tests exercise only the fresh-file added path.
| current = env.get(KEY) | ||
| if current == args.url: | ||
| emit(result="unchanged", file=args.file, base_url=current, | ||
| note="already routed to this proxy") |
There was a problem hiding this comment.
Fix before merge — --upstream and --bin are silently dropped on two of cmd_add's three exit
paths, and one of them is the repair case that motivated the keys.
The new writes happen only after both early returns. Measured:
1. Already routed to the same URL — the re-run case. A user whose install died at the last step
(which is how all four DAM attempts ended), or whose install predates these keys, re-runs
/context-guru:install to repair it:
$ settings.py add --file s.json --url http://127.0.0.1:8787/anthropic \
--upstream http://gw:4000 --bin /opt/cg/context-guru-proxy
result=unchanged
note=already routed to this proxy
exit=0
env keys after: ['ANTHROPIC_BASE_URL']
Success-shaped, exit 0, and neither key is written. There is no way to add the upstream to an
already-routed project through this script — re-running install, the obvious remedy, is a no-op. On a
gateway platform that leaves every later session's hook starting a proxy aimed at
api.anthropic.com, which is precisely the failure the ANTHROPIC_UPSTREAM key exists to prevent.
2. The repointed path. Our own URL on a different port:
result=repointed previous=http://127.0.0.1:8787/anthropic
env keys after: ['ANTHROPIC_BASE_URL']
The routing moves, the chaining config does not follow it — so changing the configured port quietly
un-chains the proxy.
Both are silent, and the skill cannot detect either from the output (see my note on the emit line).
The unchanged branch is the one I would not leave: its whole meaning is "nothing to do here", and
with these arguments present that is now false. Suggested shape — compute the desired env once and
compare, rather than treating the base URL as the sole thing that can be out of date:
want = {KEY: args.url}
if args.upstream: want[UPSTREAM_KEY] = args.upstream
if args.bin: want[BIN_KEY] = args.bin
if all(env.get(k) == v for k, v in want.items()):
emit(result="unchanged", ...)
return 0and then let the write path below handle every remaining case, repointed included, so there is one
place that knows what a complete install looks like. That also makes result=unchanged mean
"complete", which is what the skill reads it as.
Worth a test on each: add --upstream over an already-routed file must land the key, and the same
over a port change must carry it across. The two new tests only cover the fresh-file added path,
which is why both of these got through.
| # covered by the same permission rule as the rest of the command, and needs no symlink. | ||
| BIN_ARG="" | ||
| while [ $# -gt 0 ]; do | ||
| case "$1" in |
There was a problem hiding this comment.
Fix before merge — the argument loop has no *) branch, so a mistyped or valueless flag is
discarded in silence, and --upstream will swallow the next flag as its value.
You flagged this file as the weakest coverage; here is what running the combinations produces. In each
case the proxy is launched and reports success:
--upstream http://gw:4000 -> proxy got: --anthropic-upstream http://gw:4000 (correct)
--upsteam http://gw:4000 TYPO -> proxy got: (none)
--upstream (no value) -> proxy got: (none)
--upstream --bin /some/path -> proxy got: --anthropic-upstream --bin
Nothing printed a warning in any of the three broken cases.
Rows 2 and 3 mean a proxy that forwards straight to api.anthropic.com on a platform whose
gateway holds the credential and rewrites model names — the exact configuration you found makes every
request fail, reached by a single transposed letter with no diagnostic. Row 4 is worse: --bin's value
is consumed as the upstream URL and --bin is lost, so the script both misconfigures the upstream
and falls back to resolving the binary by name, which is the failure --bin was added to fix. Two
defects from one typo.
This is also the one place where property 5 ("never fails the session") cuts the wrong way if applied
too literally: the fix is not to exit non-zero, it is to say something. There is already a log
breadcrumb for the declined-gate case, on exactly the reasoning that silence is indistinguishable from
"the script never ran" — the same argument applies here:
--upstream|--bin)
case "${2:-}" in
''|-*) note "ignoring $1: expected a value, got '${2:-}'" ;;
*) [ "$1" = --bin ] && BIN_ARG="$2" || UPSTREAM_ARG="$2"; shift ;;
esac ;;
*) note "ignoring unrecognised argument '$1'" ;;Keep every exit at 0; just make the discard visible. A note reaches the installing session, where
the skill and the user can act on it, which is where these arguments come from.
Given the coverage gap, a small table-driven test over these four rows against a sentinel binary that
records its own argv would pin all of it — the harness in TestHookMakesTheProxyIdentifiable already
does exactly that capture.
| rule naming this script can cover an env-prefixed invocation. `CONTEXT_GURU_FORCE=1` asks for what it | ||
| means and leaves the routing variable alone. | ||
|
|
||
| If step 3 found a gateway to chain behind, add `ANTHROPIC_UPSTREAM=<their gateway>` — or better, tell |
There was a problem hiding this comment.
Fix before merge — step 4 contradicts itself, and the losing half is the shape that got denied on a
hosted agent.
Line 190, in this same step:
Everything goes in as arguments, and nothing as an environment prefix. Two reasons, both learned
the hard way on a hosted agent…
Then lines 236-240:
CONTEXT_GURU_FORCE=1asks for what it means and leaves the routing variable alone.If step 3 found a gateway to chain behind, add
ANTHROPIC_UPSTREAM=<their gateway>— or better,
tell the user to set the plugin's Upstream base URL option, whichstart-proxy.shreads by
itself…
Three problems, in descending order:
ANTHROPIC_UPSTREAM=<their gateway>is an environment prefix — forbidden fifty lines above,
ungrantable under prefix-matched Bash rules, and superseded by the--upstreamargument this very
commit added. An agent reading top-to-bottom hits the prohibition first and the instruction second,
and the instruction is the actionable one.CONTEXT_GURU_FORCE=1is the superseded form of--unrouted. Still accepted for
compatibility, per the script's comment — but recommending it here reintroduces the ungrantable
invocation, and the paragraph presents it as the fix for the classifier denial when the actual fix
was making it an argument.- "which
start-proxy.shreads by itself" is only true in the hook path. Finding 4 of this round
is thatCLAUDE_PLUGIN_OPTION_*does not reach a Bash tool call — and step 4 is the skill running
the script as a Bash tool call. So in this step the option is precisely what the script cannot
read, and calling it "better" points the reader at the thing that just failed on a hosted agent.
Deleting the paragraph is most of the fix, since --unrouted --upstream --bin twenty lines earlier
already says all of it correctly. If the plugin option is worth mentioning, it belongs in step 6 as
"this makes chaining apply in later sessions automatically, because hooks do see plugin options" —
framed as the hook-path benefit it is, and paired with the --upstream write which is what makes it
work in the session you are in.
This is the highest-value thing in this review relative to effort: three of your four DAM failures were
consent-gate denials caused by this exact shape, and the file an agent executes still recommends it.
| save(args.file, data) | ||
| emit(result="added", file=args.file, base_url=args.url, | ||
| replaced=current if current else "", backup=saved or "(new file)", | ||
| other_env_keys=len([k for k in env if k != KEY])) |
There was a problem hiding this comment.
Low, but it undercuts the skill's contract. Two reporting gaps now that add can write three keys:
1. Neither new key is reported. result=added names base_url, replaced and backup, so a
skill that (correctly) acts on key=value output rather than re-reading the file cannot confirm the
upstream or the bin path landed. Given the whole reason --upstream exists is that a missing key
fails silently in a later session, "did it get written" is exactly the fact worth emitting:
upstream=args.upstream or "", bin=args.bin or "",2. other_env_keys now counts our own keys as the user's. It is
len([k for k in env if k != KEY]), and env includes the two keys we just added. Measured on a file
whose only pre-existing variable was THEIR_OWN:
other_env_keys=3 # actual user-owned keys: 1
cmd_show has the same expression, so /context-guru:status misreports it too. The count is there to
tell the skill how much of the user's own configuration is in play, and it now inflates by up to two on
every chained install. Excluding all three of our keys fixes both:
OURS = {KEY, UPSTREAM_KEY, BIN_KEY}
... len([k for k in env if k not in OURS])| purpose — exiting clears in-memory cache state. If they want a shorter one, that is a | ||
| `store.ttl_seconds` conversation, not a flag to force. | ||
|
|
||
| ### 6. Tell them what happens next |
There was a problem hiding this comment.
Nit — two steps are both numbered ### 6. (this one and "Prove it, and only then say it worked" at
285), so the skill runs 1, 2, 3, 4, 5, 6, 6.
Trivial to fix and worth fixing anyway: this is a document whose reader executes it in order and
reports which step it is on, and a duplicate number makes "I finished step 6" ambiguous between
"verified it works" and "told the user what happens next" — which are not interchangeable, since one
is a gate and the other is a wrap-up.
…ts are reported Second review of 4cb2817, and all three blocking findings landed in the two files I had flagged as thinnest and highest blast radius. Reproduced each before fixing. ## Re-running the install was a no-op that reported success `unchanged` meant 'the base URL matches', and cmd_add's two early returns wrote nothing else. So: add --url <same> --upstream http://gw:4000 --bin /opt/cg/proxy -> result=unchanged, exit 0, env keys after: ['ANTHROPIC_BASE_URL'] There was no way to add the upstream to an already-routed project through this script at all — and re-running the install is exactly what somebody does after an attempt dies at the last step, which is how every one of the four hosted-agent attempts ended. The repair silently did nothing and said it worked. The repointed path had the same hole, so changing the configured port un-chained the proxy. cmd_add now computes the COMPLETE desired env once and judges every exit against the whole set: * `unchanged` means complete — routed AND every other key already correct; * `completed` is new: routed already, keys filled in, `added_keys=` naming which; * `repointed` carries the chaining keys across a port change. ## Three malformed invocations launched a proxy and reported success The argument loop had no default branch and took `$2` on faith: --upsteam <url> one transposed letter -> no upstream at all --upstream missing value -> no upstream at all --upstream --bin <p> swallowed the flag -> upstream='--bin', and --bin lost as well Rows 1-2 leave a proxy forwarding to api.anthropic.com, which on a platform whose gateway rewrites model names makes every request fail. Exiting non-zero is not the fix — property 5 is that this never fails a session — so it reports what it discarded, the same argument as the declined-gate breadcrumb. My first version of that fix had a bug of its own, caught by running it: the `shift` sat outside the accepted branch, so rejecting a value still consumed the NEXT FLAG — `--upstream --bin /path` lost --bin too and reported '/some/path' as unrecognised. The shift belongs inside the accepted branch. ## The skill contradicted itself, and the losing half was the denied shape Step 4 said 'everything goes in as arguments, and nothing as an environment prefix', then fifty lines later told the agent to add `ANTHROPIC_UPSTREAM=<gateway>` and recommended `CONTEXT_GURU_FORCE=1`. An agent reads the prohibition first and the actionable instruction second — and the instruction was the exact shape that had been denied twice. Deleted, in step 3 as well as step 4. Also corrected there: 'which start-proxy.sh reads by itself' was true only of the hook path. My own earlier finding is that CLAUDE_PLUGIN_OPTION_* does not reach a Bash tool call, and that step IS a Bash tool call, so in that step the configured option is precisely what the script cannot read. ## And the smaller ones `other_env_keys` counted our own keys as the user's — 3 reported where the user owned 1, and /context-guru:status repeated the wrong number back. Now excludes all three keys this script owns. `add` reports the upstream and bin values so the skill can confirm they landed. Two steps were both numbered 6. ## Verification Nine new subtests over the two behaviours that had none: the re-run repair, `unchanged` only when complete, a port change keeping the chaining keys, the env-key count, and all five argument shapes (good, typo, missing value, swallowed flag, nonsense) asserting both what is SAID and what reaches argv. `go test -race ./...` and `gofmt -l` clean. The reviewer also re-verified the not-platform-specific claim three ways of their own rather than repeating mine, and it holds. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
amiddavid
left a comment
There was a problem hiding this comment.
Review of 762442f — all three fixed, verified by running them; two low findings left
I re-ran my own measurements against the new code and added a regression matrix, because cmd_add was
restructured and that is the highest-blast-radius function in the plugin.
The three fixes, confirmed
1. settings.py — the complete-desired-env approach works, and nothing regressed.
| Case | result | env keys after |
|---|---|---|
re-run with --upstream --bin (the repair case) |
completed, added_keys=ANTHROPIC_UPSTREAM,CONTEXT_GURU_BIN |
all three |
| re-run with nothing missing | unchanged |
unchanged |
port change with --upstream |
repointed |
chaining carried across |
add over a stranger's gateway, no --force |
conflict, exit 2 |
untouched |
remove with no --url over a stranger's |
conflict, exit 2 |
untouched |
| add where user owns one key | added, other_env_keys=1 |
correct |
The last two matter most to me: those were round-one's blocking finding and the round-three fix, and
restructuring the function around desired did not disturb either. changed is computed before
env.update(desired), so added_keys reports what was actually missing rather than the whole set.
2. start-proxy.sh — all four space-separated shapes now report what they discarded, and the
shift-inside-the-branch fix is correct: --upstream --bin2 /x no longer swallows the following flag.
3. install/SKILL.md — the contradiction is gone, and so is the step-3 instance I had not spotted.
Grepping the whole skills tree for CONTEXT_GURU_FORCE=1, ANTHROPIC_UPSTREAM=<, "reads by itself",
and any env-prefixed script invocation returns nothing. Steps renumbered 1-7. Deleting rather than
qualifying was the right call.
On your shift bug
Worth saying plainly: you found that by running the fix rather than reading it, which is the whole
lesson of the last four rounds applied to your own work in the same session. That is the harder
direction to apply it in.
It also leads to the second finding below, which is the same class one level up — so the discipline is
working but has not yet reached the tests that certify it.
| {"good", []string{"--upstream", "http://gw:4000"}, "", "http://gw:4000"}, | ||
| {"typo", []string{"--upsteam", "http://gw:4000"}, "unrecognised argument '--upsteam'", ""}, | ||
| {"no value", []string{"--upstream"}, "needs a value", ""}, | ||
| {"swallowed flag", []string{"--upstream", "--bin", "/nope/x"}, "needs a value", ""}, |
There was a problem hiding this comment.
The row named for your shift bug passes against that bug. This is the vacuous-evidence class
again, and I would not have found it without your own description of what the buggy parser did.
The assertion is that stderr contains "needs a value". I reconstructed the pre-fix parser (shift
outside the accepted branch) and ran this exact row against it:
$ buggy.sh --upstream --bin /nope/x
ignoring --upstream: it needs a value, and got '--bin'
ignoring unrecognised argument '/nope/x'
-> BIN_ARG='' UPSTREAM_ARG=''
does the buggy output contain "needs a value"? YES
So the buggy parser satisfies both of this row's assertions — wantSaid matches, and wantUp is
correctly empty. The row is green either way, which means the defect it was added for is still
uncovered.
What actually separates the two versions is what happens to --bin:
- buggy:
--binis consumed as a rejected value,BIN_ARGstays empty, and/nope/xis reported as
an unrecognised argument. - fixed:
--upstreamis rejected alone, then--bin /nope/xparses normally andBIN_ARG=/nope/x.
Either of those is a cheap discriminator. The positive one is stronger, and it fits the rule you said
you are adopting — assert that the machinery worked, not only that it complained:
{"swallowed flag", []string{"--upstream", "--bin", "/nope/x"},
"needs a value", "", /* wantBinHonoured: */ true},asserting the run reports the binary /nope/x as missing (which only happens if --bin was honoured),
or negatively that the output does not contain unrecognised argument '/nope/x'.
The four other rows in this table are sound — good asserts a positive --anthropic-upstream value in
argv, which is exactly the control that makes the rest mean something.
| # consume the next FLAG too, so `--upstream --bin /path` lost --bin as well and reported | ||
| # '/some/path' as an unrecognised argument. Reject the value, keep the flag that followed it. | ||
| --upstream) if takes_value --upstream "${2:-}"; then UPSTREAM_ARG="$2"; shift; fi ;; | ||
| --upstream=*) UPSTREAM_ARG="${1#--upstream=}" ;; |
There was a problem hiding this comment.
Low, and the last hole in the parsing: the = forms bypass takes_value, so an empty value is
accepted in silence while the space-separated form reports it.
--upstream http://gw:4000 -> --anthropic-upstream http://gw:4000
--upstream -> SAID: ignoring --upstream: it needs a value…
--upstream= -> (none) SAID: (nothing)
--bin= behaves the same way — BIN_ARG becomes empty and the script falls back to resolving the
binary by name, silently, which is the failure --bin exists to prevent.
The realistic trigger is not a user typing a bare --upstream=. It is the skill interpolating a value
that turns out to be empty: --upstream="$GATEWAY" with GATEWAY unset expands to exactly
--upstream=, and step 4 builds these command lines from values discovered in step 3. So this is the
one shape most likely to arrive from the install path itself, and it is the one the new validation does
not see.
One line each:
--upstream=*) takes_value --upstream "${1#--upstream=}" && UPSTREAM_ARG="${1#--upstream=}" ;;
--bin=*) takes_value --bin "${1#--bin=}" && BIN_ARG="${1#--bin=}" ;;takes_value also rejects only --*, so a single-dash -x would be accepted as a value; -* would
be more thorough, though I cannot construct a realistic caller that produces it.
Neither = form appears in TestStartProxyReportsArgumentsItCannotUse's table, which is why this
survived — two more rows cover it.
Two findings from the third review of 762442f. The first is the same class as the three I hit this week, one level up: **the test row I wrote to cover my own `shift` bug passed against that bug.** The reviewer reconstructed the pre-fix parser and ran the row: $ buggy.sh --upstream --bin /nope/x ignoring --upstream: it needs a value, and got '--bin' ignoring unrecognised argument '/nope/x' -> BIN_ARG='' UPSTREAM_ARG='' contains "needs a value"? YES Both assertions in that row — the message appears, no upstream reaches argv — are satisfied by the BUGGY parser. Green either way, on the one defect it existed to pin. And it only surfaced because I had described in the commit body what the buggy parser did, which is an argument for writing those down. What separates the versions is `--bin`: buggy eats it as the rejected value (falling back to resolving the binary by name, which fails on exactly the machines --bin exists for), fixed rejects `--upstream` alone and parses `--bin <path>` normally. So the row is now its own test asserting a POSITIVE — the proxy was launched via the binary --bin named — plus, negatively, that the path was not reported as a stray argument. Proven rather than assumed, since a test written for a specific defect and never watched fail against it is a test on trust: MUTATION LANDED (shift moved outside the accepted branch): True with the bug -> FAILED (good, the test discriminates) --- FAIL: TestRejectingAValueDoesNotEatTheNextFlag --bin was eaten as --upstream's rejected value … nothing started the path after --bin was treated as a stray argument … restored byte-identical: True with the fix -> passed ## The = forms bypassed the value check `--upstream=` was accepted silently where `--upstream ` was reported, and `--bin=` fell back to name resolution without a word. The realistic trigger is not somebody typing a bare `--upstream=` — it is a caller interpolating one: `--upstream="$GATEWAY"` with GATEWAY unset expands to exactly that, and the install skill builds these command lines from what it discovered a step earlier. Both forms go through takes_value now, and the empty case is in the table. Neither form was in that table, which is why this survived — the same gap that produced the finding above. `go test -race ./...` and `gofmt -l` clean. CI was green on 762442f before these two changes. Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
Review of 4b39f79 — both findings fixed, no findings leftThe extracted test is right, and it asserts the discriminator that matters. Moving that case out of What makes it sound, reading it:
I could not run That is the separation the test keys on, so I am confident in it without having executed it. Your Both One note from my own side, since it cuts the other wayMy first pass at verifying this reported So the hermeticity question I went looking for was already answered in the code. Recording it because a StateAll findings from five rounds are resolved. The merge gate remains |
First real end-to-end run: install, two live preset sessions, and one blockerThe PR body is honest that this has never run end to end. I ran it — real install, real Read + ran the existing testsAll four scripts, three Revert-verification (mutated the SOURCE, reverted with
Real numbers, both presetsProxy built from this worktree (
Turn 2's
Aside confirmed along the way: Blocker: the configured proxy port can never be honoured by
|
…rs report as data
Osher's end-to-end review — real install, live `claude -p` sessions through a real proxy, revert-
verification of four existing tests, teardown by pidfile. It found one blocker, one major, and a test
of mine that could not see its own subject.
## Blocker: the configured port could never be honoured
I had documented that CLAUDE_PLUGIN_OPTION_* does not reach a Bash tool call — and documented it only
for UPSTREAM, then left PORT reading exactly that variable. So:
* `${CLAUDE_PLUGIN_OPTION_PORT:-8787}` in a skill always expanded to 8787;
* `start-proxy.sh` had no argument form for the port, so nothing could override it;
* the one workaround left, an env prefix, is ungrantable by this plugin's own stated rule.
The consequence is the failure this plugin exists to prevent. With a configured port of 4041 the
routing key named 8787, while every later SessionStart/UserPromptSubmit hook read the CONFIGURED port
and self-gated on it — so the hooks saw an unrouted project and did nothing, the running proxy had no
auto-restart behind it, and once it idle-exited nothing brought it back. Silently. An unmodified install
session found and reported this itself, unprompted, which is the part I find most damning.
Fixed at both ends:
* `start-proxy.sh --port <n>`, parallel to --upstream/--bin, with LOG, HEALTH and the pidfile all
re-derived from it — a port that reached --listen but not the pidfile would leave a proxy uninstall
could not find;
* `settings.py config`, which READS the values Claude Code stores at
`pluginConfigs["<plugin>"].options` in a settings file, checked in precedence order. The values
were on disk the whole time.
Then the 10 remaining `${CLAUDE_PLUGIN_OPTION_*:-default}` expansions across all three skills were
replaced with placeholders pointing at the discovery step, because every one of them was a silent wrong
answer: 1 in status (reporting on the wrong port), 2 in uninstall (building a URL that matches nothing,
so the removal finds nothing to remove), 7 in install.
## Major: settings.py answered an OS error with a traceback
The docstring promises one key=value line per fact and the skill is told to read those rather than
guess. Every failure path honoured it except genuine OS errors in backup()/save() — an unwritable
directory produced a raw Python traceback and no `result=` line at all, so the caller had nothing to act
on exactly when something was already wrong. Non-destructive in every case observed, but that is not the
same as legible. `result=error reason=os_error detail=…`, exit 4.
## My mode test could not see its own subject
`TestSettingsPreservesFileMode` chmodded the fixture to 0600 — which is precisely what
`tempfile.mkstemp()` creates on its own, so deleting the copymode/chmod block entirely left the test
green. Osher demonstrated it: mutation applied, test passed, and a real 0640 file silently narrowed to
0600. The fixture is 0640 now, which is neither mkstemp's default nor umask-derived, so passing requires
copymode to have run.
That is the fourth vacuous test found in this PR, and the second found by someone else rather than by me.
## Verification
Four mutations, each proven to land, each failing the named test, each file restored byte-identical:
drop the OSError handler -> TestSettingsReportsOSErrorsAsData FAILED
remove the copymode/chmod block -> TestSettingsPreservesFileMode FAILED (green before)
remove the --port argument -> TestTheConfiguredPortCanActuallyBeHonoured FAILED
make config discovery return {} -> TestTheConfiguredPortCanActuallyBeHonoured FAILED
One thing the sweep caught that I would otherwise have shipped: introducing `PORT="<port>"` into
uninstall/SKILL.md broke TestUninstallDoesNotSignalAProcessThatIsNotOurs, which EXECUTES that fenced
block — it was running a template with an unsubstituted placeholder, so every path looked inert for the
wrong reason. The test now fills the placeholder the way the skill instructs the model to, and fails
loudly if it disappears.
`go test -race ./...` and `gofmt -l` clean.
Assisted-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: DAVID AMID <DAVIDA@il.ibm.com>
The Claude Code plugin:
/plugin marketplace add rossoctl/context-guru→/plugin install→/context-guru:install.#141 has merged, so this is no longer stacked — it is two commits on
main, 17 files, and the diff is the plugin and its docs. Three skills over four scripts and two hooks.Default routing scope is
.claude/settings.local.json: one repo, gitignored,--globalan explicit opt-in. A base URL pointing atlocalhostbreaks Claude Code wherever it is routed when the proxy is down, so blast radius is the thing the default optimises for.Read this first: it cannot work yet, and that is not a defect in the diff
install.shresolves a GitHub release, and no tag has published assets. Onmaintoday the third command ends atno_release_found.So the merge order matters more than the code review: run
release.yamlviaworkflow_dispatch(snapshot, publishes nothing — it has never executed, and #141 landedwrap_in_directory: truewhose archive layout only a real goreleaser run confirms), cutv0.1.0, then merge this. Landing it first puts a README onmaintelling strangers to run three commands, the last of which fails.The plugin has therefore never run end to end in a real session. Everything below was verified by unit tests, by driving the scripts directly, and by reading — not by installing it.
What the review of the combined PR found here, and what changed
All six blocking findings were in this half:
uninstallkilled the user's own session and left the proxy runningargvvia--listen; no pattern matchinginstall.shcould not install, and its documentedgo installfallback was absentdownload_failedas a reported outcome and akey=valuecontract curl no longer breaksstatuscannot diagnose itUserPromptSubmithook — the only thing that runs without a model callcacheadvertisedcontext_guru_expandO_EXCL+ microsecond stamps; the replaced value is recorded and restoredOn finding 1 the mechanism is worth stating, because the fix follows from it: the port was passed through
LISTEN_ADDRin the environment, sopkill -f "context-guru-proxy.*$PORT"matched no proxy — and did match the shell running it, which is the session's own Bash tool. The narrow pattern the skill offered as the safe option was the one that bit. So the fix is a handle, not a better pattern.On
is_oursI did not take the review's suggestion. A URL-shape rule ((127.0.0.1|localhost|[::1]):\d+/anthropic) makes litellm's default read as ours, so uninstall would delete somebody else's routing — the existing test failed the moment I tried it.addnow records the URL it wrote and later runs read that record; anything unrecorded stays a conflict, for add and remove alike.One change since that review
install.shfinds the binary in the tarball rather than assuming the archive root. #141 setwrap_in_directory: true— deliberately, because a flat archive plus the documentedtar xzfwith no-Coverwrites theREADME.mdandLICENSEof whatever directory the user is standing in, which for an evaluator is their own project. Without this change the first install anyone attempted would have failed withbinary_not_in_tarball, reading as a broken release rather than a moved file. Found by checking #141's packaging change against this script, not by waiting for it to fail.Verification
The shell and Python helpers are tested from Go (
context-guru-plugin/plugin_test.go) sogo test ./...and CI cover them. Seven mutations, each proven to have landed in the source before its result counted, each failing its named test and passing when restored — full output in the commit body. Coverage includes: settings merge/conflict/removal/backup/mode/symlink behaviour, the hook's silence in unrouted projects, its idempotence, its non-failure when the binary is missing, and its wait for/healthz.One process note kept in the open: my first attempt at the backup mutation reverted only the timestamp granularity and left the
O_EXCLloop, so the filename was still unique and the test passed — proving nothing. Reverting half a fix is its own route to a vacuous result.Worth a reviewer's attention
SessionStarthook runs in every project the user has, because the plugin installs at user scope. It self-gates on$ANTHROPIC_BASE_URLnaming its own port — matching the port, not "localhost", so somebody routing to litellm on 4000 is not hijacked. It is synchronous (closes the race with the session's first request), idempotent (SessionStartalso fires on clear/compact/resume/fork), and exits 0 on every path: a hook that fails here is a plugin that can brick every session on the machine.settings.pyedits the user's realsettings.json. It merges exactly one key, backs the file up first (and prunes to 10), refuses to overwrite a base URL it did not write, restores what it replaced on removal, preserves mode, and follows symlinks. It refuses rather than rewrites a file it cannot parse.statuschecks the commonest one at runtime: outside a git repository there is no environment snapshot, socachesplitskips and the saving is exactly zero.statusalso no longer readsacted: 0/savings_pct: 0as a verdict — those count content removal, and this component relocates a breakpoint.🤖 Generated with Claude Code