Skip to content

fix(server): a bad publisher.json answers 424, and a hidden source says why it was refused - #1228

Merged
jswir merged 7 commits into
mainfrom
jswir/manifest-4xx-and-curation-404
Sep 24, 2026
Merged

jswir merged 7 commits into
mainfrom
jswir/manifest-4xx-and-curation-404

Conversation

@jswir

@jswir jswir commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Two bug fixes, one commit each. Both came out of testing the modeling workflow against a local Publisher after #1206, and a service-side session hit both independently.

1. A bad publisher.json answered 503

A malformed explores, an unknown scope, or two scope values that disagree answered 503 "Package-load worker pool unavailable". That's the author's mistake, but callers logged it as a server outage.

  • Cause: the manifest is read inside the package-load worker. The three throws were plain Errors. On the way back, the pool rewrapped everything except compile errors as a worker outage.
  • Fix: they now throw PackageManifestError. It sets name, which the pool already sends across, so the class survives and Package lets it through at both worker call sites.
  • Status: 424, the same status a package that fails to compile already gets on reload. The request was fine; the package it depends on is not.

2. A hidden source's 404 read like a typo

In a package curated by index.malloy, a modeler who saves users.malloy and queries it gets No queryable model "users.malloy". Over MCP that became Resource not found ... verify the identifier is spelled correctly. The name is real.

The 404 was worded like a missing name so that a hidden gated source can't be told from one that doesn't exist. An ungated hidden source has nothing to protect that way. Curation isn't access control, and /compile (exempt from the boundary) already answers a hidden file differently from a missing one.

Now, when the target is real and the refusing model has no #(authorize) anywhere, the 404 says why:

No queryable model "users.malloy". It is not on this package's published surface,
"index.malloy": only what that file exports is queryable, and only through it.
Fix: import it in "index.malloy", add it to that file's export { ... }, and address
the query to "index.malloy".
  • Still a 404: this is OffSurfaceError, a subclass of NotQueryableError.
  • Unchanged: a name that doesn't exist, and every refusal in a model with any gate, keep the plain message. The gate check uses hasAnyAuthorizeNote(), which answers true for unreadable IR, so any doubt gets the plain message.
  • Source and view both named: the refusal now names the source, since that's what is off the surface.
  • MCP: execute_query passes the message through with "this is curation, not a typo", instead of the spelling advice.
  • Docs and skills: docs/discovery-and-access.md, malloy-publish and malloy-source-unreachable are updated. The MCP skills bundle is regenerated.

Behaviour change: two tests pinned the identical 404 for packages with no gates (query_boundary.spec.ts, surface_widening.integration.spec.ts). They now pin the new contract. A new test pins that a gated model still refuses a hidden name and a missing one with identical text.

How this was checked

  • Bug 1: a new test loads all three manifest errors through the real worker pool and asserts 424 with the author's message. With the pool change reverted, it fails with the exact 503 text.
  • Live server: a local build served a scaffolded package built from the malloy-samples ecommerce data. The hidden file's 404 explained itself over REST and MCP. A missing file stayed plain. A malformed explores answered 424 on reload.
  • Unit suite: 4190 pass. 35 config.spec failures are the known local bun 1.4 issue and also fail on main.
  • Integration: 347 pass, run with MCP_PORT=4044 because 4040 was taken locally. compile_authorize_http (3) and concurrent_package (1) fail the same way on unmodified main. One MCP semantic spec couldn't bind port 4050, which another local process held.

Not in this PR

  • Bug 1's own message still suggests "explores": ["index.malloy"], a deprecated key. Worth rewording separately.
  • Modelers still can't query a new file without editing index.malloy. The fix for that, a server-level draft mode, is a separate design.

🤖 Generated with Claude Code

jswir and others added 3 commits September 23, 2026 16:29
A malformed "explores", an unknown "scope", or two "scope" homes that
disagree are the author's mistake, but they answered 503 "Package-load
worker pool unavailable". The manifest is read inside the package-load
worker, the errors were plain Errors, and on the way back the pool
rewrapped everything except compile errors as a worker outage. Callers
logged an author's typo as a server fault.

The three throws now raise PackageManifestError. It sets `name`, which
the pool's wire shape already carries, so deserializeError restores the
class and Package lets it through at both worker call sites. It maps to
424, the status a package that fails to compile already gets on reload:
the request was fine, the package it depends on is not.

Signed-off-by: James Swirhun <james@credibledata.com>
A modeler who saves a new file in a package curated by its index.malloy
and queries it gets `No queryable model "users.malloy".` Over MCP that
became "Resource not found ... verify the identifier is spelled
correctly". Both read as a typo, for a name that is real.

The 404 was worded like a missing name so a hidden GATED source cannot
be told from one that does not exist. An ungated hidden source has
nothing to protect that way: curation is not access control, and
/compile, exempt from the boundary, already answers a hidden file
differently from a missing one.

So a refusal now explains itself (OffSurfaceError, still a 404) when the
target is real and the refusing model has no #(authorize) anywhere,
using hasAnyAuthorizeNote, which answers true for unreadable IR. The
message names the surface and the fix, worded by the Package for an
index.malloy surface or an explores list. A name that does not exist,
and every refusal in a gated model, keep the plain message. A refusal
naming both a source and a view now names the source, since that is
what is off the surface. The MCP error classifier passes the message
through with "this is curation, not a typo" instead of the spelling
advice.

Two tests pinned the identical 404 for ungated packages and now pin the
new contract; a new test pins that a gated model still refuses a hidden
name and a missing one identically. Docs, the malloy-publish and
malloy-source-unreachable skills, and the MCP skills bundle are updated.

Signed-off-by: James Swirhun <james@credibledata.com>
…and make the off-surface fix text correct

Review findings on the two previous commits.

A bad publisher.json still answered 503 in three cases:
- compile with scope "package" rewrapped every worker rejection as a
  worker outage. It now lets PackageManifestError through, as load and
  reload do.
- a JSON syntax error threw a bare SyntaxError inside the worker.
- valid JSON that is not an object (null, an array) crashed on the next
  property read.
The last two now throw PackageManifestError naming the file.

PackageManifestError crosses the worker boundary by an isManifestError
flag set with instanceof, like compile errors, instead of matching a
hand-copied name string.

The off-surface "Fix:" sentence told a modeler to import a source that
was already declared in index.malloy, and treated a file as a name for
export {}. The Package now passes the surface to each model, and the
model words the fix by what was refused: a whole file, a source this
file can already see, or a name queried through a notebook.

Also:
- declare 424 (new PackageManifestError response) on create-package,
  get-package and compile-model-source
- count a bad manifest as policy_rejected in the package-load metric
- say "#(authorize) or #(access_filter)" wherever the gate check is
  described; the code already treats both as gates
- fix comments, docs and the scaffolder template that still said a
  hidden source always reads the same as a missing one
- pin the gated named-source and compiled-backstop refusals, every fix
  sentence, and the explores-list wording in tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: James Swirhun <james@credibledata.com>

@Sha-Bang Sha-Bang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 should-fix, 1 nit.

  • packages/server/src/service/environment.ts:839: compile_model at scope: "package" still turns a bad publisher.json into a 503.
  • packages/server/src/mcp/handler_utils.ts:67: PackageManifestError has no MCP branch, so reload_package calls it an unexpected internal error and says to retry.
  • packages/server/src/service/query_boundary.spec.ts:1030: the gated-model test only pins the ad-hoc refusal. The named-source and hidden-file refusals in a gated model are not pinned.
  • packages/server/src/service/package_worker_path.spec.ts:694: the new it.each separated the "kept last" note from the test it describes.

environment.ts:839 (outside this diff, so not inline): this is a third worker call site. The package-scope compile calls getPackageLoadPool().loadPackage directly, and this catch rewraps every rejection as ServiceUnavailableError. Edit publisher.json to a bad explores and run compile_model with scope: "package", and you still get 503 "Package compile worker unavailable: Invalid "explores" ...", the outage-shaped answer this PR removes elsewhere. The new comment at errors.ts:20-22 says a bad manifest no longer lands in that branch, and on this path it still does. Could this catch let PackageManifestError through (and probably ModelCompilationError/MalloyError too), the way Package.loadViaWorker now does?

Comment thread packages/server/src/mcp/handler_utils.ts
Comment thread packages/server/src/service/query_boundary.spec.ts Outdated
Comment thread packages/server/src/service/package_worker_path.spec.ts Outdated
Review findings on #1228.

- classifyToolError had no branch for PackageManifestError, so
  reload_package and compile_model called a bad publisher.json an
  unexpected internal error and told the agent to try again later. It now
  passes the message through and says to fix the file and reload.
- A package-scope compile now lets compile errors through the worker
  catch too, not only PackageManifestError, the same split as load and
  reload.
- The gated-model test now pins the whole-file refusal: a hidden file
  with its own gate gets the plain "No queryable model". The gate check
  is per model, so the file needs its own gate for this to be the case.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: James Swirhun <james@credibledata.com>
@jswir

jswir commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator Author

On environment.ts:839: eb35be9 lets PackageManifestError through the package-scope compile catch, pinned in compile_scopes.spec.ts. 8cbd5ec also lets MalloyError and ModelCompilationError through, so all three worker call sites now split errors the same way.

jswir and others added 3 commits September 24, 2026 12:57
…ex.malloy comment

The comment added in eb35be9 spelled out the authorize annotation. The
server refuses caller-submitted Malloy that contains one, even in a
comment, so compiling the generated index.malloy answered 400 and the
scaffolder e2e test failed. The comment now names the gates in words.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: James Swirhun <james@credibledata.com>
This PR changes published skills and scaffolder template content, and
both current versions are already on npm.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: James Swirhun <james@credibledata.com>
@jswir
jswir merged commit 11066be into main Sep 24, 2026
23 checks passed
@jswir
jswir deleted the jswir/manifest-4xx-and-curation-404 branch September 24, 2026 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants