Skip to content

FEE-890 Create a generic upsert route that accepts all root CrossApp types (2/2) - #1622

Open
sid597 wants to merge 12 commits into
fee-890-crossapp-upsert-stepsfrom
fee-890-create-a-generic-upsert-route-that-accepts-all-root
Open

sid597 wants to merge 12 commits into
fee-890-crossapp-upsert-stepsfrom
fee-890-create-a-generic-upsert-route-that-accepts-all-root

Conversation

@sid597

@sid597 sid597 commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

Reviewer brief

FEE-890 adds one route that takes the five root CrossApp types for a space and writes them with one upsert_concepts call. It ships as two PRs, stacked on #1619 (ENG-2005, the validators):

  1. FEE-890 Create a generic upsert route that accepts all root CrossApp types (1/2) #1621. packages/database only: the request type and its validator, and the pure steps the route runs.
  2. This PR. The route in apps/website, its unit tests and the bearer-token client.

Why two and not one: together they are 527 changed lines excluding tests (386 in #1621, 141 here). The split follows the package boundary: shared code first, then the route that uses it.

  • Result: POST /api/internal/space/{id}/graph checks the caller, checks the body, converts it and makes one upsert_concepts call. It answers { upserted: { localId: dbId }, failed: [{ localId, kind, reason }] }. Nothing calls the route yet. It supersedes the July draft ENG-2039 Create a generic upsert route #1237.
  • Review focus: These are defaults. Each can change in a later PR and lives in one place.
    • Bearer-token auth. The caller sends its plugin's Supabase access token. createBearerTokenClient builds a client per request with that token, so RLS applies as the caller. No cookies, no service key.
    • The path. /graph follows Michael on #1237 ("or just /graph") and Marc-Antoine on FEE-890 (2026-10-23): space/[id] in the path, no upsert segment.
    • Inline embedding vectors pass through as the contract allows. The route computes none.
    • Row failures come back per local id with HTTP 200: unique_conflict for -1 (any unique violation, such as a name or a Content clash), error for -2.
  • Review focus: The route checks that the caller is an editor of the space before it reads the body, and answers 403 otherwise. Without that check, a non-editor gets HTTP 200 with every row failed, because upsert_concepts catches the RLS error per row.
  • Review focus: Status codes. 400 for a bad space id or body (including a date Postgres can't store, a RID node or relation type, and schema references the body does not define whose lookup query passes 4,000 characters as postgrest-js encodes it; the local gateway refuses a URL past about 8,200 characters), 401 for a missing or refused token, 502 when the auth service can't be reached, rate-limits the check or fails with a server error, 403 for a non-editor, 422 when a schema referenced by local id is in neither the body nor the space (a RID in a slot definition or triple end is not looked up, and an unknown one fails its row with error; a triple's relation written as a RID gets 422 even when it names a stored type), 500 for a database failure. Nothing is written in any of those cases. Database failures and the 502 are reported through internalError.
  • Review focus: Michael asked on #1237 what a user can do with the error. The route names each failed item by local id and kind and says whether it hit a unique conflict. The action function that calls the route turns that into what the user sees. No caller exists yet, so that message is written with the first caller.
  • Risk or follow-up: Request size. Vercel's request limit applies before the handler runs, so callers keep sending batches, as Roam sync does. The route bounds only the schema lookup's URL length.
  • Risk or follow-up: Not in this PR: moving Roam publish, Roam sync and Obsidian sync onto the route (no tickets yet), and the end-to-end test and its CI job (FEE-891).
  • Risk or follow-up: An unexpected exception goes through the existing handleRouteError, which answers 400 and logs to the console only.
  • Risk or follow-up: A string holding U+0000 or a lone UTF-16 surrogate passes the validators. Postgres refuses it in jsonb, so one such item fails the whole upsert_concepts call. Roam and Obsidian already call upsert_concepts directly with the same exposure. The route then answers 500 and reports it.
  • Risk or follow-up: The existing CORS helper matches allowed origins by prefix. No ticket yet.
  • Risk or follow-up: Items that reference each other, or an item whose slot names its own local id, can't be ordered. When neither row exists yet, both come back in failed with error and nothing is stored for them; when one already exists, both are written (read from the SQL, not run). No ticket yet.

Verification

  • pnpm ci:validate passes: check-types 8/8 tasks, test:unit 6/6 tasks (website: 64 tests).
  • route.test.ts calls the handler with a mocked client and covers 400 (including a year-0 date, a RID node type, one 5,000-character schema id, one 1,500-character ~ id and 1,000 schema ids), 401, 502, 403, 422, 500, 200, an empty body and the CORS preflight. Every response is checked for its CORS header.
  • Run live against a local database and next dev; see the Loom section. The 502 and 500 paths are covered by the unit tests only.

Loom video

fee-890-terminal-v2-proof.mp4

One video covers both FEE-890 PRs.

Setup: a local build of this branch at 37a4277b. Later commits refuse a non-document scale, answer 502 for an auth server error, refuse dates outside years 1 to 9999 and RID types, order entries with an explicit stack, rename name_conflict to unique_conflict and bound the schema lookup; unit tests cover those, not the video. The route ran under next dev from the PR worktree, against an isolated local Supabase stack started from this branch's migrations, with edge functions served so create-space could make two new spaces. Every call is curl, and every case reads Postgres back on screen.

  • 0:01 Setup: branch head, local stack, two new spaces A and B, each signed in as its anonymous account.
  • 0:12 One call writes the whole graph: nine items, none with modifiedAt. The cited node is written before the node citing it, nodes and the relation have their schema, and the triple carries the relation type's label and complement.
  • 0:29 Every row's last_modified equals its created.
  • 0:36 The same node sent again with a later modifiedAt updates the same row, and its last_modified moves forward. No new row is added.
  • 0:46 A triple missing a field, an item whose rid names another space, and a node with assets each get 400 naming the field. Nothing is written.
  • 0:58 No token, or a token the auth server rejects, gets 401. Nothing is written.
  • 1:08 Another space's token gets 403 here, and nothing is written. The same token works on its own space.
  • 1:18 A node type in neither the body nor the space gets 422 listing the missing id. A type that is only in the database works, and the node lands with that schema.
  • 1:32 A node reusing an existing name comes back in failed as name_conflict (now unique_conflict), while the other node in the call is written.
  • 1:43 An empty body gets 200 and writes nothing. The CORS preflight from roamresearch.com gets the allow-origin header.

Scope check

  • Ran $scope-check against FEE-890 and the final diff.
  • Scope beyond Done When: Refusing schema metadata, node assets, a non-document scale and RID node or relation types is narrower than "all aspects of upserting"; FEE-890 Create a generic upsert route that accepts all root CrossApp types (1/2) #1621's Reviewer brief gives the reasons. Whether FEE-890 ships without these is open with sid; no Decision: comment on FEE-890 records it yet. The schema-lookup bound refuses some long reference lists a gateway would accept, so a long list gets 400 instead of a reported 500.

Standards check

  • Ran $dg-pr-adherence-check against the final diff and PR metadata.

Local delegated full review

  • Ran a comprehensive review of the entire final diff in a subagent with a fresh context. Use $dg-delegated-full-review when no other full-review workflow is available.
  • Ran Review Lab (DiscourseGraphs/review-lab at 01b07f3) on 9d89088e: one P3 finding (B3), listed under Risk; nothing blocks review. Review Lab page

@vercel

vercel Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
discourse-graph Ready Ready Preview Nov 16, 2026 6:54am UTC

Request Review

@linear-code

linear-code Bot commented Nov 13, 2026

Copy link
Copy Markdown

FEE-890

@supabase

supabase Bot commented Nov 13, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project zytfjzqyijgagqxrzbmz because there are no changes detected in packages/database/supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@sid597
sid597 marked this pull request as ready for review November 16, 2026 04:04
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T04:09:01.873076Z 0ca8eed Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Devin Review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0ca8eedb4b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch was successfully deployed

1 active deployment
Preview — 9d89088e Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant