Skip to content

Commit e7c0184

Browse files
Merge pull request #28 from Zipstack/docs/spec-upgrade-skill
[DOCS] Add a skill for the spec sync and regeneration pipeline
2 parents d077662 + 4dbba55 commit e7c0184

1 file changed

Lines changed: 106 additions & 0 deletions

File tree

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
---
2+
name: spec-upgrade
3+
description: Sync the OpenAPI spec from the Unstract backend into this repo and regenerate the transport layer. Use whenever the backend adds or changes a Document Studio API deployment endpoint, when the `sdk-drift` CI job goes red, when `specs/docstudio-oss.json` is behind the backend, or when someone asks to "regenerate the SDK", "refresh the spec", "pick up the new endpoint", or "release unstract-client". Reach for this even when the request only mentions the generated code or a new endpoint — the upgrade is an ordered pipeline, and skipping a step leaves the drift gate red or the spec silently stale.
4+
---
5+
6+
# Upgrading the spec and regenerating the client
7+
8+
Nothing tells this repo when the backend's spec moves. Someone notices, and then
9+
runs this pipeline. The steps are ordered because each one's output is the next
10+
one's input, and because two of them are the only thing standing between a stale
11+
copy and a client that looks generated but isn't.
12+
13+
## The pieces
14+
15+
| Thing | Where |
16+
|---|---|
17+
| Vendored spec | `specs/docstudio-oss.json` |
18+
| Generator wrapper | `tools/gen_sdk.sh` (pins the generator, records `SPEC_SOURCE`) |
19+
| Generated transport | `src/unstract/api_deployments/_sdk_docstudio/` — never hand-edited |
20+
| Hand-written facade | `src/unstract/api_deployments/client.py` |
21+
| Drift gate | `sdk-drift` job in `.github/workflows/test.yml` |
22+
| Release | `.github/workflows/main.yml`, `workflow_dispatch` |
23+
24+
Upstream, the spec is produced by the backend that serves these endpoints
25+
(`Zipstack/unstract`, `manage.py generate_docstudio_spec`, committed at
26+
`specs/docstudio-oss.json`). It is a copy here, not a fork.
27+
28+
## The sequence
29+
30+
1. **Copy the spec byte-for-byte** from the backend commit you are upgrading to.
31+
Do not reformat it and do not edit it here — the drift gate regenerates from
32+
whatever is committed, so an edit here becomes a client that no backend
33+
serves.
34+
35+
2. **Move `SPEC_SOURCE` in `tools/gen_sdk.sh` in the same commit.** It records
36+
the backend repo, path, revision and sha256. Without that update, nothing
37+
distinguishes a current copy from one the backend has moved past — the script
38+
and the drift gate both report clean either way, which is exactly the failure
39+
this pipeline exists to prevent.
40+
41+
3. **Regenerate:**
42+
43+
```bash
44+
./tools/gen_sdk.sh
45+
```
46+
47+
It rebuilds the tree from scratch with a pinned generator. If it exits
48+
non-zero on a generator warning, believe it: the generator downgrades a
49+
schema it cannot parse to a warning, drops the endpoint or model, writes the
50+
rest and exits 0 — so the warning is the only signal that an operation is
51+
missing.
52+
53+
4. **Review the generated surface, with new files included:**
54+
55+
```bash
56+
git add -N -- src/unstract/api_deployments/_sdk_docstudio
57+
git diff --stat -- src/unstract/api_deployments/_sdk_docstudio
58+
git diff -- src/unstract/api_deployments/_sdk_docstudio
59+
```
60+
61+
The `--stat` is the shape of the change; the full diff is the change. Read
62+
both — the operations and fields that moved only show up in the second.
63+
64+
`git add -N` first because a plain diff cannot see a file the generator has
65+
newly created — which is precisely what a spec that grew an endpoint
66+
produces. Read the diff as the API change it represents: new modules are new
67+
operations, removed fields are removals.
68+
69+
5. **Make the new surface reachable, in `client.py`.** Regeneration only moves
70+
the transport. If the spec added an endpoint callers should be able to use,
71+
the facade is where it becomes public API. Fixes belong here or upstream in
72+
the spec, never in the generated tree — regeneration overwrites that wholesale.
73+
74+
6. **Run the tests:** `uv run pytest tests/`. `tests/test_compat.py` compares
75+
this client against the last released one, vendored under `tests/baseline/`.
76+
Refresh that baseline only when you mean to move the parity reference point,
77+
with `tools/refresh_baseline.sh <released-version>`; a spec upgrade on its own
78+
is not a reason to move it.
79+
80+
## CI
81+
82+
`sdk-drift` regenerates from the committed spec and fails on any diff, so a
83+
hand-edit of the generated tree and a spec change nobody regenerated over both
84+
fail the same way. If it is red, run step 3 and commit the result.
85+
86+
## Versioning and release
87+
88+
Choose the bump by what changed for callers: **major** when the spec removed or
89+
renamed something callers depend on, **minor** for new endpoints or new
90+
behaviour, **patch** for fixes that keep the surface identical. A generated diff
91+
with removals in it is the signal for major — spec upgrades produce those.
92+
93+
Do not touch `__version__` in `src/unstract/api_deployments/__init__.py` in your
94+
PR. The in-repo value is the *last released* version; `main.yml` reads it,
95+
applies the bump you pick at dispatch time, and commits the result itself. Bumping
96+
it in the PR makes the release skip a version.
97+
98+
Release by dispatching **Release Tag and Publish Package** on `main` and choosing
99+
the bump. It publishes to PyPI *before* it tags — a failure after publish is
100+
retried by hand against a live artifact, not by re-publishing.
101+
102+
## Downstream
103+
104+
`unstract-cli` pins this client exactly and vendors a copy of the same spec.
105+
After a release lands on PyPI, that pin needs bumping there; see the
106+
`bump-client-pins` skill in that repo.

0 commit comments

Comments
 (0)