Skip to content

test(crawl): add a first-install acceptance test for the crawl command - #32

Merged
StefanoGuerrini merged 4 commits into
feat/crawl-command-v2from
test/crawl-acceptance
Oct 12, 2026
Merged

StefanoGuerrini merged 4 commits into
feat/crawl-command-v2from
test/crawl-acceptance

Conversation

@StefanoGuerrini

@StefanoGuerrini StefanoGuerrini commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #31 (feat/crawl-command-v2).

Summary

This PR adds an acceptance test for the zenrows crawl command of @zenrows/cli. It installs the release artifact the way a new user does and runs every Crawl feature against the live API. It stays in its own PR, so the feature PR stays focused.

What it does

acceptance/crawl/run.sh builds the release artifact from this checkout: npm install -g of the npm pack tarball. It installs the artifact in a clean container with digest-pinned images. The container gets no source, no build caches and no tokens, and it gets the key by name only. The run follows the read-me (README) Crawl quickstart, then runs checks 1 to 21 and the CLI extras. Each check writes one JSON result with status PASS, FAIL or SKIP. A markdown report lands in acceptance/crawl/out/, which git ignores. The files list in package.json keeps the harness out of the published package.

# Feature # Feature
1 Create 12 Not found
2 Create with html 13 Invalid input
3 Idempotency 14 Gated option
4 Wait 15 Bad key
5 Wait runs out 16 Errors carry the server code
6 Get 17 Naming
7 List 18 Copy
8 Results 19 Failed crawl
9 Content 20 Results after stop
10 Download 21 Documented defaults
11 Stop

The CLI extras check that every subcommand prints JSON with --json. They check that Ctrl-C during crawl wait prints the resume hint and exits 130. They also check that the CLI rejects --wait and --html as unknown flags. acceptance/crawl/README.md gives the expected result of each check, the pinned versions, and how to read the report.

Three checks changed after the first commit:

  • Check 14 sends --output-format json. A refusal before the API (INVALID_USAGE) now counts as a pass, as does a 422 (CRAWL_INVALID_REQUEST).
  • Check 9 needs a fetched page. The stop in check 20 can land before the crawl fetches a page. Then check 9 starts one more small html crawl and waits for its end.
  • The naming check (17) now also covers content_id.

Cost and safety

Every crawl uses depth 1 and max_pages 10 or less. A run starts five crawls, and check 9 can start one more. A run has at most 2 crawls active at a time, because an account has 3 active jobs, shared with Batch. If a create gets a 429, the run waits Retry-After and tries again. A clean-up step stops every crawl the run started, also when a check fails or on Ctrl-C. The script never prints the key. Before it ends, it scans every file under out/ for the key.

How to run it

The host needs bash, git and Docker, and a key with Crawl access.

export ZENROWS_API_KEY="$(cat path/to/key-file)"
bash acceptance/crawl/run.sh              # every check
bash acceptance/crawl/run.sh --no-key     # offline checks only; live checks SKIP
bash acceptance/crawl/run.sh --report     # rebuild the report from out/
REF=v1.2.3 bash acceptance/crawl/run.sh   # test a tag, branch or SHA from GitHub

The report is acceptance/crawl/out/crawl-first-install-report.md. Its verdict is "ready" when it has no blocker or major finding.

Squash-merge with the title as the message: the branch commits carry trailer lines that should not land.

acceptance/crawl/ builds the release artifact from this checkout, installs
it in a clean container the way a new user would, and runs every Crawl
feature against the live API: README example, create, wait, results,
content, download, stop, errors, naming and docs. One JSON result per
check and a markdown report land in acceptance/crawl/out/ (git-ignored).
It is kept out of the published package.
The naming check now covers content_id, the argument of the content
operation, so a client that names it differently fails check 17. The MCP
surface calls crawl_content with content_id.
…al before the API as a pass on check 14

Check 9 read content from the crawl check 20 stops; a stop that lands before any page is fetched left nothing to read. It now falls back to a one-item html crawl that runs to the end. Check 14 passes when the client refuses the gated output format before calling the API, as well as on the API's 422.
@StefanoGuerrini StefanoGuerrini changed the title test(crawl): add a first-install acceptance test for the Crawl client test(crawl): add a first-install acceptance test for the crawl command Oct 12, 2026
@StefanoGuerrini
StefanoGuerrini marked this pull request as ready for review October 12, 2026 14:23
@StefanoGuerrini
StefanoGuerrini merged commit b6b559e into feat/crawl-command-v2 Oct 12, 2026
2 checks passed
@StefanoGuerrini

Copy link
Copy Markdown
Contributor Author

Merged into the feature branch with a merge commit. It ships to main with #31.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant