Skip to content

feat(core): cache SDK outputs with their .git, long paths and a raised size limit - #893

Merged
totollygeek merged 2 commits into
masterfrom
feat/cache-sdk-outputs
Oct 21, 2026
Merged

totollygeek merged 2 commits into
masterfrom
feat/cache-sdk-outputs

Conversation

@totollygeek

Copy link
Copy Markdown
Contributor

What & why

A target that installs a git-checkout SDK such as Flutter's into .zuke/tools couldn't be cached remotely, for three reasons:

  1. The archive skipped the SDK's own .git, and restore refused it. Flutter runs git on its own checkout, so the SDK is broken without it.
  2. The 2 GiB bound on what a restore inflates to was hard-coded, so a 2.3 GB SDK was always refused.
  3. tar() threw on any entry name or symlink target longer than 100 bytes, and deep SDK trees have many of them.

The fixes:

  • .git under .zuke/tools: a .git segment is now allowed when it comes after a .zuke/tools pair. This grants a store nothing new, because a store that can write there can already replace the SDK's binaries and scripts.
    • Still refused: the workspace's .git, a nested src/.git, and anything under .zuke outside tools.
    • Also refused now: a segment with trailing dots or spaces, such as .git. or .git , because Windows drops them and would resolve the entry to the real directory. The adversarial review found this older hole.
  • Configurable restore limit:
    • RemoteCacheStore gains an optional maxInflatedBytes, which every built-in store accepts in its options: FileSystemCacheStoreOptions (new), HttpCacheStoreOptions, S3CacheStoreOptions and GitHubActionsCacheStoreOptions.
    • A store chosen from the environment reads ZUKE_REMOTE_CACHE_MAX_INFLATED_BYTES. The variable is read only once a store is chosen, so a bad value can't fail a build that has no remote cache.
    • The executor passes the store's bound to restoreOutputs. A bound that isn't a positive whole number is refused, because NaN would otherwise switch the check off.
  • Long tar names: tar() now writes a pax extended header (typeflag x) with path and linkpath records for long values, which untar and GNU/BSD tar already read. Archives with only short names are byte-identical to before, so artifacts stay reproducible.

Tests:

  • Unit tests cover:
    • pax round trips (multi-byte names, a long link target) and .git allowed under tools but refused elsewhere, including the trailing-dot spellings;
    • environment parsing, including the lazy read;
    • bound validation, and a cache that honours a store's limit.
  • The integration test in tests/integration/caching_test.ts runs a real build whose .zuke/tools/sdk output has a .git and a path over 100 bytes. It restores both on a fresh checkout, then shows that a lower ZUKE_REMOTE_CACHE_MAX_INFLATED_BYTES turns the restore into a warned rebuild.

The adversarial review also checked:

  • byte-identity with the old writer over 2000 random short-name archives;
  • pax record lengths across every digit boundary;
  • the .., case and backslash variants of a .git escape.

Known limits:

  • The restore still buffers the whole archive in memory, so the bound is also the memory ceiling.
  • An HTTP or S3 store also caps the compressed download at 512 MiB by default (maxArtifactBytes), so a large SDK needs that raised too. The GitHub Actions store's default wire cap is 2 GiB.

Related issues

Closes #890

Checklist

  • This PR closes a tracking issue (Closes #<n> above).
  • The PR title is a Conventional Commit (type(scope): summary).
  • deno task ci passes locally (lint, fmt, type-check, tests, spell).
  • Tests were added or updated; coverage stays at 95%+ (lines and branches).
  • Docs updated in the same PR (README.md, JSDoc, docs/) when behaviour changed.
  • Public API changes were regenerated with ./zuke apiDocs (llms.txt, llms-full.txt, package README ## API).
  • No any, no as casts or ! non-null assertions in src/ (narrow with type guards instead).
  • No dead code.
  • No duplicated logic.
  • SOLID shapes respected.
  • A new package was wired into all six places, if applicable.
  • The code is written using AI assisted coding.

Generated by Claude Code

…d size limit

A target that installs a git-checkout SDK such as Flutter's into .zuke/tools
could not be cached. The archive dropped the SDK's own .git, the tar writer
refused any name or link target past ustar's 100 bytes, and the 2 GiB bound
on what a restore inflates to was fixed.

A .git inside .zuke/tools now archives and restores, while the workspace's
own .git and any nested one elsewhere stay refused, as does one spelled with a
trailing dot or space, which Windows would resolve to the real directory. The
tar writer emits a pax extended header for a long name or link target, which
untar and modern tar already read. Every built-in store takes
maxInflatedBytes, an env-selected store reads
ZUKE_REMOTE_CACHE_MAX_INFLATED_BYTES once a store is chosen, the executor
passes the store's bound to the restore, and a bound that is not a positive
whole number is refused rather than disabling the check.

Closes #890

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015ni2F1uPKAVqstioh8EpG8
@zuke-build

zuke-build Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

🤖 Zuke AI review

🔎 AI review — review

✅ Passed — no findings.

Reviewer Verdict Score Severity Findings
🛡️ security review ✅ passed 0/10 none 0
🧹 generic review ✅ passed 0/10 none 0
🛡️ security review — ✅ passed · 0/10

Score: 0/10 · Severity: none · 0 finding(s)

Tokens: 34990 in · 174 out · 35164 total

Budget: 61,238 tokens of 500,000 tokens

Commands

Only comments from maintainers the host itself attributes as such are read; anyone else's are ignored before anything sees them.

Say What happens
@zuke-build review Runs the review again now, with no push: every rebuttal is adjudicated and answered where it was made.
@zuke-build accept <id> <reason> Accepts a finding as intended for this pull request: it is recorded with your reason, its thread is closed, and no reviewer raises it again here — reworded, moved to another file or escalated. In the finding's own thread the id may be left out.
Reply in a finding's thread, or quote its id in a comment Contests it: the next run re-checks the finding against your argument and dismisses it when the argument holds, or says what is missing.

Refuted by verification (not reported):

Finding Reason
New tar PAX fallback allows long entries but still depends on archive readers handling them safely The diff does not introduce a security weakness here: tar() now emits pax headers only for overlong names/link targets, while restore paths still validate names and links via assertSafeEntryName, assertSafeLinkTarget, and the protected-path checks in restoreOutputs/isForbiddenPath, so the evidence shows a supported format extension rather than a new unsafe extraction path.

Reviewer summary, written before verification: The change is largely a safe documentation and bounded-cache enhancement, with one potentially confusing exposure of the new PAX tar behavior but no clear security break in the diff.

🧹 generic review — ✅ passed · 0/10

Score: 0/10 · Severity: none · 0 finding(s)

Tokens: 22966 in · 405 out · 23371 total

Budget: 99,533 tokens of 500,000 tokens

Commands

Only comments from maintainers the host itself attributes as such are read; anyone else's are ignored before anything sees them.

Say What happens
@zuke-build review Runs the review again now, with no push: every rebuttal is adjudicated and answered where it was made.
@zuke-build accept <id> <reason> Accepts a finding as intended for this pull request: it is recorded with your reason, its thread is closed, and no reviewer raises it again here — reworded, moved to another file or escalated. In the finding's own thread the id may be left out.
Reply in a finding's thread, or quote its id in a comment Contests it: the next run re-checks the finding against your argument and dismisses it when the argument holds, or says what is missing.

Refuted by verification (not reported):

Finding Reason
Tar pax-header support is implemented without preserving the original entry-level validation path The diff keeps the low-level invariants in tar() itself: it still validates mode, encodes the inputs there, and tarHeader() is now a private helper only called from that path with pre-truncated byte arrays, so there is no new reachable malformed-header path shown here.
Env-selected inflate limit is validated in a separate helper that duplicates the same guard shape used by restoreOutputs The two validations are not inconsistent drift: envInflatedLimit() only parses the environment variable once in envCacheStore, while restoreOutputs() validates the runtime maxBytes parameter for all callers, and the shared policy is already centralized in envInflatedLimit() rather than duplicated.
Documentation change broadens .git allowances without a corresponding explicit rule in code comments for all store types isForbiddenPath() explicitly encodes the special case for .git under .zuke/tools via the tools index check, so the implementation already matches the broadened documentation rule instead of leaving it only in a comment.

Reviewer summary, written before verification: The change adds configurable inflate limits and pax tar support with good test coverage, but it also introduces a couple of maintainability and behavior risks around validation and duplicated path-protection logic.

@zuke-build

zuke-build Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

🤖 Zuke AI review

🔎 AI review — review

✅ Passed — no findings.

Reviewer Verdict Score Severity Findings
🛡️ security review ✅ passed 0/10 none 0
🧹 generic review ✅ passed 0/10 none 0
🛡️ security review — ✅ passed · 0/10

Score: 0/10 · Severity: none · 0 finding(s)

Tokens: 35232 in · 185 out · 35417 total

Budget: 62,297 tokens of 500,000 tokens

Commands

Only comments from maintainers the host itself attributes as such are read; anyone else's are ignored before anything sees them.

Say What happens
@zuke-build review Runs the review again now, with no push: every rebuttal is adjudicated and answered where it was made.
@zuke-build accept <id> <reason> Accepts a finding as intended for this pull request: it is recorded with your reason, its thread is closed, and no reviewer raises it again here — reworded, moved to another file or escalated. In the finding's own thread the id may be left out.
Reply in a finding's thread, or quote its id in a comment Contests it: the next run re-checks the finding against your argument and dismisses it when the argument holds, or says what is missing.

Refuted by verification (not reported):

Finding Reason
PAX long-name support should preserve the existing archive path validation surface The diff shows tar() now emits pax headers for long names/link targets, but restoreOutputs still enforces assertSafeEntryName, assertSafeLinkTarget, and protected-path checks via isForbiddenPath, so the archive format extension does not open a new extraction bypass.

Reviewer summary, written before verification: The change mainly adds configurable inflated-size limits and broader tar support; the only notable risk is a newly accepted archive shape that increases the attack surface, but the restore path still validates names and targets before writing.

🧹 generic review — ✅ passed · 0/10

Score: 0/10 · Severity: none · 0 finding(s)

Tokens: 23377 in · 206 out · 23583 total

Budget: 101,484 tokens of 500,000 tokens

Commands

Only comments from maintainers the host itself attributes as such are read; anyone else's are ignored before anything sees them.

Say What happens
@zuke-build review Runs the review again now, with no push: every rebuttal is adjudicated and answered where it was made.
@zuke-build accept <id> <reason> Accepts a finding as intended for this pull request: it is recorded with your reason, its thread is closed, and no reviewer raises it again here — reworded, moved to another file or escalated. In the finding's own thread the id may be left out.
Reply in a finding's thread, or quote its id in a comment Contests it: the next run re-checks the finding against your argument and dismisses it when the argument holds, or says what is missing.

Refuted by verification (not reported):

Finding Reason
env-selected cache limit is not applied to the local directory store path when the env var is malformed but a store is otherwise present The env var is not silently ignored when a store is present: envCacheStore() now calls envInflatedLimit() in every selected backend branch (HTTP, S3, directory, and Actions), and if the selected store is absent it deliberately returns undefined without parsing, which the added tests cover.

Reviewer summary, written before verification: The change is generally cohesive and well-tested, with a small hardening concern around the new cache-size environment knob handling and documentation drift risk.

@totollygeek
totollygeek merged commit 216de10 into master Oct 21, 2026
16 of 17 checks passed
@totollygeek
totollygeek deleted the feat/cache-sdk-outputs branch October 21, 2026 18:14
@github-actions github-actions Bot mentioned this pull request Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: let cached outputs keep .git under .zuke/tools, and make the restore size limit configurable

2 participants