Skip to content

feat(fs): Node fs parity — node: imports, fs/promises, full method set (#122) - #123

Merged
spacedevin merged 5 commits into
mainfrom
feat/issue-122-node-fs-parity
Jul 6, 2026
Merged

feat(fs): Node fs parity — node: imports, fs/promises, full method set (#122)#123
spacedevin merged 5 commits into
mainfrom
feat/issue-122-node-fs-parity

Conversation

@spacedevin

Copy link
Copy Markdown
Member

Refs #122. Brings tish:fs toward Node fs / fs/promises parity on the bytecode VM (the default tish run backend).

Specifiers (resolver)

  • node: prefix is stripped → node:fs / node:fs/promises resolve like the bare forms
  • new fs/promises subpath; added the missing bare tty alias
import { readFileSync } from "node:fs"
import { readFile } from "node:fs/promises"

Methods (crates/tish_runtime/src/fs_ext.rs)

One *_core per op + a macro deriving the sync export (value / error-object — tish convention) and the promise export (fulfilled/rejected Promise — Node's fs/promises convention):

readFile/readFileBytes, writeFile, appendFile, exists, stat/lstat (Node-like Stats: isFile()/isDirectory()/size/mtimeMs/…), readdir, mkdir, rm/rmdir/unlink, rename, copyFile/cp, realpath, readlink, truncate, mkdtemp, access, plus constants (F_OK/R_OK/W_OK/X_OK).

Naming

Node names are primary (readFileSync, statSync, existsSync, …); the existing tish names (readFile, readDir, fileExists, isDir) are kept as aliases with their current contracts, so existing code (e.g. scii) is unaffected.

Async

tish:fs/promises exposes the async forms; fs now implies the promise feature (settled-promise machinery). All shipped/CI builds use --features full (already has promise via http), so the feature change is a no-op there.

Tests

Integration fixtures exercise the sync surface via node:fs (write/append/read/stat/mkdir/copy/readdir/rename/rm + constants) and the async surface via node:fs/promises (await + access rejecting on a missing path). Existing readFileBytes test and the full scii suite stay green.

Scope

This PR lands the resolver + VM backend — the default tish run path — with full sync + fs/promises parity. The native codegen (tish build), the interpreter backend, and the Node callback forms get the same surface as follow-ups, tracked in #122.

@codacy-production

codacy-production Bot commented Jun 12, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 79 complexity · 0 duplication

Metric Results
Complexity 79
Duplication 0

View in Codacy

🟢 Coverage 56.30% diff coverage · -0.05% coverage variation

Metric Results
Coverage variation -0.05% coverage variation (-1.00%)
Diff coverage 56.30% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (f7af3ac) 28592 21777 76.16%
Head commit (79ddce3) 28682 (+90) 21832 (+55) 76.12% (-0.05%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#123) 119 67 56.30%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@github-actions

github-actions Bot commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Test results

179 tests  +3   179 ✅ +3   8m 20s ⏱️ + 1m 48s
 27 suites +1     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 79ddce3. ± Comparison against base commit ff7d024.

♻️ This comment has been updated with latest results.

@spacedevin

Copy link
Copy Markdown
Member Author

Update — extended to all backends. This now covers the full surface on every backend, not just the VM:

  • Native codegen (tish build): Node names + new sync methods + a tish:fs/promises module emitted in generated Rust; the await machinery is now emitted for async fs programs (not only http), so await readFile(...) compiles without the HTTP stack. Verified by building + running a native binary that uses node:fs + node:fs/promises.
  • Interpreter (--backend interp): bridges to tishlang_runtime::fs_ext via value_convert (eval↔core); core promises convert to eval CorePromise, so fs/promises awaits work on interp too.

fs_parity tests now run the sync and fs/promises fixtures on both vm and interp. The only remaining follow-up tracked in #122 is the Node callback forms (fs.readFile(path, cb)).

@spacedevin

Copy link
Copy Markdown
Member Author

Callback form addedfs.readFile(path, (err, data) => …), writeFile(path, data, cb), stat(path, cb), etc. The sync functions are now dual: a trailing function arg switches to the Node (err, result) callback form; otherwise they return synchronously (scii's suite stays green — no code reads writeFile's return).

Final backend coverage

form VM (tish run) native (tish build) interp (--backend interp)
sync node:fs
async node:fs/promises
callback fs.x(…, cb) ⚠️ ⚠️

⚠️ Callbacks are VM-only for now: native codegen has a pre-existing limitation lowering a multi-arg callback closure passed to a native fn, and the interpreter can't bridge an eval function to a core native. Both are structural (not fs-specific) and tracked in #122. Sync + promises are full on every backend.

Tests: fs_parity now covers sync + promises on both vm/interp and the callback form (with the error path) on vm.

#122)

Brings tish:fs toward Node `fs` / `fs/promises` parity on the bytecode VM
(the default `tish run` backend).

Resolver (shared across backends):
- strip a leading `node:` so `node:fs` / `node:fs/promises` resolve like the
  bare forms; add the `fs/promises` subpath and the missing `tty` alias.

Runtime (crates/tish_runtime/src/fs_ext.rs):
- a Node-compatible fs surface with one `*_core` per op and a macro deriving
  the sync export (value / error object — tish convention) and the promise
  export (fulfilled/rejected Promise — Node's fs/promises convention).
- methods: readFile(Bytes)/writeFile/appendFile/exists/stat/lstat/readdir/
  mkdir/rm/rmdir/unlink/rename/copyFile/cp/realpath/readlink/truncate/
  mkdtemp/access, a Node-like `Stats` (isFile/isDirectory/size/mtimeMs/…),
  and `constants` (F_OK/R_OK/W_OK/X_OK).
- `fs` now implies `promise` (fs/promises returns settled promises).

VM registration:
- `tish:fs` gains the Node names (readFileSync/statSync/existsSync/…) and the
  new methods; existing tish names (readFile/readDir/fileExists/isDir) keep
  their current contracts as aliases.
- new `tish:fs/promises` module exposes the async forms.

Tests: integration fixtures exercise the sync surface via `node:fs` and the
async surface via `node:fs/promises` (await + access-rejects-on-missing).

Scope: this lands the resolver + VM backend (the default path). The native
codegen (`tish build`) and the interpreter backend get the same surface, plus
the Node callback forms, as follow-ups tracked in #122.

Refs #122
Brings the same Node fs / fs/promises surface to the other two backends so all
of `tish run` (vm + interp) and `tish build` (native) reach parity.

Native codegen (tish build):
- emit the Node-named + new sync methods and a tish:fs/promises module in the
  generated Rust (referencing tishlang_runtime::fs_ext directly).
- emit the await machinery (await_promise / promise_object) for async programs
  whenever fs is enabled, not only http — so `await readFile(...)` from
  fs/promises compiles without the http stack.

Interpreter (tish_eval):
- bridge a Node-compatible fs surface to tishlang_runtime::fs_ext through
  value_convert (eval args -> core, call, core -> eval). Core promises convert
  to eval CorePromise, so fs/promises awaits work on interp too. `fs` now pulls
  in the runtime dep for the bridge.

Tests: fs_parity now exercises the sync and fs/promises fixtures on BOTH the vm
and interp backends; the native path is verified by building and running a
binary that uses node:fs + node:fs/promises.

Refs #122
The sync fs functions are now dual: when the last argument is a function they
take the Node callback form — run the op, then invoke `cb(null, result)` on
success or `cb(err, null)` on failure (synchronously, since the I/O is sync) —
and otherwise return synchronously as before. So `fs.readFile(p, (err,data)=>…)`,
`writeFile(p, data, cb)`, `stat(p, cb)`, `unlink(p, cb)`, … all work.

The tish names readFile/writeFile/readDir/readFileBytes now route to the dual
fs_ext functions too (sync behavior unchanged — scii's full suite stays green;
no code reads writeFile's return).

Backend coverage: the callback form works on the VM (the default `tish run`).
Native codegen has an existing limitation lowering a multi-arg callback closure
passed to a native fn, and the interpreter can't bridge an eval function to a
core native — so callbacks are VM-only for now; sync + fs/promises remain on all
backends. Tracked in #122.

Test: fs_parity_callbacks exercises the (err, data) form and the error path
(missing file -> non-null err) on the VM.

Refs #122
Node's mkdtemp appends 6 random chars; the timestamp-hex suffix was both
predictable (a temp-dir security smell — symlink/collision races) and
collision-prone under rapid calls. Use a random base-36 suffix (~51 bits) with
create_dir's exclusive semantics and a collision-retry loop. Adds rand as an
fs-gated dep of tish_runtime (already in the tree via tish_builtins).
Verified distinct/unique/secure; fs_parity + parity + native batch all green.
@spacedevin
spacedevin force-pushed the feat/issue-122-node-fs-parity branch from 7e2eab3 to 5963eae Compare July 6, 2026 22:51
@spacedevin

Copy link
Copy Markdown
Member Author

Rebased onto current main (was 213 commits behind — Value/ObjectData API era) and modernized:

  • Rebased clean onto main; compiles with no API-drift fixes needed (the *_core + macro design ports as-is).
  • Passes all modern gates that didn't exist when this was opened: cross-backend parity (interp==vm==node) 95/0, native batch 1/0, the fs_parity integration test 3/0; node:fs sync + fs/promises verified identical across interp/vm/native.
  • Security fix: mkdtemp now uses a random base-36 suffix (with exclusive create_dir + collision-retry) instead of a predictable timestamp — the old {prefix}{nanos:x} was predictable (temp-dir symlink/race smell) and collision-prone under rapid calls.
  • Confirmed the sync error-object convention still matches modern main (read_filemake_error_value, not a throw), and the boxed byte-array return matches main's existing readFileBytes contract — so no behavior drift.

Ready for review/merge.

@spacedevin

Copy link
Copy Markdown
Member Author

Callback-form status update (re-tested on the rebased branch):

  • Native: the limitation is GONE. readFile(path, (err, data) => …) fires the callback correctly on the native backend — success path ((null, data)) and error path ((errObj, null)), byte-identical to the VM. The codegen work in the ~200 commits since this PR was opened (emit_arrow_function now binds every param via args.get(i)) resolved the multi-arg-closure-to-native-fn lowering. So the "native codegen has a pre-existing limitation" note is stale.
  • Interp: still divergesreadFile(path, cb) falls back to the sync form (returns the content; the callback never fires). This is genuinely structural: fs is registered as a self-less Value::Native bridged to core fs_ext, and eval_to_core can't wrap an eval closure as a core value, so the bridge can't invoke the callback. It IS resolvable — the interp already invokes eval callbacks for map/filter/sort via self.call_func (70 call-sites in eval.rs); the fs callback form just needs the same eval-level handling (hook the Value::Native dispatch at eval.rs:3838: match the fs callback fns, run the sync op, split value/error into (err, data), and invoke the callback via self.call_func) instead of the core bridge.

Sync + promises remain full on every backend (interp/vm/native). Net: callbacks now work on VM and native; only the interp callback form is outstanding — a bounded interp change, not a fundamental limit.

…every backend

Resolves the last gap in #122's callback surface. The native codegen limitation
was already fixed upstream (multi-arg closure lowering); this closes the
interpreter side.

The interpreter binds fs to self-less `Value::Native` fns bridged to core
`fs_ext`, and a core `Callable` can't re-enter the interpreter, so the callback
was silently dropped (readFile(path,cb) fell back to the sync form). Fix: handle
the Node callback convention in `call_func`'s Native arm, where `&self` is
available — map the fs native (old alias or fsx bridge) to its core op via
`fsx::callback_core`, run it on the non-callback args, split the Result into
`(err, data)`, and invoke the callback with `self.call_func` (the same eval-level
path map/filter/sort callbacks already use). Exposes the fs `*_core` ops as pub.

readFile/writeFile/stat/… callbacks now fire identically on interp/vm/native
(success + error paths). fs_parity callback test now asserts all three backends;
parity 95/0, native batch green.
@spacedevin

Copy link
Copy Markdown
Member Author

Interp callback form now works — full parity on every backend.

Fixed the last gap. readFile(path, (err, data) => …), writeFile, stat, … now fire the callback identically on interp / vm / native (success path (null, data) and error path (errObj, null)).

How (interp): a self-less Value::Native bridge can't re-enter the interpreter to invoke an eval callback, so I handled the Node callback convention in call_func's Value::Native arm where &self is available — fsx::callback_core maps the fs native (old alias or fsx bridge, via fn_addr_eq) to its core op, runs it on the non-callback args, splits the Result into (err, data), and calls the callback with self.call_func (the same eval-level path map/filter/sort already use). The fs *_core ops are now pub.

The fs_parity callback test now asserts all three backends (was VM-only). Green: fs_parity 3/0, cross-backend parity 95/0, native batch 1/0. So "sync + promises + callbacks" are now full on every backend.

@spacedevin
spacedevin merged commit 48c4692 into main Jul 6, 2026
32 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant