feat(genext2fs): add @deroll/genext2fs, Node.js bindings for xgenext2fs - #194
Merged
Merged
Conversation
🦋 Changeset detectedLatest commit: 927b1ba The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Binds cartesi/genext2fs to Node.js as an N-API addon, so a tar archive can be turned into an ext2 drive image from inside a Node process instead of shelling out to a container or an installed CLI. xgenext2fs is a command line program, not a library: a single 100 KB C file that parses argv, prints to stderr and calls exit() on every error. Rather than fork it, both upstream projects are vendored as git submodules and compiled unmodified: deps/genext2fs cartesi/genext2fs v1.5.6 deps/libarchive libarchive v3.8.9, required by the fork's tar reader src/xgenext2fs_lib.c textually includes xgenext2fs.c with main renamed, exit() redirected through longjmp() and stdout/stderr redirected into temporary files, which is what turns the CLI into a callable function whose failures unwind and whose diagnostics become strings. Only the 16 libarchive translation units needed to read an uncompressed tar are compiled, and src/libarchive_formats.c narrows support_format_all/support_filter_all to them, so the addon links against nothing but libc. Neither build system runs under node-gyp, so the two generated config.h files are replaced by the hand-written ones under config/. The TypeScript layer exposes tarToExt2/tarToExt2Buffer for the main use case, createImage for multi-layer images, and genext2fs(args) as a raw argv escape hatch, each with a blocking counterpart. Options map one to one onto the CLI flags, gzipped archives are inflated in JS, and calls are serialized behind a mutex because xgenext2fs keeps parser state in globals. xgenext2fs' own size estimate comes out a few blocks short for some archives; createImage retries deterministically with a slightly larger explicit size rather than surfacing a mid-build failure. Note the package is GPL-2.0-only, inherited from xgenext2fs, unlike the Apache-2.0 packages around it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZ516LsVnnu4ySXKkXsNbB
tarToExt2Buffer/tarToExt2BufferSync went in on the premise that they saved a round trip to disk. They did not: xgenext2fs only ever writes to a file, so they built into a temporary directory and read the result back, which is exactly what a caller can do in two lines when they want the bytes. The genext2fs(args) raw-argv escape hatch existed for options the typed surface did not model. There are none: of the 22 long options, 19 map onto ImageOptions or a layer type, --version is the exported `version`, and --help is meaningless in a library. Checking that list also turned up two mistakes in CreatorOs — "masix" is not recognized by lookup_creator_os and falls back to Linux, "GNU" is an accepted alias for hurd, and a raw number is accepted for an OS the tool has no name for. What is left is tarToExt2/createImage plus their blocking counterparts. Tests moved onto the remaining API rather than being dropped, and gained coverage the escape hatch used to provide: the full option surface in one call, colon rejection in layer paths, error status/stderr, and autoSize growth versus autoSize: false. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JZ516LsVnnu4ySXKkXsNbB
tuler
force-pushed
the
claude/genext2fs-nodejs-binding-5ij7ae
branch
from
August 4, 2026 20:43
b3db476 to
927b1ba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Binds cartesi/genext2fs to Node.js as an N-API addon, so a tar archive can be turned into an ext2 drive image from inside a Node process instead of shelling out to a container or an installed CLI.
Approach
xgenext2fsis a command line program, not a library: a single 100 KB C file that parsesargv, prints tostderrand callsexit()on every error. Rather than fork it, both upstream projects are vendored as git submodules and compiled unmodified bynode-gyp:deps/genext2fsv1.5.6deps/libarchivev3.8.9add2fs_from_tarball()callsarchive_read_*unconditionally, so libarchive is a hard requirement (the README's "builtin tarball parser" note is stale)Three pieces make that work:
src/xgenext2fs_lib.ctextually includesxgenext2fs.cwithmainrenamed,exit()redirected throughlongjmp(), andstdout/stderrredirected into temporary files — which is what turns the CLI into a callable function whose failures unwind and whose diagnostics come back as strings. Upstream is not patched at all.src/libarchive_formats.cnarrowsarchive_read_support_format_all/support_filter_allto tar + none, so only 16 of libarchive's 217 translation units are compiled and the addon links against nothing but libc. No zlib/bz2/lzma, no system libarchive; gzip is inflated in JS instead.config/holds hand-written replacements for the two generatedconfig.hfiles, sincenode-gypcannot run./configure. POSIX features are asserted directly, with__APPLE__/__linux__branches for the few that genuinely differ.Submodules over copying the sources: provenance is exact, bumps are a
git checkout, and it matches the existing@deroll/cmiopattern. The published tarball carries only the files actually compiled.Because xgenext2fs keeps parser state in globals and
chdir()s while reading directory layers, calls are serialized behind a mutex in the addon; async calls still generate off the main thread.API
Four functions:
tarToExt2(tar, output, options?)— the main use case, archive as a path or bytes, gzip inflated transparentlycreateImage(output, options?)— images assembled from several layers (tarball/directory/devtable, each optionally at a path inside the image)tarToExt2Sync/createImageSync— the same work on the calling threadThere is deliberately no raw-argv escape hatch: of xgenext2fs' 22 long options, 19 map onto
ImageOptionsor a layer type,--versionis the exportedversion, and--helpis meaningless in a library. Auditing that list also correctedCreatorOs—masixis not recognized bylookup_creator_os()and silently falls back to Linux,GNUis an accepted alias forhurd, and a raw number is accepted for an OS the tool has no name for.Errors reject with the tool's own diagnostic, carrying
status,stdoutandstderr.Two things worth a look before merging
License. xgenext2fs is GPL-2.0-only (not "or later") and this package compiles it in, so
@deroll/genext2fsis GPL-2.0-only — unlike every other package in the repo, which are Apache-2.0. Called out inpackage.json, the package README, the docs andCLAUDE.md, but whether that belongs in this monorepo is a call for a human.Upstream sizing bug. With no
-b, xgenext2fs under-estimates the image by 3–13 blocks for some archives and dies mid-build withcouldn't allocate a block (no free space). The standalone CLI does the same — which is why Cartesi's tooling always passes an explicit-b.createImageabsorbs it: on that specific failure it retries with a size derived from the failed attempt (the partial file is truncated toblocks * blockSize, so the estimate is readable from disk). Deterministic, so images stay reproducible.autoSize: falseopts out andsizeInBlockspins it.Also in this PR
genext2fs-prebuildmatrix (linux/macOS × x64/arm64) gated on the same "is this version already on npm" check ascmio/cm, with prebuilds collected into the package before publish. Unlike its siblings this job needs no system packages.genext2fssection (intro, API, options) plus top-nav and sidebar entries.--recurse-submodules..changeset/pre.json.Verification
19 tests pass — e2fsck-clean images, byte-identical output across runs with
faketime, gzip andUint8Arrayinput matching path input, the full option surface in a single call, concurrent conversions,autoSizegrowth versusautoSize: false, errorstatus/stderr, and the addon staying usable after a failed run. I also packed the tarball, extracted it into a clean directory and confirmed the from-source build works end to end, which is the real check on thefileslist and the submodule strategy.bun run lint,bun run buildandbun run testare green across the workspace, and the docs site builds with the twoslash snippets typechecked.Not included: an entry in
apps/examples— happy to add one if you want it.🤖 Generated with Claude Code
https://claude.ai/code/session_01JZ516LsVnnu4ySXKkXsNbB