Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/java-notices.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@scalar/java-fmt': patch
---

Complete the third-party notices: list commonmark-java 0.28.0 (BSD-2-Clause), which google-java-format bundles and which is compiled into the artifact, with its license text. Also ship TeaVM's `NOTICE` file, as Apache-2.0 asks. No change to the artifact or its output.
15 changes: 15 additions & 0 deletions .changeset/kotlin-ktrs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
'@scalar/kotlin-fmt': minor
---

Replace the TeaVM-compiled ktfmt 0.64 with [ktrs](https://github.com/Hexay/ktrs) 0.5.3, a Rust reimplementation of ktfmt, compiled to wasm. Output is held to ktfmt **0.65** by a conformance test that runs the real ktfmt jar. That test asserts byte-identical output and matching diagnostics; over okhttp and kotlinx.coroutines (1,699 files) in every style, every file matched.

The artifact goes from 0.82 MB to 0.20 MB and imports nothing, so `formatSync` now works under Node without `init()`, and the Node floor drops from 24.15.0 to 18.

Breaking changes:

- The `./runtime` export (TeaVM's generated runtime) is gone. There is no runtime file any more.
- The `BootModule`, `FormatFunction` and `ModuleLoader` types are removed. `ArtifactSource`, `KotlinFormatter` and `KotlinFormatterBoot` take their place.
- `ktfmtVersion` is now `'0.65'`, and `ktrsVersion` is new.
- Parse errors read as ktfmt's CLI prints them, without the file name (`1:7: error: Expecting ')'`), instead of carrying a Java class name prefix (`...ParseError: 1:7: ...`).
- An unknown `style` or `trailingCommas`, or a negative or fractional width or indent, now throws instead of silently falling back to a default.
100 changes: 51 additions & 49 deletions .claude/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Overview

`formatters` provides code formatters — Ruby, Java, C#, Swift, PHP and Rust —
`formatters` provides code formatters — Ruby, Java, Kotlin, C#, Swift, PHP and Rust —
under one hard constraint: **formatting must work with nothing installed but
Node.** No Ruby install, no gems, no JVM, no .NET SDK, no Swift toolchain, no PHP
or Composer, no Rust toolchain, no native binaries, no postinstall downloads. The approach follows the
Expand Down Expand Up @@ -48,8 +48,8 @@ guarded by a `peek()` that answers "is it booted" without one. One limit shapes
it, measured rather than assumed: a browser main thread refuses both
`WebAssembly.Module` and `WebAssembly.Instance` above 8MB, so booting in a
browser always uses async `WebAssembly.instantiate` and only trap recovery tries
the synchronous form. Node has no such limit, and Ruby uses that: its Node
source reads, decompresses and compiles synchronously, so its `formatSync`
the synchronous form. Node has no such limit, and Ruby and Kotlin use that: their
Node sources read, decompress and compile synchronously, so their `formatSync`
needs no `init()` at all.

`format.ts` and `boot-module.ts` are factories over an artifact source rather
Expand Down Expand Up @@ -125,7 +125,7 @@ each pipeline downloads a toolchain into a gitignored subdirectory and a glob
would pull whatever TypeScript those contain into the project.

Three JavaScript files are not ours to convert and must stay as they are:
`packages/csharp/runtime/*.js` and `packages/{java,kotlin}/*_fmt.runtime.mjs`
`packages/csharp/runtime/*.js` and `packages/java/java_fmt.runtime.mjs`
are generated by the .NET and TeaVM toolchains and shipped verbatim, and
`build/csharp_fmt/main.mjs` is the csproj's `WasmMainJSPath`, which the .NET SDK
requires to be JavaScript.
Expand Down Expand Up @@ -306,51 +306,53 @@ resolved at run time, the analysis cannot fold it, and the generic

### `@scalar/kotlin-fmt` (`packages/kotlin`)

Reference: **ktfmt**. Status: exact.

Runs actual ktfmt 0.64 with the Kotlin compiler's own PSI parser, compiled to
WasmGC by the same TeaVM fork the Java package uses. Built by
`build/java_fmt_teavm/kotlin-probe/build.sh`; the directory is named for the
feasibility probe it started as.

**The exactness claim is two measurements, not one,** because output could change
in two places. `gate2.sh` asks whether the one patched ktfmt file changes its
output on a JVM; `conformance.sh` asks whether compiling that to wasm changes it.
Same 589 files from kotlin-stdlib, kotlinx-coroutines and ktfmt itself, three
styles, **1767/1767 byte-identical each**, failures encoded the same on both
sides so a reworded diagnostic would count. Together they chain from stock ktfmt
to this module.

**ktfmt as shipped does not compile, and the reason is narrower than it looks.**
Pointing TeaVM at `Formatter.format` reports 94 missing members — StAX,
`ZipFile`, `Runtime.exec`, `ForkJoinPool`, AWT and Swing. That is
`KotlinCoreEnvironment`, the compiler's *CLI environment*, which resolves a
classpath and instantiates extension points named by string in
`META-INF/extensions/compiler.xml`. Parsing needs none of it: a bare
`CoreApplicationEnvironment` with `KotlinParserDefinition` registered drops the
frontier to 40 and takes all of that with it. `patches/ktfmt.patch` is that swap,
20 lines of one file, and `gate2.sh` is the evidence it changes nothing.

**TeaVM resolves reflection statically,** so `Class.forName`,
`getDeclaredField`, `getConstructor` and `Proxy` all need telling at build time.
`kotlin-probe/plugin/` is that: a build-time TeaVM extension, never compiled to
wasm. Scope its rules by the mechanism that needs them, not by package — a
reflectable member is a dependency root, so a package-wide rule made every class
in `com.intellij` reachable and failed to compile at all.

**One `Thread.yield()` cost a third of the module.** `AsyncMethodFinder`
propagates async-ness from every `@Async` method to its callers over a call graph
that keeps exception-construction and virtual `toString` edges, so one reachable
`Thread.yield()` in kotlinx-coroutines made 7,055 of 20,513 methods coroutines —
including the `stringToJs` the module initializer calls, whose coroutine prologue
dereferences a fiber that does not exist yet, so the module trapped at
instantiation. Cutting one root does nothing: removing `yield` moved all 7,055 to
`Thread.sleep`. The fork's single-threaded stand-ins leave none reachable.

**Licensing is unrestricted,** unlike the Java package. ktfmt reads Kotlin with
the Kotlin compiler's PSI and uses google-java-format only for its layout engine,
so javac is not in this module and there is no GPL component; everything embedded
is Apache-2.0.
Reference: **ktfmt** (0.65). Status: **exact by test**, because it is a
reimplementation.

One wasm artifact, `kotlin_fmt.wasm.br`: [ktrs](https://github.com/Hexay/ktrs),
ktfmt reimplemented in Rust with its own port of the Kotlin compiler's lexer and
parser, built by `build/kotlin_fmt/build.sh` from upstream at a pinned release
commit. `build/kotlin_fmt/entry` is a small crate over `ktrs-fmt` that exposes
plain exports; nothing in ktrs is patched.

**It is `wasm32-unknown-unknown` and imports nothing.** Formatting is a pure
function of a string, so there is no WASI and no shim: `src/kotlin-formatter.ts`
instantiates it with an empty import object. Options cross as integers, a
negative one meaning "the style's value", so ktrs's presets stay the only copy of
ktfmt's defaults. The module is 864 KB - under the 8 MB a browser main thread
allows synchronously - so instantiation is synchronous everywhere, and after a
trap the next call re-instantiates from the cached module.

**Traps are the engine's stack, mostly.** Wasm calls run on the engine's native
stack, so how deeply ktrs's recursive parser can nest is V8's or JSC's limit,
not the module's: a larger linker stack measured no deeper. It is still deeper
than ktfmt, whose JVM overflows at 2,000 nested parentheses. A panic aborts on
this target and traps the same way.

**Source is parsed as a script**, which is how ktfmt reads stdin and how the
TeaVM build parsed everything (ktfmt 0.64 always parsed `temp.kts`). A `.kt`
file would be held to a stricter grammar that rejects top-level statements.
The conformance test writes `.kts` files so the oracle parses the same way.

**The exactness rule applies in the form it takes for a reimplementation.**
`test/ktfmt-conformance.test.ts` is the evidence: it runs the real ktfmt
release jar, version `KTFMT_VERSION` in `build.sh` and checksum-pinned by
`build/kotlin_fmt/ktfmt.sh`, which CI runs, and asserts byte-identity and
identical diagnostics in every style, over its samples and any corpus named by
`KOTLIN_FMT_CONFORMANCE_CORPUS`. Over okhttp and kotlinx.coroutines (1,699
files) it matched every file in every configuration. Bump `KTRS_COMMIT` only
with a corpus run, and move `KTFMT_VERSION` to the release that ktrs version
tracks.

**It replaced ktfmt compiled by TeaVM**, which was exact by construction. That
build cost a Node 24.15 floor, an async-only boot, a 0.82 MB artifact, a patched
copy of ktfmt's `Parser` and patches to the TeaVM fork. It is in git history
as `build/java_fmt_teavm/kotlin-probe`. The fork patches it alone needed are
still in `build/teavm`, because removing one changes the Java module.

**Licensing is unrestricted.** ktrs is MIT OR Apache-2.0; the Kotlin compiler,
IntelliJ, ktfmt, google-java-format and ec4j code it ports is Apache-2.0, and
`licenses/NOTICE.md` carries those notices. No JDK code is in the module.

### `@scalar/csharp-fmt` (`packages/csharp`)

Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ body:
id: node-version
attributes:
label: Node version
description: Output of `node -v`. Java and Kotlin need 24.15 or newer.
description: Output of `node -v`. Java needs 24.15 or newer.
placeholder: v24.15.0
validations:
required: true
Expand Down
23 changes: 19 additions & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,9 @@ jobs:

# packages/ruby's conformance test is what backs the "exact by test" claim in the README, and
# it skips itself when there is no native RuboCop to compare against - so without these
# steps it never actually runs here. RuboCop installs in seconds, which makes Ruby the one
# reference toolchain cheap enough for CI to carry; the others need a .NET SDK, a Rust
# toolchain or a JDK and still skip.
# steps it never actually runs here. RuboCop installs in seconds, which makes Ruby one of the
# two reference tools cheap enough for CI to carry - ktfmt, below, is the other; the rest
# need a .NET SDK, a Rust toolchain or a locally built toolchain and still skip.
#
# The Ruby itself does not decide the output: RuboCop parses through the pinned prism gem,
# because the package targets Ruby 3.4. It is pinned anyway, so a run is reproducible.
Expand All @@ -106,6 +106,21 @@ jobs:
gem install "$gem" --no-document -v "$(sed -n "s/^gem \"$gem\", \"\(.*\)\"$/\1/p" build/rbcop_fmt/Gemfile)"
done

# packages/kotlin is ktrs, a reimplementation of ktfmt, so it is held to the real ktfmt the same
# way: test/ktfmt-conformance.test.ts skips without the jar. The jar is the pinned release,
# checked against a pinned checksum, and runs on the JDK the runner image already carries.
# Cached, because the jar is 71 MB and never changes for a given pin: the key is the script,
# which holds the checksum, and build.sh, which holds the version. The script still verifies
# the checksum of a restored jar.
- name: Cache the ktfmt jar
uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5.1.0
with:
path: build/kotlin_fmt/work
key: ktfmt-${{ hashFiles('build/kotlin_fmt/ktfmt.sh', 'build/kotlin_fmt/build.sh') }}

- name: Download the ktfmt that ktrs is held to
run: build/kotlin_fmt/ktfmt.sh

- name: Run unit tests
run: bun run test

Expand All @@ -118,7 +133,7 @@ jobs:
timeout-minutes: 20

# Both ends of the supported range, because each one catches what the other cannot. 24.15.0 is
# the floor the Java and Kotlin packages declare in `engines`, and running it here is the only
# the floor the Java package declares in `engines`, and running it here is the only
# thing that keeps that number honest. The newest release catches the reverse: a Node that
# stops running an artifact these packages promise it runs.
#
Expand Down
17 changes: 8 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,9 @@ build/teavm/toolchain/
build/java_fmt_teavm/toolchain/
build/java_fmt_teavm/target/
build/java_fmt_teavm/*.wasm
build/java_fmt_teavm/kotlin-probe/work/
build/java_fmt_teavm/kotlin-probe/target/
# Scratch build directories: run.sh and any hand-run comparison gives each TeaVM
# version its own, because a build that ends "with errors" leaves the previous
# classes.wasm in place. None of them is shipped - the artifact the package uses
# is packages/kotlin/kotlin_fmt.wasm.br, written by build.sh from target-ktfmt.
build/java_fmt_teavm/kotlin-probe/target-*/
build/java_fmt_teavm/kotlin-probe/async-dump*.txt
# Emitted next to every module for diagnosis; not evidence of anything, and the
# committed artifact is the module itself.
*.wasm-deobfuscator.wasm
build/java_fmt_teavm/kotlin-probe/build/
__pycache__/
# dotnet wasm build scratch (the SDK, the wasm-tools workload, the publish tree)
build/csharp_fmt/toolchain/
Expand Down Expand Up @@ -84,3 +75,11 @@ build/rust_fmt/*.wasm
build/rbcop_fmt/rbcop
build/rbcop_fmt/toolchain/
build/rbcop_fmt/*.wasm

# ktrs build scratch (the ktrs checkout, the uncompressed module, the ktfmt jar
# the conformance test compares against)
#
# `ktrs` has no trailing slash on purpose, matching rust_fmt above.
build/kotlin_fmt/ktrs
build/kotlin_fmt/work/
build/kotlin_fmt/*.wasm
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ Every package exports the same three things, from both builds:
| | |
|:---|:---|
| `format(source, options?)` | async; boots on demand. The one to reach for. |
| `formatSync(source, options?)` | synchronous; needs `await init()` first, and throws until it has - except Ruby under Node, whose boot is synchronous too. |
| `formatSync(source, options?)` | synchronous; needs `await init()` first, and throws until it has - except Ruby and Kotlin under Node, whose boot is synchronous too. |
| `init(options?)` | boots the module. Takes `{ url, bytes, encoding }` in the browser build only. |

Booting is the only asynchronous step there has ever been — fetching or reading
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ entry stays the default, and `src/index.browser.ts` is the only file that may no
touch `node:` built-ins — while `src/index.ts` is the only one that may.

Each package exports `format` (async), `formatSync` (needs `await init()` first,
except Ruby under Node, which boots synchronously) and `init`. Booting is the only asynchronous step, so the two formatters run the
except Ruby and Kotlin under Node, which boot synchronously) and `init`. Booting is the only asynchronous step, so the two formatters run the
same code and produce the same bytes.

## Exactness is a claim about a named tool
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ output against the real tool.

## Getting set up

You need [Bun](https://bun.sh) and Node 24.15 or newer — `bun run test:node` loads the Java and
Kotlin artifacts, and Node 24.0 through 24.14 will not compile them. The repo pins 26.7.0 in
You need [Bun](https://bun.sh) and Node 24.15 or newer — `bun run test:node` loads the Java
artifact, and Node 24.0 through 24.14 will not compile it. The repo pins 26.7.0 in
`.nvmrc` and installs that in CI, so `nvm use` gets you the Node everything is developed against;
24.15 is only the floor, and CI runs both ends. Nothing else — the wasm artifacts are committed,
so a fresh clone builds and tests without any language toolchain.
Expand Down
Loading