Repository navigation
Conversation
Add apollo-ios-codegen/rust, a Rust implementation of apollo-ios-cli that produces byte-identical output to the Swift + graphql-js implementation in apollo-ios-codegen/Sources, verified against the official CLI binary of every Apollo iOS release from 1.15.1 to 2.4.0 and against the CLI built from main. Crates (one per Swift module): graphql-compiler (apollo-compiler frontend converted to the Swift-equivalent model, graphql-js semantics reproduced explicitly), ir (IRBuilder, entities, selection trees, merged selections), template-string + template-string-macros (TemplateString and a template_string! proc macro with Swift's interpolation forms), apollo-codegen-lib (configuration, file discovery, templates, file generators), codegen-cli (generate, init, fetch-schema, generate-operation-manifest), apollo-ios-cli (executable) and utilities. Beyond the Swift CLI, and only when the --bazel-* flags are given: tree artifact output modes (schema_types, operations, test_mocks), exact per-file selection with --bazel-generate-for, scoped test mocks (scope, includeTypes, excludeTypes, baseModule, includeTypealiases) for base/feature mock module partitions, and a persistent Bazel worker (--persistent_worker, singleplex and multiplex) that shares the parsed schema across requests and caches compilation results. The worker protocol types are generated at build time from proto/worker_protocol.proto with protox, so no protoc is needed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tvpi5XEW1Kq76SKme8vnvo
Integration tests for the Rust crates: CLI behaviour (generate, init, manifests, validation errors, exit codes), Bazel modes and the persistent worker (every mode and selection byte-identical to plain generate on the repository's AnimalKingdom fixture), template rendering against the Swift test expectations (mock objects, operations, fragments), configuration round-trips and validation, pluralizer fidelity, the IR integration suite and the template-string semantics, plus the graphql-compiler fixtures. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tvpi5XEW1Kq76SKme8vnvo
Format the whole tree with rustfmt and make `cargo clippy --workspace --all-targets -- -D warnings` pass: std HashMap/HashSet replaced with IndexMap/IndexSet (the workspace denies them for deterministic output), unused imports, dead helpers and a few idiom lints fixed, type aliases for the complex callback types, and the Swift-style test names allowed explicitly. Internal planning-ticket references are dropped from comments. The worker benchmarks are timing-sensitive and now `#[ignore]`d; run them with `cargo test --release -p apollo-ios-cli -- --ignored bench_`. No generated output changes: the parity harness is still at 0 differing files against the Swift CLI. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tvpi5XEW1Kq76SKme8vnvo
…k defaults Port the test mock changes main has beyond 2.4.0 (apollographql#1066, apollographql#1068): - `options.requireNonOptionalMockFields` (default true). When false, every convenience-initializer parameter of a mock is optional and defaults to nil. Like Swift, the key is only encoded when it is false. - Default values for interface and union fields pick the first implementor that is actually generated: with `reduceGeneratedSchemaTypes` only referenced object types have a mock, and when none is left the parameter stays non-optional without a default. The implementor's name is rendered with the same typename rules as object fields. `MockObjectTemplate` therefore receives the IR's referenced types, as the Swift template does. The Bazel integration test no longer asserts an exact `Dog` field count, since the repository fixture gained `adoptionDate`. Verified byte-for-byte against the Swift CLI built from main (a04a5dc) on the upstream fixtures, the options matrix, KitchenSink, init, manifests, invalid inputs and the mock checks, including new rows with `requireNonOptionalMockFields: false` with and without `reduceGeneratedSchemaTypes`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tvpi5XEW1Kq76SKme8vnvo
Describe the crate layout, how to build and test, the Bazel flags and the persistent worker, and add scripts/parity-smoke.sh, which generates the AnimalKingdom fixture with the Swift and the Rust CLI from identical inputs and diffs the trees. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tvpi5XEW1Kq76SKme8vnvo
Add .github/workflows/rust-codegen.yml: rustfmt and clippy (-D warnings), release build and `cargo test --workspace` on macOS and Ubuntu, and a parity smoke job that builds the Swift CLI and the Rust CLI and diffs their AnimalKingdom output. It only runs for changes under apollo-ios-codegen. Existing workflows are untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Tvpi5XEW1Kq76SKme8vnvo
|
@erneestoc: Thank you for submitting a pull request! Before we can merge it, you'll need to sign the Apollo Contributor License Agreement here: https://contribute.apollographql.com/ |
✅ AI Style Review — No Changes DetectedNo MDX files were changed in this pull request. Review Log: View detailed log
|
👷 Deploy request for apollo-ios-docc pending review.Visit the deploys page to approve it
|
|
Thanks for sharing this @erneestoc! Sorry I haven't gotten to it yet. This is going to take some time to review and consider, but I can see this potentially being a huge win for performance and safety. I'd love to get away from bridging Javascript into Swift if we can cover everything with apollo-rs. |
Large apps with hundreds of operations and test mock targets spent most of their codegen time redoing the same work in every request: - GenerationFilter::matches read the current directory and normalized every filter file for each definition tested. A test mocks request tests every definition of the app, which cost ~190ms per request. The filter is now prepared once per request (GenerationFilter::matcher). - Every test mocks request built the IR of every operation to collect the mock fields. The worker now collects them once per compilation and shares the result between test mocks requests (collect_mock_fields / generate_test_mocks_from_collected_fields). - Every compilation converted the whole schema into a TypeRegistry (~25ms on GitHub's schema). It is now converted once per parsed schema and shared, except for configs with schema customizations, which write to the types. - Concurrent requests for the same compilation wait for one compile instead of each compiling, and compilations that served several requests are kept apart from one-off ones, so hundreds of operations targets no longer evict the app-wide compilation shared by test mocks and schema types. On a synthetic app (GitHub schema, 300 features, 612 codegen actions, cold worker, 16 cores) codegen went from 11.4s to 1.1s and an incremental build after editing one query from 13-20s to 1.1s. Generated files are byte-identical, with and without schema customizations.
TypeRegistry::from_schema copied the whole type map once per object, once per input object and once per interface implementing interfaces, and scanned every type once per interface to find its implementors. The cost grew with the square of the schema: on a ~500k-line schema (13x GitHub's) converting the types took ~3.5s, every time a schema is loaded (each one-shot CLI run, and a worker's first request). The per-iteration copies are replaced by moving the map in and out of the registry used for lookups, which sees exactly what the copy did (the current type is only inserted after its fields are resolved), and the implementors are collected in one pass, in the same order. One-shot operations action, median of 20 modules: 54 -> 39ms on GitHub's schema, 373 -> 91ms at 4x, 3416 -> 237ms at 13x. Schema types at 13x: 3710 -> 479ms. Generated files are byte-identical.
|
@AnthonyMDev completely agree and understand! Let me know if you have any questions. 🙂 |
What is this?
apollo-ios-cli(erneestoc/apollo-ios-dev), written with AI assistance and verified by an automated byte-for-byte harness against the official CLI binary of each Apollo iOS version. It produces the same files, with the same bytes, as the Swift + graphql-js implementation.apollo-compiler1.31) instead of graphql-js running in JavaScriptCore.I am not expecting this to be merged as-is. It is a large technology drift from the Swift + JavaScript implementation, it was built outside the project, and a port of this size needs more eyes than mine. I am opening it because the problems below are real for big codebases, the port exists and is verified, and I would like to find a form of collaboration that is useful to Apollo.
On maintainability: keeping the Rust port in sync with Swift/JS has turned out to be manageable. Every released Apollo iOS version from 1.15.1 through 2.4.0 (34 versions) is ported, and each one is verified byte-for-byte against the official
apollo-ios-clibinary of that version. The verification inputs are the upstream fixtures (the five SwiftScripts targets and theTests/TestCodeGenConfigurationsconfigs of the matching upstream tag), a version-gated options matrix (12 to 15 configs per version), a synthetic KitchenSink schema and operation set under 9 to 11 configs,init(18 cases), operation manifests (generateandgenerate-operation-manifest, persisted queries and legacy), 43 invalid-input cases, and test mocks (plain, scoped, and Bazel modes, compiled and run against the version's ownApolloTestSupport). The port has also been exercised on a large modular production codebase with millions of lines of generated Swift.https://github.com/erneestoc/apollo-ios-dev contains branches from versions 1.15.1 to 2.4.0, checked for byte identical output on a huge schema. (minor formatting drift only on 1.15.3 and 1.15.4)
Problems
These are the issues a large modular app runs into with the current codegen. None of them is a bug; they are consequences of the CLI's design assumptions.
ApolloCodegen.buildcreates aGraphQLJSFrontend, which creates a JavaScriptCore context and evaluates the bundled graphql-js frontend (ApolloCodegen.swift,GraphQLJSFrontend.swift). The schema, the compilation result and the validation state live inside that JS context and are bridged object by object. Keeping that context alive across requests, sharing it between threads, or caching a parsed schema independently of a compilation is not something the current architecture supports.output.testMocksis a single location, every object type referenced by any operation gets a mock, and a mock's fields are the union over all operations. A feature's test support module cannot own only its own mocks.ApolloAPI.Results
Measured on one machine: Apple M4 Max, 48 GB, macOS 26.2,
rustc 1.94.0,hyperfine 1.20.0, Swift toolchain 6.2.3. The Swift CLI is the official universalapollo-ios-clibinary from the apollographql/apollo-ios GitHub release of each version (arm64 slice). The Rust CLI is acargo build --releaseof the matching<ver>-rust-paritybranch. Both rungenerate --path <config>from identical copies of the inputs; the generated trees were diffed afterwards and are identical (0 differing files).Wall clock is the hyperfine median of 5 runs after 1 warmup (
hyperfine -N --warmup 1 --runs 5). Peak RSS is the best of 3 runs of/usr/bin/time -l(maximum resident set size).Fixtures: AnimalKingdom is the repository's
AnimalKingdomAPItarget (211-line schema, 16 operation files,schemaDocumentation, initializers and test mocks on; 58 generated files). GitHub API is the repository'sGitHubAPItarget (38,245-line schema, 3 operations,legacySafelistingCompatibleOperations; 232 generated files). KitchenSink is the harness's synthetic fixture (307-line schema, 10 operation files, 62 generated files).Persistent worker, measured through the real worker protocol (length-delimited
WorkRequest/WorkResponseon stdin/stdout, inputs with digests, 11 sequential--bazel-mode operations --bazel-generate-for <file>requests, one file per request, Rust 2.4.0):The numbers on 1.25.7 are the same within a millisecond. The in-process benchmark that ships with the port (
cargo test --release -p apollo-ios-cli bench_warm_vs_cold, AnimalKingdom, full configuration) reports cold 14.9 ms versus warm 8.0 ms. rules_apollo's own measurement on its example from a clean build: one multiplex worker served all 12 codegen requests and parsed the schema once.Caveats: the Swift CLI time includes JavaScriptCore startup and evaluation of the frontend bundle, which is a fixed cost that matters less on very large operation sets; this is one machine and one run per cell; the Rust CLI is a release build without profile-guided optimization; the repository fixtures are small compared to production schemas, so absolute times are small, but the ratio and the worker behavior are what matter for a build graph with hundreds of codegen actions. Commands and the driver script are in the appendix.
Architecture
flowchart LR subgraph inputs S[schema .graphqls] O[operations .graphql] end S --> AR[apollo-rs parse + validate] O --> AR AR --> AD["graphql-compiler adapter<br/>compilation result, graphql-js semantics"] AD --> IR["ir<br/>IRBuilder, entities, merged selections"] IR --> T["apollo-codegen-lib<br/>templates + file generators"] T --> F[generated .swift files, manifests] subgraph bazel [Bazel layer] W["apollo-ios-cli --persistent_worker<br/>schema cache, compilation cache"] --> T R["rules_apollo<br/>apollo_schema / apollo_operations / apollo_test_mocks"] --> W endCrates (
apollo-ios-codegen/rust, about 55k lines of Rust)graphql-compilerGraphQLCompiler(Swift) +apollo-codegen-frontend(TS)GraphQLSchema,GraphQLType,GraphQLValue,CompilationResult). No apollo-compiler type crosses this crate's boundary.irIRIRBuilder,Definition,Entity,EntitySelectionTree, direct and merged selections,ScopeDescriptor, inclusion conditions, field collector.template-string+template-string-macrosTemplateStringtemplate_string!proc macro with the same interpolation forms as Swift (if:,ifLet:,forEachIn:,list:,comment:,documentation:,section:).apollo-codegen-libApolloCodegenLibcodegen-cliCodegenCLIgenerate,init,fetch-schema,generate-operation-manifest, the Bazel flags.apollo-ios-cliutilitiesUtilitiesReproducing graphql-js semantics on apollo-rs
The output must match what Swift gets from graphql-js, not what a clean GraphQL implementation would do. The adapter reproduces those behaviors explicitly, each one with a comment pointing at the graphql-js or TS source it mirrors:
compileToIRencounters types (addReferencedType: object adds its interfaces, interface adds its implementors, union its members; introspection types excluded;@typePolicyimplementors handled as the TS compiler does).__typenamestripped and prepended per selection set, operation root excluded) and the__typenameinjection rules by parent node kind, followed by a graphql-js-compatible printer (variable-definition wrapping as in graphql-js 17 for 2.1.0+, block form only for multi-line items). This is what the operation source, the operation ids and the manifests are computed from.__typenameselections filtered before the IR sees them;referenced_fragmentslisting direct spreads only, not the transitive closure apollo-rs computes.Numbersemantics for Int overflow,Double.descriptionfloat printing, multi-line string indentation in arguments, nested input-object defaults resolved by name.GLOB_NOSORT, so operations come in rawreaddirorder per directory. The port walks directories the same way, follows symlinks (Bazel presents inputs as symlinks) and checks literal paths directly.validation.rs) mirrorsGraphQLJSFrontend.validateDocument: the merged document is validated with graphql-js'sspecifiedRulesminusNoUnusedFragmentsRule, plus the Apollo rules fromvalidationRules.ts(no anonymous operations, no__typenamealias, deferred inline fragments need a type condition and a label, disallowed field and input parameter names, schema-namespace conflicts). Errors are rendered as<path>:<line>:error:<message>with graphql-js wording, exit 1, never a panic.IR and templates
The IR is a direct port:
IRBuilderandRootFieldBuilder,DirectSelectionsandMergedSelections, theEntitySelectionTreethat memoizes selections per entity across type scopes,ScopeDescriptorandTypeScope,InclusionConditions, deferred fragments, and the field collector that the test mocks read. Async Swift code became synchronous Rust.The templates are ported one file at a time on top of
template_string!, which keeps Swift's semantics: list interpolation with separators and terminators, indentation taken from the column of the interpolation, removal of lines that an empty interpolation leaves behind, comment and documentation prefixes with no trailing space on empty lines. Several parity bugs were in exactly these rules (forEachInblank-line handling, deprecation-message escaping, mock annotation indentation), which is why the templates are not rewritten in a different style.Per-version branches
One branch and one GitHub release per upstream version:
<ver>-rust-parityfor 1.15.1 through 2.4.0. Each branch carries that version's template semantics with no runtime switches, so a release is a drop-in replacement for the official CLI of that exact version (--versionprints the Apollo version;CODEGEN_VERSIONis a constant per branch). Every branch is the previous one plus the version-specific commits. A fix that applies everywhere is made on the lowest affected branch and cherry-picked forward through the chain, with the harness run at every step.Bringing a new upstream version over: branch from the previous head, bump
CODEGEN_VERSION, run the harness against that version's official binary, and port the template and frontend diffs between the two upstream tags until the harness reports 0 differing files. Version drift found this way so far includes deferred-fragment metadata placement changes across 1.16 to 2.0,SchemaMetadatarendered as a dictionary from 1.25.4,@_spi(Execution)onobjectTypefrom 2.0,@testable importin mocks from 1.24.0,nonisolatedfrom 2.1.0, and capitalization rules from 2.3.0.Bazel mode
All of this is additive: without
--bazel-output-dirthe CLI behaves exactly as upstream.--bazel-output-dir <dir>: write a tree artifact instead of the configured paths; the config's output section only selects the module layout. Pruning is off in this mode.--bazel-mode schema_types | operations | test_mocks: schema types (plus mocks underTestMocks/when configured), operations and fragments only, or mocks only.--bazel-generate-for <file>(repeatable) generates exactly the definitions in those files;--bazel-framework-path <prefix>is the older prefix form. The schema module still sees every operation, so referenced types are right.--bazel-keep-schema-configuration: keepSchemaConfiguration.swiftandCustomScalars/in the artifact when the rule does not provide them.--bazel-strip-import <module>and--bazel-optimize-schema-metadata(a dictionary lookup inSchemaMetadata.objectType(forTypename:), driven by the configured namespace).scope(all|referencedByOperations),includeTypes,excludeTypes,baseModule,includeTypealiasesinsideoutput.testMocks.absolute|swiftPackage, and the matching--bazel-mocks-*flags. A partition is a base module (shared types and theMockObject+Interfaces/MockObject+Unionstypealias files) plus per-feature modules that import it; every type is generated exactly once and the union of the partition equals the unscoped output file for file. Unscoped output is unchanged.Worker
apollo-ios-cli --persistent_workerimplements the Bazel worker protocol, singleplex (request_id == 0, inline, in order) and multiplex (non-zero ids, one thread per request, responses in completion order, stdout writes serialized). The parsed schema is shared across requests as anArc<CompiledSchema>keyed on the schema file paths and Bazel digests, so one multiplex worker parses each schema once even though every operations target has its own config. The compilation result is cached per config digest plus schema and operation digests (8 entries). Validation and I/O errors come back as aWorkResponsewithexit_code = 1, and a panic inside a request is caught and reported the same way, so a bad operation never takes the worker down.WorkRequest.sandbox_diris not implemented; with--worker_sandboxingBazel falls back to sandboxed singleplex workers.The harness as the compatibility contract
parity.sh <ver>downloads the officialapollo-ios-cliof that version, exports the inputs from the upstream tag, runs both CLIs from identical copies anddiff -rs whole trees, then runsinit, the manifests, the invalid-input set (exit status must match, Rust must not panic) and the mock checks (Swift and Rust mocks compiled side by side with XCTest asserting identical behavior, plus a base/feature partition built and tested against the version's own runtime). The summary line isdiffering-files=N failed-runs=M, and every release is cut at 0/0. Upstream's committed generated Swift is deliberately not the reference, because several tags changed templates without regenerating; the official binary is.Compatibility and verification
differing-files=0 failed-runs=0on the full harness as of 2026-10-05 (33 versions in the last chained run log, 2.4.0 re-verified for this document: 0 differing files on both copies above).cargo test --workspaceis green on every branch; 1448 passed, 0 failed, 20 ignored on 2.4.0 (release profile, run for this document), 1436 to 1448 across branches.Build Rust CLIworkflow buildsaarch64-apple-darwin,x86_64-unknown-linux-gnuandaarch64-unknown-linux-gnuon every release and uploads tarballs with sha256 sums. 35 releases exist (34 upstream versions plus a fork-only 1.15.4 that has no upstream counterpart).x86_64-apple-darwinbuild (by decision; Intel Macs build from source); the port never emits theSpecified by:scalar documentation line because Swift never does either (it readsspecifiedByUrlfrom a graphql-js value that exposesspecifiedByURL); the harness avoids one input that crashes every Swift CLI 1.15.1 to 2.4.0 (a custom-scalar variable with a list or object literal default), so that case is not compared; Swift traps on...Fragment @deferwithout a label, the Rust CLI reports a validation error.fetch-schemais ported but not part of the byte-for-byte harness;worker_bench::bench_warm_vs_coldis timing-sensitive and occasionally flaky.Proposal
apollo-ios-codegen-rust), with the parity harness in CI as the contract: every tag of apollo-ios gets a matching branch and release that is verified byte-for-byte against the official binary before it is published. Apollo would own the namespace and the review bar; An AI pipeline could do the porting work per release and keep the test harness green. This is the shape that gives us users a supported path to Bazel and persistent workers without committing Apollo to replace its implementation.