Skip to content

feat(js): service-scoped type resolution + generic substitution on TypeResolver#1323

Merged
ukint-vs merged 11 commits intomasterfrom
feat/sails-js-type-resolver-scope
May 7, 2026
Merged

feat(js): service-scoped type resolution + generic substitution on TypeResolver#1323
ukint-vs merged 11 commits intomasterfrom
feat/sails-js-type-resolver-scope

Conversation

@ukint-vs
Copy link
Copy Markdown
Member

@ukint-vs ukint-vs commented Apr 23, 2026

Closes #1317.

Rework (2026-05-06) — addresses vobradovich's CHANGES_REQUESTED review ("violates self-sufficient service IDL concept"). Aligns this PR with the contract being formalized in #1341: a service IDL is independently distributable; service signatures resolve types from the service's own types plus its extends chain, never from program.types ambient scope. Internal-only API change — TypeResolver and the new resolveNamed/resolveGenerics/substituteGenerics/resolveInService surface stays the same. See the latest commit for details.

Summary

Adds three public methods on TypeResolver and a convenience accessor on SailsProgram so downstream consumers (wallets, explorers, schema tooling, custom codegen) stop re-implementing type-scope and generic-substitution out-of-band.

New API

class TypeResolver {
  // Constructor: a single pre-merged Type[]; later entries override earlier ones
  // on name collision (last-write-wins).
  constructor(types: Type[]);

  // Look up a named user Type from this resolver's scope.
  // With concrete generics, returns a substituted concrete Type with type_params stripped.
  resolveNamed(type: TypeDecl): Type | undefined;
  resolveNamed(name: string, generics?: TypeDecl[]): Type | undefined;

  // Recursively substitute type parameters through a TypeDecl tree.
  // Pure, idempotent, same-ref return on unchanged subtrees.
  // Throws on cyclic substitution maps ({T: T} or {T: U, U: T}).
  substituteGenerics(typeDecl: TypeDecl, substitutions?: Record<string, TypeDecl>): TypeDecl;

  // Zip user type's type_params with a concrete generics list.
  resolveGenerics(userType: Type, generics?: TypeDecl[]): Record<string, TypeDecl>;
}

class SailsProgram {
  // Direct AST lookup scoped to a named service. Resolves from the service's own
  // `types` plus the types of every service it extends, transitively. Program-level
  // `types` are NOT in scope (they belong to program/ctor declarations).
  // Backed by a lazy O(1) Map index — safe to call in tight walker loops.
  resolveInService(serviceName: string, typeDecl: TypeDecl): Type | undefined;
}

Per-service TypeResolver instances seeded by SailsProgram.services see only the service's own types plus the transitive extends chain. The chain walk is depth-first with cycle detection; on a pathological A extends B, B extends A graph the constructor throws with the chain in the message. Service-local types shadow base-service types on name collision.

getTypeDeclString was also updated to call the renamed resolveGenerics helper instead of the inline copy at lines 229-233.

Why

Two correctness bugs in consumer code before this change:

  1. Name collisions across services. Two services each declaring Packet with different shapes silently collided when a consumer flattened types into a single map. No public API narrowed a lookup to a single service's scope.
  2. Generic substitution. Envelope<[u8]>'s payload field came through as a bare {kind:'generic', name:'T'} leaf, causing walkers to silently pass hex strings through untouched or SCALE-encode with wrong bytes. The substitution algorithm already lived inside getTypeDeclString — this PR factors it out and exposes it.

vara-wallet had ~180 lines of workaround (coerceHexToBytesV2, getRegistryTypes, ad-hoc substitution recursion) that can now collapse to a few method calls.

Tests

24+ tests across the resolver + parser-resolver suites.

Unit (js/test/idl-v2-type-resolver.test.ts):

  • substituteGenerics: bare-param substitution, recursion through slice/array/tuple/named-with-generics, pass-through for unknown names, idempotence, non-mutation, chain resolution (T → U → u32), self-referential throw, cyclic chain throw, unknown-kind throw.
  • resolveNamed: returns user Type for named decl (raw and concrete-generics paths), returns undefined for primitives/slices/arrays/tuples/unknown names/bare type_params.
  • resolveGenerics: zip, no type_params, extra generics truncated.
  • Last-write-wins merge: locals override on name collision; registry reflects the last-declared shape.
  • Cycle guard (rework): self-cycle (A extends A) and mutual cycle (A extends B, B extends A) throw with the chain reported; diamond inheritance does not throw.

Integration (js/test/idl-v2-parser-type-resolver.test.ts):

  • Two services each declaring Packet with different shapes → resolveInService('A', ...) vs 'B' return their own definitions.
  • Unknown service name → undefined.
  • Self-sufficient contract (rework): a service resolver does NOT see program-level types — resolveInService returns undefined and service.registry.hasType('Shared') is false. Mirrors the JS test added in chore: Clarify service-local IDL type scopes #1341.
  • Extends-chain visibility (rework): an extender sees its base service's types through the transitive merge, both via program.resolveInService('Child', ...) and via child.typeResolver.resolveNamed(...).
  • Sibling isolation (rework): two services with same-named Packet types resolved through SailsProgram.resolveInService return distinct shapes — no cross-service bleed.
  • Envelope<[u8]> → walker feeds struct fields through substituteGenerics + resolveGenerics to resolve payload from T to [u8].

Notes for reviewers

  • substituteGenerics has a runtime cycle guard (visited set, throws with the full chain on detection) rather than just the JSDoc warning. Maps produced by resolveGenerics from parsed IDL can't produce cycles, but hand-merged or wire-sourced maps can.
  • SailsProgram.resolveInService walks the AST directly (rather than going through this.services, which rebuilds SailsService + a fresh TypeRegistry on every access), and is backed by a lazy Map<serviceName, Map<typeName, Type>> that pre-merges each service's transitive extends scope. O(1) per call after the first invocation.
  • substituteGenerics returns the same reference for subtrees that contained no substitutions — walkers traversing large trees with sparse type parameters don't allocate new nodes unnecessarily.
  • The extends-chain merge in _collectServiceScopeTypes uses the same visited-set + immutable-copy cycle-guard pattern as _substituteGenerics, so a throw mid-recursion leaves no poisoned shared state.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces service-scoped type resolution and generic type substitution for the Sails IDL v2 implementation. Key enhancements include the addition of resolveInService to SailsProgram for scoped type lookups and the introduction of ambientTypes in SailsService to support program-level types. The TypeResolver has been significantly updated with logic for shadowing ambient types, recursive generic substitution with cycle detection, and helper methods for mapping type parameters. Feedback was provided regarding the performance of resolveInService, suggesting the use of Maps for $O(1)$ lookups to prevent bottlenecks in large IDLs.

Comment thread js/src/sails-idl-v2.ts
ukint-vs added a commit that referenced this pull request Apr 23, 2026
- Build a lazy `Map<serviceName, Map<typeName, Type>>` index in
  `SailsProgram` on first `resolveInService` call. O(1) per call after
  that, no cost for consumers who never use the method. Addresses the
  gemini-code-assist comment on sails-idl-v2.ts:242.
- Swap `JSON.parse(JSON.stringify(input))` in the "does not mutate"
  test for `structuredClone(input)` to satisfy `unicorn/prefer-structured-clone`
  (fixes the lint job failure on the v1.6.1.0 JS CI).
…peResolver

Closes #1317.

Adds three public methods on `TypeResolver` and a convenience accessor on
`SailsProgram` so downstream consumers (wallets, explorers, custom codegen)
stop re-implementing type-scope + generic-substitution out-of-band.

- `TypeResolver.resolveNamed(typeDecl)` — look up a named user `Type` from
  the resolver's scope, honoring ambient-vs-local shadowing.
- `TypeResolver.substituteGenerics(typeDecl, substitutions)` — pure,
  idempotent tree walk that substitutes bare `{kind:'named', name:'T'}`
  leaves through chains (`{T: U, U: u32}` resolves `T` to `u32`). Runtime
  cycle guard on self-referential or cyclic maps. Same-reference return
  on unchanged subtrees avoids pointless allocation in walkers.
- `TypeResolver.genericsSubstitutions(userType, generics)` — zip a user
  type's `type_params` with a concrete generics list into a sub map.
  `getTypeDeclString` now uses this helper instead of an inline copy.
- `TypeResolver` constructor gained an `ambientTypes` parameter;
  `SailsProgram` threads `program.types` into every `SailsService`
  (including the `extends` chain) so service-local types shadow
  program-level ambients inside each per-service resolver.
- `SailsProgram.resolveInService(serviceName, typeDecl)` — direct AST
  lookup that skips `SailsService`/`TypeResolver` construction, safe to
  call in tight walker loops.

Tests: 21 new (17 unit + 4 parser-driven integration) covering all five
`TypeDecl` variants, shadowing with registry-level collision, chain
resolution, cycle detection, unknown-kind rejection, and service-scoped
collision across two services declaring the same type name with
incompatible shapes.
- Build a lazy `Map<serviceName, Map<typeName, Type>>` index in
  `SailsProgram` on first `resolveInService` call. O(1) per call after
  that, no cost for consumers who never use the method. Addresses the
  gemini-code-assist comment on sails-idl-v2.ts:242.
- Swap `JSON.parse(JSON.stringify(input))` in the "does not mutate"
  test for `structuredClone(input)` to satisfy `unicorn/prefer-structured-clone`
  (fixes the lint job failure on the v1.6.1.0 JS CI).
…cl::Generic

Per maintainer feedback on #1323:

- `TypeResolver` constructor reverts to `(types: Type[])`. Callers that want
  ambient-vs-local shadowing pass `[...ambientTypes, ...localTypes]` — the
  merge is the caller's concern, not the resolver's. Moved the merge to
  `SailsService`, which is the only caller that needs it.
- `substituteGenerics` and `resolveNamed` now target the explicit
  `TypeDecl::Generic` AST variant (landed in #1314). A bare `{kind:'named'}`
  decl is always a concrete user-type reference and is no longer treated as
  a type-parameter fallback; substitution only fires on `{kind:'generic'}`
  leaves. Cycle detection, chain resolution, and same-reference returns on
  unchanged subtrees all unchanged.
- Tests updated to use `{kind:'generic', name:'T'}` for type parameters.
  Ambient-types describe block renamed to "last-write-wins merge (shadowing
  via call-site merge)" to reflect the new API shape.
@ukint-vs ukint-vs force-pushed the feat/sails-js-type-resolver-scope branch from 8155c74 to dd74a92 Compare April 23, 2026 14:32
@ukint-vs
Copy link
Copy Markdown
Member Author

Rebased on master and addressed maintainer feedback:

  1. TypeResolver constructor reverted to (types: Type[]). [...ambientTypes, ...localTypes] merging now happens at the call site in SailsService. The resolver no longer knows about the ambient-vs-local distinction; callers pass a single pre-merged array and rely on last-write-wins semantics for shadowing.
  2. Adapted to TypeDecl::Generic (from feat: Migrate type registry to IDL AST, add explicit generic type declarations #1314). substituteGenerics now targets the explicit {kind:'generic'} variant for type-parameter leaves; a bare {kind:'named'} decl is always a concrete user-type reference and is no longer used as a type-param fallback. resolveNamed returns undefined for kind:'generic'. Tests updated accordingly.

Cycle detection, chain resolution (T → U → u32), and same-reference returns on unchanged subtrees are preserved. 64/64 offline tests + lint green locally.

Graph review surfaced SailsService.extends as a test gap — it propagates
ambient types to child services at sails-idl-v2.ts:612, but no offline
test exercised that path. Adds a parser-driven test: Child extends Base,
program declares Shared, verify baseThroughExtends.typeResolver resolves
Shared through the propagated ambient types.
Comment thread js/src/type-resolver-idl-v2.ts
Comment thread js/src/type-resolver-idl-v2.ts Outdated
Comment thread js/src/type-resolver-idl-v2.ts Outdated
…solveGenerics

- Rename `substituteGenerics` → `resolveGenerics`.
- Hide `genericsSubstitutions` (now `_genericsSubstitutions`). Consumers no longer
  manually zip `type_params` with a `TypeDecl.generics` list — that's resolver logic.
- Extend `resolveNamed` with a second overload `(name: string, generics?: TypeDecl[])`.
  When concrete generics are provided (via either overload or via `typeDecl.generics`),
  the result is a concrete substituted `Type` with `type_params` stripped:
    `Envelope<u32>` → `{ kind: 'struct', fields: [{id, u32}, {payload, u32}] }`.
  Without generics, the raw user `Type` is returned unchanged.

Per vobradovich feedback on #1323.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vobradovich
Copy link
Copy Markdown
Member

Design rationale: self-sufficient service IDL

The ambientTypes plumbing in TypeResolver / SailsService reflects a stricter scoping rule than what the parser permits today as an authoring convenience.
Recording the contract here so the runtime, the codegen, and idl-gen stay consistent over time.

The rule

A service IDL is self-sufficient and independently distributable. A <service>.idl parses, validates, and resolves all of its functions and
events signatures using only its own types {} block (plus primitives and Option / Result / NonZero*).

Cross-scope type references are restricted to:

From → To Mechanism Status
Service → its base service's types extends Base@<interface_id> (types merged into the extender) implemented
Program ctor → service type Service::Type qualified syntax TBD per docs/idl-v2-spec.md lines 419–420
Service → sibling service type forbidden enforced by rs/idl-parser-v2/src/post_process.rs::Validator
Service → program type, unqualified soft-allowed by parser as authoring convenience, NOT a contract see below

The parser pushes program types into the root scope before validating each service (post_process.rs:32-44). That's a DRY affordance for hand-written multi-service IDLs in a single file.
Generators and the runtime must not rely on it for distributability. A type used by a service S must be present in S.types for S.idl to be parseable on its own.

program.types should not be treated as an ambient scope for every service. The service IDL boundary needs to stay self-contained: a service’s functions/events/throws should be resolvable from that service’s own types plus any explicitly extended service interfaces, not from the enclosing program. extends is a service-to-service dependency keyed by interface_id, so it can be distributed independently from a concrete program. The namespace TBD in the spec is better interpreted in the opposite direction: program/root declarations may eventually reference service-local types via Service::Type, but services should not implicitly see program.types. Otherwise the same service IDL becomes decodable only when bundled with a particular program IDL, which breaks independent service distribution.

@vobradovich vobradovich self-requested a review May 6, 2026 09:20
Copy link
Copy Markdown
Member

@vobradovich vobradovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

violates self-sufficient service IDL concept

ukint-vs and others added 2 commits May 6, 2026 19:47
Aligns with the self-sufficient service IDL contract being formalized
in #1341: a service IDL is independently distributable and resolves its
function/event/throws payload types from its own `types` block plus the
types of every service it explicitly extends, transitively. Program-level
`types` are scoped to program/ctor declarations and are no longer threaded
into per-service resolvers.

- SailsService: remove `ambientTypes` constructor param + `_ambientTypes`
  field. Resolver is now seeded by `_collectServiceScopeTypes`, a module-
  level helper that walks the `extends` chain depth-first with cycle
  detection (mirrors `_substituteGenerics`'s visited-set pattern).
- SailsProgram: stop passing `program.types` into services. Rewrite
  `resolveInService` to return only hits from the merged service-local +
  extends-chain scope. Drop `_programTypeIndex`; the lazy index is now
  per-service `Map<typeName, Type>` with extends pre-merged.
- Tests: flip the two ambient-themed assertions (now require
  `resolveInService(...)` and `registry.hasType(...)` to fail for a
  program-only type, matching #1341's parallel JS tests). Rewrite the
  extends-chain test to assert base-service types instead. Add a sibling-
  bleed test. Reframe the call-site shadowing block as a generic
  last-write-wins test.
- TypeResolver constructor JSDoc: drop the ambient/local framing in
  favour of a generic last-write-wins description.

Closes #1317.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add SailsService extends-chain cycle-guard tests: self-cycle (`A → A`)
  and mutual cycle (`A → B → A`) throw with the chain reported; diamond
  inheritance (A → [B, C], B → C) does not throw.
- Trim two over-explained inline comments in `_collectServiceScopeTypes`
  call sites flagged by /simplify quality review.
- Make the extends-chain test fixture parser-agnostic: Base now exposes
  `Ping() -> u32` instead of `GetShared() -> Shared`. Keeps Base.types
  populated as the merge target without referencing extender-resolved
  types, so the IDL parses cleanly under both current master and the
  tightened validator in #1341.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ukint-vs ukint-vs requested a review from vobradovich May 6, 2026 16:09
ukint-vs and others added 2 commits May 6, 2026 20:10
CI lint flagged 4 errors from unicorn:
- consistent-function-scoping: ident/unit helpers were inside the
  describe block; lifted them to module scope as mockIdent/mockUnit.
- no-nested-ternary: replaced the inline `i.name === 'A' ? a : ...`
  ternary chains in the lookup closures with a tiny `lookupFromMap`
  helper that takes a `Record<string, IServiceUnit>`. Each test now
  builds its name → unit map explicitly.

Tests are otherwise unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two tests broke after the simplify-pass cleanups landed:

1. Cycle-guard "diamond" test — `SailsService.constructor` calls
   `_getEvents` which unconditionally invokes `InterfaceId.from(service.interface_id)`
   at line 585 (before iterating events). The mock units left
   `interface_id` undefined, so `InterfaceId.fromBytes(undefined)` threw
   TypeError. Fixed by giving mocks an 8-byte zero placeholder
   (`ZERO_IFACE`) and explicit empty `funcs`/`events` arrays. Self-cycle
   and mutual-cycle tests were unaffected because `_collectServiceScopeTypes`
   throws inside the resolver init, before `_getEvents` runs.

2. Extends-chain integration test — fixture pinned `Base@0xb45ddc41cf66e2da`
   but the parser computes interface_ids from interface contents (per
   `docs/interface-id-spec.md`), and the prior fixture rewrite
   (`GetShared() -> Shared` → `Ping() -> u32`) changed the content hash.
   Following the convention used by #1341's added tests, drop explicit
   `@0x...` annotations and let the parser compute them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts:
#	js/test/idl-v2-parser-type-resolver.test.ts
@ukint-vs ukint-vs merged commit ee8c558 into master May 7, 2026
2 checks passed
@ukint-vs ukint-vs deleted the feat/sails-js-type-resolver-scope branch May 7, 2026 11:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sails-js v2: TypeResolver needs service-scoped lookup + generic substitution for downstream walkers

2 participants