Skip to content

[Issue #798] Transforms PoC: TypeScript#825

Draft
SnowboardTechie wants to merge 1 commit into
HOLD-transformsfrom
bryan/issue-798-transforms-poc-typescript
Draft

[Issue #798] Transforms PoC: TypeScript#825
SnowboardTechie wants to merge 1 commit into
HOLD-transformsfrom
bryan/issue-798-transforms-poc-typescript

Conversation

@SnowboardTechie
Copy link
Copy Markdown
Collaborator

@SnowboardTechie SnowboardTechie commented May 12, 2026

Summary

TypeScript proof-of-concept for the plugin transformation interface under @common-grants/sdk/extensions, mirroring the Python PoC (#810) so the ADR-0022 / ADR-0017 contract is validated in both SDKs. Ships buildTransforms(), TransformResult<T>, PluginError, the mapping runtime, a definePlugin() extension for meta + transformSchemas, and a runnable round-trip example.

Changes proposed

New public surface under @common-grants/sdk/extensions:

  • buildTransforms() — compiles a pair of ADR-0017 mapping objects into typed (toCommon, fromCommon) callables with call-time structural validation. Optional commonModel Zod schema turns parse failures into PluginError[] instead of thrown exceptions.
  • TransformResult<T> — unconditional { result, errors } return shape (ADR-0022 Decision ADR - Specification framework #7).
  • PluginError — structured error carrying path, handler, sourceValue, cause (ADR-0022 Decision ADR - Website hosting #9). Docstring documents the PII surface on both sourceValue (full input record) and message (data-bearing on the Zod-validation path).
  • transformFromMapping(), getFromPath(), DEFAULT_HANDLERS — mapping runtime, six built-in handlers (const, field, match, switch alias, numberToString, stringToNumber).
  • definePlugin() accepts optional meta: PluginMeta and transformSchemas: Partial<Record<ExtensibleSchemaName, ObjectSchemasInput>>. Existing callers passing only extensions are unaffected.
  • New supporting types: Handler, ObjectSchemasInput, ObjectSchemas, PluginMeta, PluginCapability, ObjectMappings, PluginExtensionsObjectConfig, PluginExtensions, TransformSchemasInput.

Security guards (mapping JSON may be reconstituted from untrusted sources via mergeExtensions(), per ADR-0022 Decision #8):

  • __proto__ rejected as an output field name at buildTransforms() call time (validateMapping()) and at walk time (transformFromMapping()).
  • Custom handler names that collide with DEFAULT_HANDLERS or shadow Object.prototype keys (constructor, toString, etc.) rejected at call time.
  • stringToNumber's thrown message does not embed the source value (it would otherwise flow into PluginError.message).

Out of scope (matches Python PoC, deferred to full SDK):

Files:

  • New: src/extensions/transforms.ts, src/extensions/transformation.ts, examples/transforms.ts, __tests__/extensions/transforms.spec.ts, __tests__/extensions/transformation.spec.ts, .changeset/transforms-poc-typescript.md
  • Modified: src/extensions/types.ts (10 new transform types), src/extensions/define-plugin.ts (meta + transformSchemas), src/extensions/index.ts (exports trimmed to match Python's extensions/__init__.py surface), src/extensions/README.md (new "Plugin transformations" section + API reference table), __tests__/extensions/define-plugin.spec.ts (five new tests), package.json (example:transforms script).

Context for reviewers

How verified:

  • pnpm run checks — eslint, prettier, tsc --noEmit clean.
  • pnpm run test — 427 tests across 24 suites. PoC adds 30 handler-runtime tests, 12 buildTransforms tests (both directions covered for handler errors), 5 new definePlugin tests for meta + transformSchemas.
  • pnpm run example:transforms — synthetic grants.gov record round-trips through toCommon → Zod-validated CommonGrants OpportunityfromCommon, with custom join / split handlers and verified opportunity_number recovery.

Conform-before-extend. ADR-0022's TypeScript code blocks and the Python PoC at 799-transform-poc-fetch were treated as the spec, transliterated snake_case → camelCase, swapped Pydantic → Zod, and reused existing TS SDK type patterns (const generics, mapped types). Two intentional divergences from the ADR's TS shape are documented in code comments:

  • DefinePluginOptions.transformSchemas (not schemas) mirrors Python's transform_schemas workaround for the collision with the existing Plugin.schemas field. Full SDK target: resolve in [TS SDK] Extend definePlugin() to accept schemas with toCommon/fromCommon #756.
  • BuildTransformsOptions.commonModel: z.ZodType<TCommon, z.ZodTypeDef, any> uses any in the contravariant input position to accept schemas with input/output asymmetry (e.g. .transform() producing Date from string). The ADR's ZodType<TCommon> would reject the SDK's own OpportunityBaseSchema.

Why HOLD-transforms. Issue #798 sits in the transforms feature bucket (#656, #745, #756, #757, #768, #798, #799, #813) per the branching strategy doc. The bucket batches to main at the C2 (July 21) checkpoint. PR #810 already targets HOLD-transforms for the Python PoC.

Cross-SDK design findings for the full SDK work in #756 / #757 (10 items, including the schemas naming collision, commonModel-must-be-extended-schema gotcha, plain-object normalization, lack of a TS code generator for an extended common model, Zod-strip interaction with derived fields, and the any widening on commonModel) live in the issue-work state dir, not the repo.

Additional information

Example output (from pnpm example:transforms):

=== toCommon (native → CommonGrants) ===
{
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "title": "Research into conservation techniques",
  "status": { "value": "open" },
  "customFields": {
    "legacyId":      { "name": "legacyId",      "fieldType": "integer", "value": 12345 },
    "agencyName":    { "name": "agencyName",    "fieldType": "string",  "value": "Department of Examples" },
    "applicantTypes": { "name": "applicantTypes", "fieldType": "array",  "value": ["state_governments"] },
    "compositeLabel": { "name": "compositeLabel", "fieldType": "string", "value": "ABC-123-XYZ-001 — Research into conservation techniques" }
  },
  ...
}

=== fromCommon (CommonGrants → native) ===
{
  "data": {
    "opportunity_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "opportunity_title": "Research into conservation techniques",
    "opportunity_number": "ABC-123-XYZ-001",  // recovered from compositeLabel via split handler
    ...
  }
}

✓ round-trip verified for fields covered by both mappings

@github-actions github-actions Bot added dependencies Pull requests that update a dependency file sdk Issue or PR related to our SDKs typescript Issue or PR related to TypeScript tooling ts-sdk Related to TypeScript SDK labels May 12, 2026
@SnowboardTechie SnowboardTechie added the enhancement New feature or request label May 12, 2026
Port the Python transforms PoC (PR #810, branch 799-transform-poc-fetch)
to @common-grants/sdk so the ADR-0022 / ADR-0017 contract is validated
in both SDKs before either is locked in for full implementation.

Public additions under @common-grants/sdk/extensions:

- buildTransforms() — compile a pair of ADR-0017 mapping objects into typed
  (toCommon, fromCommon) callables with call-time structural validation.
  Optional commonModel Zod schema turns parse failures into PluginError[]
  rather than thrown exceptions.
- TransformResult<T> — unconditional { result, errors } return shape
  (ADR-0022 Decision #7).
- PluginError — structured error class with path / handler / sourceValue /
  cause (ADR-0022 Decision #9). Docstring documents the PII surface on both
  sourceValue (carries the full input record) and message (data-bearing on
  the Zod-validation path because Zod's default error map embeds received
  values).
- transformFromMapping(), getFromPath(), DEFAULT_HANDLERS — mapping
  runtime; six built-in handlers (const, field, match, switch alias,
  numberToString, stringToNumber).
- definePlugin() accepts optional meta and transformSchemas. Existing
  callers passing only `extensions` are unaffected.

Security hardening (mapping JSON may be reconstituted from untrusted
sources via mergeExtensions(), so the runtime must fail loud on hostile
shapes):

- buildTransforms() rejects custom handler names that collide with the
  default registry or shadow Object.prototype keys (constructor, toString,
  __proto__, etc.) at call time.
- validateMapping() rejects `__proto__` as an output field name at build
  time; transformFromMapping() rejects it again at walk time so the JSON
  attack vector (own-enumerable __proto__ key from JSON.parse) fails fast
  in both places.
- stringToNumber's error message does not embed the source value (would
  flow into PluginError.message and bypass the sourceValue PII guard).

Out of scope (matches Python PoC; deferred to full SDK):

- Auto-generation of transforms from declarative
  extensions.schemas[obj].mappings inside definePlugin() (Decision #6 TODO).
- Always-on commonModel validation inside definePlugin() — opt-in at
  buildTransforms() for now (Decision #7 TODO).

Includes:
- examples/transforms.ts round-trip (pnpm example:transforms)
- README "Plugin transformations" section + API reference table
- 5 new define-plugin specs, 30 transformation-handler specs,
  12 buildTransforms specs (427 tests total, all passing)
- Minor changeset bump for @common-grants/sdk

Targets HOLD-transforms per the SDK Plugin Enhancements branching strategy.
@SnowboardTechie SnowboardTechie force-pushed the bryan/issue-798-transforms-poc-typescript branch from 8394a3c to bda51e1 Compare May 13, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file enhancement New feature or request sdk Issue or PR related to our SDKs ts-sdk Related to TypeScript SDK typescript Issue or PR related to TypeScript tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant