Skip to content

[http-client-python][ICM] Escape generated string literals and wire names - #12342

Merged
Yuchao Yan (msyyc) merged 13 commits into
mainfrom
users/msyyc/python-discriminator-string-literals
Nov 4, 2026
Merged

Yuchao Yan (msyyc) merged 13 commits into
mainfrom
users/msyyc/python-discriminator-string-literals

Conversation

@msyyc

@msyyc Yuchao Yan (msyyc) commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fix Python string-literal generation described in this report and #12405.

  • Use Python literal serialization for string constants and discriminator class arguments instead of adding quotes around specification values.
  • Apply the same fix to discriminator initialization and JSON input templates.
  • Escape property and discriminator wire names in JSON-template keys, field metadata, and functional TypedDict declarations. Keep optional-property ordering consistent and escape wire names for their enclosing documentation strings.
  • Escape constant descriptions and literal type annotations in complete model docstrings. Encode discriminator wire names in both polymorphic input and response example headers.
  • Use the existing operation docstring escaping helper for parameter documentation so quoted default values remain valid in complete sync and async operations.
  • Apply the same helper to client and configuration parameter documentation without escaping closing docstring delimiters or double-escaping default values.
  • Preserve Python expressions and literal escapes through JSON template rendering. Keep code examples unwrapped and escape them for their enclosing docstrings.
  • Add regression coverage for quotes, backslashes, newlines, control characters, Unicode, and expression-like text. Check literal AST nodes, annotations, repeated model construction, discriminator selection, serialization, shared enum generation, required and optional wire names, and TypedDict annotation keys.
  • Update equivalent quotation-style expectations and add a fix changelog.

Validation

  • Python emitter build, package formatting checks, repository-wide pnpm format, and emitter/generator lint passed.
  • Emitter tests: 35 passed. A cold browser-import test initially exceeded its five-second timeout; rerunning the suite with a 20-second test timeout passed all tests without code changes.
  • Related Python generator tests: 281 passed, 1 deselected.
  • The string-literal regression file now covers 204 cases, including complete DPG/TypedDict serializer output, complete sync/async operations, clients and configurations with parameter defaults, and complete polymorphic input/response examples embedded in runtime operation docstrings. Each review finding was reproduced before applying its fix.
  • Generated and imported a Python SDK from a minimal TypeSpec service with quotes, backslashes, newlines, and tabs in encoded property and discriminator wire names. Parsed all 23 generated Python files and confirmed exact serialization keys, repeated construction, TypedDict keys, and discriminator dispatch.
  • Earlier discriminator-value validation also generated JSON-mode sync and async SDKs, parsed their operation docstrings, and verified four final code examples preserve the exact discriminator value.

The deselected test, test_models_mode_typeddict_does_not_write_empty_models_folder, fails identically against an isolated copy of the unchanged PR-head generator. It is unrelated to string serialization and is not changed here. Local Python runtime validation used Python 3.13.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: ecfba725-b27d-416c-95a6-bc48a498ca8f
@pkg-pr-new

pkg-pr-new Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@12342

commit: f5b9cf3

@msyyc
Yuchao Yan (msyyc) marked this pull request as draft October 22, 2026 05:53
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

All changed packages have been documented.

  • ✅ @typespec/http-client-python
Show changes

@typespec/http-client-python - fix ✏️

Escape string literals and wire names in generated Python declarations, discriminator initializers, field metadata, and templates to preserve specification values as data.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Targeted verification confirms that downstream template rendering still corrupts escaped values and can produce invalid Python examples.

1 open issue
What changed in this PR

This PR updates the Python client generator to serialize specification strings as Python literals, preserving special characters and treating expression-like text as data.

Changes:

  • Replace manual quoting in string declarations, discriminator arguments, initializers, and template values.
  • Add regression tests for literal preservation and generated model behavior; adjust TypedDict quotation expectations.
  • Add a Python emitter fix changelog.
File Description
packages/​http-client-python/​tests/​unit/​test_typeddict.py Update two expectations for equivalent literal quoting.
packages/​http-client-python/​tests/​unit/​test_string_literals.py Add special-character and discriminator regression coverage.
packages/​http-client-python/​generator/​pygen/​codegen/​serializers/​model_serializer.py Escape discriminator initializers and class arguments.
packages/​http-client-python/​generator/​pygen/​codegen/​models/​primitive_types.py Serialize string declarations with repr.
packages/​http-client-python/​generator/​pygen/​codegen/​models/​model_type.py Escape discriminator values in template representations.
.chronus/​changes/​python-discriminator-string-literals-2026-10-22.md Document the Python string-literal fix.

Review effort: Balanced

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Python emitter diff

Baseline gh:05b9bad7c57b9b58e8fb5390f9bab081a1a9ac31 vs this PR.

No changes to generated output.

Rendered diff: inline on the run summary, or the emitter-diff-html artifact.

Informational check (eng/emitter-diff); does not block the PR.

@azure-sdk-automation

azure-sdk-automation Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ecfba725-b27d-416c-95a6-bc48a498ca8f
@msyyc

Yuchao Yan (msyyc) commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

CI investigation: the optional Azure/typespec-azure integration check fails on both the original and follow-up heads, outside this Python change. Latest run: https://github.com/microsoft/typespec/actions/runs/39734024339/job/119675416698. The downstream playground imports registerMonacoDefaultWorkersForVite from @typespec/playground, but the current API is exported from @typespec/playground/vite/workers. The downstream import needs updating; this PR does not change playground exports or bypass the check. All other current-head checks have now passed. Tracked in Azure/typespec-azure#5806; monitoring continues.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Complete model rendering still produces invalid Python for triple-quote discriminator values, which the hand-built regression does not exercise.

1 open issue
1 resolved since last review

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 27, 2026 02:13
Copilot AI balanced review requested due to automatic review settings October 27, 2026 03:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iscai-msft iscai-msft left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are a lot of diffs in the generated code, and i think most of them we don't want to change. i believe there's also a testserver case for the "too-long example code", you could see what changes when you remove the code in jinja

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Copilot AI balanced review requested due to automatic review settings October 29, 2026 09:03
@msyyc
Yuchao Yan (msyyc) marked this pull request as draft October 29, 2026 09:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Three test assertions expect single quotes even though the generated declarations retain double quotes, causing all three tests to fail.

1 open issue
5 resolved since last review

Review effort: Balanced

Yuchao Yan (msyyc) and others added 3 commits October 29, 2026 17:20
Merge main to align core dependency catalog references with Azure integration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
Replace escaping flags with explicit multiline and inline content helpers. Keep client/config delimiters in templates and preserve generated source output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: aaeb82ef-b4f1-4a88-9442-edd2ea4061cf
@msyyc
Yuchao Yan (msyyc) marked this pull request as ready for review November 2, 2026 06:20
@msyyc
Yuchao Yan (msyyc) requested review from iscai-msft and a balanced review from Copilot November 2, 2026 06:20
@msyyc Yuchao Yan (msyyc) changed the title [http-client-python] Escape generated string literals and wire names [http-client-python][ICM] Escape generated string literals and wire names Nov 2, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the Python emitter’s string-literal preservation changes.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@msyyc
Yuchao Yan (msyyc) added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit ee4f2c5 Nov 4, 2026
38 checks passed
@msyyc
Yuchao Yan (msyyc) deleted the users/msyyc/python-discriminator-string-literals branch November 4, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants