From d04618355e4c98898fd99714e31b6f682c2071f9 Mon Sep 17 00:00:00 2001 From: Farnabaz Date: Wed, 2 Sep 2026 11:39:29 +0200 Subject: [PATCH 1/7] feat: implement richer syntax support --- AGENTS.md | 19 +- docs/content/5.reference/2.auto-close.md | 29 +- packages/comark/SPEC/auto-close.md | 974 ++++++++++++ packages/comark/src/index.ts | 7 +- .../src/internal/parse/auto-close/index.ts | 1342 +++++++++++------ .../src/internal/parse/auto-close/table.ts | 26 +- packages/comark/test/auto-close-spec.test.ts | 178 +++ packages/comark/test/auto-close.test.ts | 47 +- packages/comark/test/text-escape.test.ts | 2 +- 9 files changed, 2092 insertions(+), 532 deletions(-) create mode 100644 packages/comark/SPEC/auto-close.md create mode 100644 packages/comark/test/auto-close-spec.test.ts diff --git a/AGENTS.md b/AGENTS.md index a2f652b9..0f5e9007 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -527,13 +527,30 @@ result.frontmatter // Record result.meta // Record ``` -### autoCloseMarkdown(markdown) +### autoCloseMarkdown(markdown, options?) + +Self-healing markdown for streaming. Heals incomplete CommonMark/GFM per +`packages/comark/SPEC/auto-close.md`, then completes Comark components / tables / frontmatter. ```typescript autoCloseMarkdown('**bold text') // '**bold text**' autoCloseMarkdown('::alert\nContent') // '::alert\nContent\n::' + +// Incomplete links get a safe placeholder URL (default protocol mode) +autoCloseMarkdown('[partial') +// '[partial](comark:incomplete-link)' + +// Math (inline `$` and block `$$`) closes by default; disable with math: false +autoCloseMarkdown('$x = 5') // '$x = 5$' + +// Plain markdown without Comark component fences +autoCloseMarkdown('**bold', { syntax: false }) ``` +Key options: `linkMode: 'protocol' | 'text-only'`, `math` (default true), `incompleteLinkPlaceholder`, +`incompleteImagePlaceholder`, `frontmatter`, `syntax`, `attributes`. Behavioral SPEC: +`packages/comark/SPEC/auto-close.md` (run via `test/auto-close-spec.test.ts`). + ## Markdown Document Model ```typescript diff --git a/docs/content/5.reference/2.auto-close.md b/docs/content/5.reference/2.auto-close.md index 65131c5f..ff5d9968 100644 --- a/docs/content/5.reference/2.auto-close.md +++ b/docs/content/5.reference/2.auto-close.md @@ -14,14 +14,17 @@ links: variant: soft --- -## `autoCloseMarkdown(source)`{lang="ts"} +## `autoCloseMarkdown(source, options?)`{lang="ts"} -Automatically closes unclosed markdown inline syntax and Comark components. Built for streaming scenarios where content arrives incrementally and may be incomplete at any point. +Automatically closes unclosed markdown inline syntax and Comark components. Built for streaming scenarios where content arrives incrementally and may be incomplete at any point. CommonMark/GFM healing follows the behavioral SPEC in `packages/comark/SPEC/auto-close.md`. **Parameters:** - `source` - The markdown content (potentially partial/incomplete) -- `options` (optional) - `frontmatter: true` completes an unclosed leading frontmatter block; `syntax: false` skips Comark component-fence closing (for content parsed without the components plugin) +- `options` (optional) - Feature flags and Comark settings: + - **Comark:** `frontmatter: true` completes an unclosed leading frontmatter block; `syntax: false` skips component-fence closing; `attributes` controls `{...}` attribute scopes (defaults to `syntax`) + - **Links / images:** `linkMode: 'protocol' | 'text-only'` (default `'protocol'`), `incompleteLinkPlaceholder` (default `comark:incomplete-link`), `incompleteImagePlaceholder` (default `comark:incomplete-image`) + - **Math:** inline `$…$` and block `$$…$$` on by default (`math: true`); set `math: false` to disable **Returns:** `string`, the source with all unclosed syntax closed @@ -32,6 +35,16 @@ Automatically closes unclosed markdown inline syntax and Comark components. Buil import { autoCloseMarkdown } from 'comark' autoCloseMarkdown('**bold text') +// '**bold text**' + +autoCloseMarkdown('[partial link') +// '[partial link](comark:incomplete-link)' + +autoCloseMarkdown('$x = 5') +// '$x = 5$' + +autoCloseMarkdown('$x = 5', { math: false }) +// '$x = 5' ``` ```typescript [Output] @@ -79,11 +92,15 @@ const result = await parseMarkdown(closed, { autoClose: false }) | Syntax | Example | Auto-closed | |--------|---------|-------------| | Bold | `**text` | `**text**` | -| Italic | `*text` | `*text*` | +| Italic | `*text` / `_text` | `*text*` / `_text_` | +| Bold-italic | `***text` | `***text***` | | Code | `` `code `` | `` `code` `` | | Strikethrough | `~~text` | `~~text~~` | -| Link | `[text](url` | `[text](url)` | -| Image | `![alt](url` | `![alt](url)` | +| Link (protocol) | `[text](url` | `[text](comark:incomplete-link)` | +| Link (text-only) | `[text` | `text` | +| Image | `![alt](url` | `![alt](comark:incomplete-image)` | +| Block math | `$$x` | `$$x$$` | +| Inline math | `$x` | `$x$` (default; `math: false` to disable) | #### Comark components diff --git a/packages/comark/SPEC/auto-close.md b/packages/comark/SPEC/auto-close.md new file mode 100644 index 00000000..0f47cdf4 --- /dev/null +++ b/packages/comark/SPEC/auto-close.md @@ -0,0 +1,974 @@ +--- +# Behavioral SPEC for autoCloseMarkdown (not a parse fixture). +# Exercised by test/auto-close-spec.test.ts — skip the Input/AST/HTML runner. +skip: true +--- + +# Auto Close Markdown Spec + +Self-healing markdown for streaming. Completes incomplete syntax so partial AI output still renders cleanly. + +options: +- incompleteLinkPlaceholder: a placeholder for incomplete links (default: `comark:incomplete-link`) +- incompleteImagePlaceholder: a placeholder for incomplete images (default: `comark:incomplete-image`) +- math: auto-close inline `$…$` and block `$$…$$` (default: `true`) + + +--- + +## Bold + +Closes unclosed `**…**`. + +```diff +- Text with **bold ++ Text with **bold** +``` + +```diff +- **incomplete ++ **incomplete** +``` + +```diff +- **first** and **second ++ **first** and **second** +``` + +```diff +- **bold text* ++ **bold text** +``` + +```diff +- Text with **bold text** ++ Text with **bold text** +``` + +```diff +- ** ++ ** +``` + +--- + +## Italic (asterisk) + +Closes unclosed `*…*`. + +```diff +- Text with *italic ++ Text with *italic* +``` + +```diff +- *incomplete ++ *incomplete* +``` + +```diff +- **bold** and *italic ++ **bold** and *italic* +``` + +```diff +- Text ending with * ++ Text ending with * +``` + +--- + +## Italic (underscore) + +Closes unclosed `_…_` and `__…__`. + +```diff +- Text with _italic ++ Text with _italic_ +``` + +```diff +- _incomplete ++ _incomplete_ +``` + +```diff +- Text with __italic ++ Text with __italic__ +``` + +```diff +- __first__ and __second ++ __first__ and __second__ +``` + +```diff +- __bold text_ ++ __bold text__ +``` + +Trailing newline still gets closed: + +```diff +- Text with _italic\n ++ Text with _italic_\n +``` + +--- + +## Bold + italic + +Closes unclosed `***…***`. + +```diff +- Text with ***bold-italic ++ Text with ***bold-italic*** +``` + +```diff +- ***incomplete ++ ***incomplete*** +``` + +```diff +- ***first*** and ***second ++ ***first*** and ***second*** +``` + +```diff +- *italic* **bold** ***both ++ *italic* **bold** ***both*** +``` + +Does not treat overlapping bold+italic as bold-italic: + +```diff +- Combined **bold and *italic*** text ++ Combined **bold and *italic*** text +``` + +--- + +## Inline code + +Closes unclosed `` `…` ``. + +```diff +- Text with `code ++ Text with `code` +``` + +```diff +- `incomplete ++ `incomplete` +``` + +```diff +- ```python print("Hello")`` ++ ```python print("Hello")``` +``` + +After a finished code fence, still closes later inline code: + +```diff +- ```\nblock\n```\n`inline ++ ```\nblock\n```\n`inline` +``` + +Leaves finished inline code alone: + +```diff +- Text with `inline code` ++ Text with `inline code` +``` + +--- + +## Strikethrough + +Closes unclosed `~~…~~`. + +```diff +- Text with ~~strike ++ Text with ~~strike~~ +``` + +```diff +- ~~incomplete ++ ~~incomplete~~ +``` + +```diff +- ~~first~~ and ~~second ++ ~~first~~ and ~~second~~ +``` + +Half-closed closing marker: + +```diff +- ~~strike text~ ++ ~~strike text~~ +``` + +--- + +## Single tilde escape + +Escapes a lone `~` between word characters so it is not read as strikethrough. + +```diff +- 20~25°C ++ 20\~25°C +``` + +```diff +- foo~bar ++ foo\~bar +``` + +```diff +- 20~25 and ~~strike ++ 20\~25 and ~~strike~~ +``` + +Leaves intentional strikethrough alone: + +```diff +- ~~strikethrough~~ ++ ~~strikethrough~~ +``` + +Does not escape edge/space cases: + +```diff +- ~hello ++ ~hello +``` + +```diff +- hello~ ++ hello~ +``` + +```diff +- hello ~ world ++ hello ~ world +``` + +Does not escape open/close cases: + +```diff +- H~2~o ++ H~2~o +``` + +--- + +## Links (default protocol mode) + +Incomplete links become a safe placeholder URL. + +```diff +- Text with [incomplete link ++ Text with [incomplete link](comark:incomplete-link) +``` + +```diff +- Visit [our site](https://exa ++ Visit [our site](comark:incomplete-link) +``` + +```diff +- [outer [nested] text](incomplete ++ [outer [nested] text](comark:incomplete-link) +``` + +```diff +- Text [outer [inner ++ Text [outer [inner](comark:incomplete-link) +``` + +Leaves finished links alone: + +```diff +- Text with [complete link](url) ++ Text with [complete link](url) +``` + +--- + +## Links (text-only mode) + +Incomplete links become plain text (no link markup). + +```diff +- Text with [incomplete link ++ Text with incomplete link +``` + +```diff +- Visit [our site](https://exa ++ Visit our site +``` + +```diff +- [outer [nested] text](incomplete ++ outer [nested] text +``` + +```diff +- Check out [this lin ++ Check out this lin +``` + +Finished links stay links: + +```diff +- Text with [complete link](url) ++ Text with [complete link](url) +``` + +--- + +## Images + +Incomplete images become a loading placeholder. + +```diff +- Text with ![incomplete image ++ Text with ![incomplete image](comark:incomplete-image) +``` + +```diff +- Text with ![incomplete image] ++ Text with ![incomplete image](comark:incomplete-image) +``` + +```diff +- ![partial ++ ![partial](comark:incomplete-image) +``` + +```diff +- ![logo](./assets/log ++ ![logo](comark:incomplete-image) +``` + +```diff +- Text ![outer [inner] ++ Text ![outer [inner]](comark:incomplete-image) +``` + +Still uses image placeholder even in link text-only mode: + +```diff +- Text ![alt](http://partial ++ Text ![alt](comark:incomplete-image) +``` + +Leaves finished images alone: + +```diff +- Text with ![alt text](image.png) ++ Text with ![alt text](image.png) +``` + +--- + +## Block math (KaTeX) + +Closes unclosed `$$…$$`. + +```diff +- Text with $$formula ++ Text with $$formula$$ +``` + +```diff +- $$incomplete ++ $$incomplete$$ +``` + +```diff +- $$first$$ and $$second ++ $$first$$ and $$second$$ +``` + +```diff +- $$formula$ ++ $$formula$$ +``` + +Multiline: + +```diff +- $$\nx = 1\ny = 2 ++ $$\nx = 1\ny = 2\n$$ +``` + +--- + +## Inline math + +Closes unclosed `$…$` when `math` is enabled (default). + +```diff +- Text with $formula ++ Text with $formula$ +``` + +```diff +- $first$ and $second ++ $first$ and $second$ +``` + +```diff +- $$block$$ and $inline ++ $$block$$ and $inline$ +``` + +With `math: false`, leaves both inline and block math alone: + +```diff +- Text with $formula ++ Text with $formula +``` + +--- + +## Incomplete HTML tags + +Strips a partial HTML tag at the end so it never flashes raw markup. + +```diff +- Hello