fix(desktop): stop cutting previews through an emoji - #5996
Conversation
`String.prototype.slice` counts UTF-16 code units. Anything outside the Basic
Multilingual Plane — every emoji, and plenty of CJK — is stored as a surrogate
pair, so a cut that lands inside one leaves a lone surrogate behind: not a
character, and rendered as `\u{FFFD}` wherever the result is shown.
`truncateByCharacters` cuts between characters instead.
Characters here means code points, not grapheme clusters: a cut can still land
between the parts of a ZWJ sequence, turning a family emoji into one person.
That is a different picture but a valid string, unlike the lone surrogate,
which is not text at all.
No call sites yet.
Signed-off-by: Taksh <takshkothari09@gmail.com>
Seven places truncate user text for display with `slice`, which counts code
units. A message ending "…thanks 🎉" cut at the 100-unit mark loses half the
emoji and the preview ends in `\u{FFFD}`: reminder previews from the message
row, the Home inbox and the channel activity popover, search result excerpts,
forum post cards, pulse note snippets and the projects activity feed.
Route all seven through `truncateByCharacters`. The limits are unchanged —
only where the cut lands is.
Signed-off-by: Taksh <takshkothari09@gmail.com>
themiguelamador
left a comment
There was a problem hiding this comment.
The shared helper is sound, but this patch is incomplete and has a mixed-unit guard bug.
Blocking correctness issues:
ForumPostCard.tsxandSearchResultItem.tsxdecide whether to append an ellipsis with UTF-16.length, then truncate by code points. An emoji-heavy string can trip the guard whiletruncateByCharactersreturns the entire string, producing a false ellipsis. Compare the helper result with the original (or otherwise use the same unit for the guard).- Five additional user-visible truncators still use UTF-16
slice: assistant activity headlines (agentSessionTranscriptPresentation.ts), project discussion snippets (discussionChannels.ts), top-bar search previews (TopbarSearch.tsx), huddle reaction participant names (HuddleBar.tsx), and notification bodies (notificationFormat.ts). Each can still end in a lone surrogate and should use the helper.
Minor: “every emoji” is outside the BMP is too broad; “most emoji” is accurate.
I applied the complete patch locally, including focused regressions for assistant headlines and discussion snippets. Verification passes: 4,960 desktop tests, TypeScript checking, Biome, file-size ratchet, pixel-text check, pubkey-truncation check, and the desktop pre-commit gate. The signed-off local commit is 69206585a; direct pushes to this contributor fork are denied to the authenticated maintainer account, as already confirmed on another PR from the same fork.
Review caught a bug this branch introduced. `ForumPostCard` and `SearchResultItem` decided whether to truncate with UTF-16 `.length` and then cut by code points. Those disagree: 150 emoji are 150 characters and 300 code units, so a `length > 200` guard fires while `truncateByCharacters` returns the string untouched — and the caller appends an ellipsis to text that was never shortened, promising more that isn't there. Add `countCharacters` and use it for the guards, so the decision and the cut are made in the same unit. Also corrects the helper's doc: "every emoji" is outside the BMP was too broad. Most are; ☺ and ✌ are not. Signed-off-by: Taksh <takshkothari09@gmail.com>
Review found five more user-visible truncators still cutting on UTF-16 code units, each able to end on a lone surrogate and each guarding in a different unit than it cuts: - assistant activity headlines (`agentSessionTranscriptPresentation.ts`) - project discussion snippets (`discussionChannels.ts`) - top-bar search previews (`TopbarSearch.tsx`) - huddle reaction participant names (`HuddleBar.tsx`) - notification bodies (`notificationFormat.ts`) All five now guard with `countCharacters` and cut with `truncateByCharacters`. Limits and trimming behaviour are unchanged. Headlines and discussion snippets get focused regressions, covering both halves: a cut that used to land mid-emoji, and emoji-heavy text that used to be ellipsised despite fitting. Signed-off-by: Taksh <takshkothari09@gmail.com>
|
both blocking issues confirmed and fixed. thanks — the guard bug was mine and i'd missed it entirely. mixed-unit guard. reproduced first: const s = '🎉'.repeat(150); // 150 characters, 300 code units
s.length > 200 // true -> guard fires
truncateByCharacters(s, 200) // returns s unchanged
// -> '...' appended to complete text
the five remaining truncators. all five confirmed real and user-visible; headlines and discussion snippets get focused regressions covering both directions: a cut that used to land mid-emoji, and emoji-heavy text that used to be ellipsised despite fitting. wording. fixed — "most emoji" in the helper doc. ☺ and ✌ are BMP; "every" was wrong. i left verification: full desktop suite 4964 passed / 0 failed, on the 403: i don't actually know why github rejected your push — i stated a reason here earlier (2fa on the fork owner's account) that i had not checked, so i've removed it. |
themiguelamador
left a comment
There was a problem hiding this comment.
Re-reviewed current head c6ae71321. The two follow-up commits correctly resolve all prior Unicode findings, including the mixed-unit false-ellipsis guards and all five remaining user-visible UTF-16 truncators.
One merge blocker remains: pnpm check:file-sizes fails because this PR leaves HomeView.tsx at 1,001 measured lines and TopbarSearch.tsx at 1,004, above the enforced 1,000-line ceiling. The smallest correction removes two separator-only blank lines and uses the already-established helper-result comparison in TopbarSearch, preserving the code-point semantics without the extra multi-line import.
Signed fix: 3fea4dcf2
https://github.com/Complear/buzz/tree/review/pr-5996-fix-v2
Verified on the final result: full desktop unit suite, TypeScript, focused Biome, file-size ratchet, pre-commit desktop gate, and git diff --check.
`pnpm check:file-sizes` failed: this file sat exactly at the 1,000-line ceiling, and the four-line import I added pushed it to 1,004. Drop `countCharacters` here and compare the helper's own result against the input instead. That keeps the guard and the cut in the same unit — the point of the earlier fix — while needing only the single-line `truncateByCharacters` import, and it reads no worse. The two separator-only blank lines inside the function go with it. Behaviour is unchanged: a string of exactly `maxLength` characters still comes back whole, and anything longer is still cut to `maxLength - 3`. Signed-off-by: Taksh <takshkothari09@gmail.com>
Same ratchet, one line over: this file also sat exactly at 1,000, so the single import line I added put it at 1,001. Reclaim the separator blank between the import block and the first constant. Nothing else in the file has slack — the surrounding JSX is dense — and the alternative was leaving `preview` cutting through an emoji here while every other surface stopped. Signed-off-by: Taksh <takshkothari09@gmail.com>
|
fixed in both files sat exactly on the ceiling, so any import at all put them over. i'd hit this same guard earlier in the session on
verification: |
Found by sweeping the display-truncation sites after #5992; no issue filed.
String.prototype.slicecounts UTF-16 code units. Anything outside the Basic Multilingual Plane — every emoji, and plenty of CJK — is stored as a surrogate pair, so a cut that lands inside one leaves a lone surrogate behind. That is not a character, and it renders as�:Seven places truncate user text for display that way, so a message ending "…thanks 🎉" can show
…thanks �:truncateByCharacters(first commit) cuts between characters instead. The limits are unchanged — only where the cut lands is.One thing stated plainly in the helper's doc: characters here means code points, not grapheme clusters. A cut can still land between the parts of a ZWJ sequence and turn a family emoji into one person. That is a different picture but a valid string, unlike the lone surrogate, which is not text at all. Grapheme segmentation would need
Intl.Segmenter, and given #5547's history with what the packaged WebKit will and won't do, that felt like a separate decision rather than something to slip into a truncation fix.Verified locally at this head:
pnpm test— 4958 passed, 0 failed (4954 before, plus the 4 new)pnpm check— clean; its 2 warnings and 2 infos are pre-existing and identical onmainpnpm build— succeededNote: I'm an outside contributor, so the workflow runs here sit at
action_requireduntil a maintainer approves them; only the DCO check reports on its own.