Skip to content

fix(cli): fold non-ASCII case when matching names - #6079

Open
Chessing234 wants to merge 3 commits into
block:mainfrom
Chessing234:fix/cli-unicode-name-matching
Open

fix(cli): fold non-ASCII case when matching names#6079
Chessing234 wants to merge 3 commits into
block:mainfrom
Chessing234:fix/cli-unicode-name-matching

Conversation

@Chessing234

Copy link
Copy Markdown
Contributor

Four user-facing name matchers in the CLI lowercase with to_ascii_lowercase, which leaves every non-ASCII letter exactly as it was. So a name in any language with cased non-ASCII letters cannot be found by the obvious query:

command file effect
channels list --name (substring and --exact) commands/channels.rs:135,216 the channel is missing from the results
users search (display_name and name) commands/users.rs:151,164 the user is missing from the results
notes ls --author <name> commands/notes.rs:220,232 hard failure: no user found with display_name "josé"
channel template lookup by name commands/channel_templates.rs:101,104 NotFound, listing names that visibly include the one asked for

notes ls --author is the sharp one — it is a resolution gate, so the command exits non-zero rather than just returning less.

This is also a CLI-vs-app disagreement: Desktop matches the same names with JavaScript's toLowerCase, which is Unicode-aware, so the two answer the same question differently.

All four now go through validate::fold_name. The expectations in its tests are node's toLowerCase output for the same inputs, since Desktop is the other implementation of this rule and they have to agree:

"ÉQUIPE"    -> "équipe"       "Straße"    -> "straße"
"ОБЩИЙ"     -> "общий"        "Džungla"    -> "džungla"
"Ünnepek"   -> "ünnepek"      "İstanbul"  -> "i̇stanbul"
"JOSÉ"      -> "josé"         "ÅNGSTRÖM"  -> "ångström"

Hex, pubkeys and UUIDs deliberately keep to_ascii_lowercase. They are ASCII by construction, and a Unicode fold there is the kind of thing that gets surprised by a Turkish dotless i. The doc comment on fold_name says so, so the next reader does not "fix" the ones that are already right.

Reverting fold_name to to_ascii_lowercase turns both new tests red.

Verified locally: cargo test -p buzz-cli --lib 352 passed (349 before), cargo clippy -p buzz-cli --all-targets -- -D warnings clean, cargo fmt --all -- --check, git diff --check. Not run: a live relay — the profile and template inputs come from fixtures in the tests.

Four user-facing name matchers lowercased with to_ascii_lowercase, which
leaves every non-ASCII letter untouched. So a channel named EQUIPE with an
accented E was unfindable by the obvious query, and buzz notes ls --author
with an accented display name failed outright with "no user found":

- channels list --name  (substring and --exact)
- users search          (display_name and name)
- notes ls --author     (name to pubkey resolution)
- channel templates     (lookup by name)

Desktop matches the same names with JavaScript's toLowerCase, which is
Unicode-aware, so this was also a disagreement between the CLI and the app
about what a query finds.

All four now go through validate::fold_name. Hex, pubkeys and UUIDs keep
to_ascii_lowercase: they are ASCII by construction, and a Unicode fold there
would only invite a dotless-i surprise.

Signed-off-by: Taksh <takshkothari09@gmail.com>
Every expectation in fold_name_tests is node's toLowerCase output for the
same input, because Desktop is the other implementation of this rule and the
two have to agree about what a query finds. Covers Latin accents, Cyrillic,
the German eszett, a titlecase digraph, and the Turkish dotted capital I,
whose fold adds a combining dot.

name_matches gains the substring and exact cases that were unreachable
before, plus a non-match so the fold has not made matching sloppy.

Reverting fold_name to to_ascii_lowercase turns both new tests red.

Signed-off-by: Taksh <takshkothari09@gmail.com>
@Chessing234
Chessing234 requested a review from a team as a code owner August 16, 2026 19:49

@themiguelamador themiguelamador left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found three remaining user-facing name-resolution paths that still used ASCII-only case matching:

  • buzz messages search --author did not match non-ASCII display names across case.
  • owned-agent name lookup did not match non-ASCII agent names across case.
  • automatic @Display Name resolution still used ASCII-only keys in the CLI and ASCII-only known-name/profile matching in buzz-sdk, so a member named ÉQUIPE was not resolved from @équipe.

I fixed all three in commit 25233b618 on Complear:review/pr-6079-fix. The patch centralizes the Unicode lowercase behavior in buzz_sdk::mentions::fold_name, applies it across the remaining lookup paths, and handles lowercase mappings that expand in byte length (for example İi plus combining dot).

Verification:

  • cargo test -p buzz-sdk --lib --quiet — 264 passed
  • cargo test -p buzz-cli --lib --quiet — 354 passed
  • cargo clippy -p buzz-sdk --all-targets -- -D warnings
  • cargo clippy -p buzz-cli --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

… to the sdk

Review finding (P1, themiguelamador on block#6079): three user-facing lookups still
matched names ASCII-only.

- `buzz messages search --author`: `match_profiles_by_name` folded with
  `to_ascii_lowercase`, so a non-ASCII display name was reachable only by
  reproducing its exact case.
- Owned-agent lookup: `eq_ignore_ascii_case` is case-blind for ASCII letters
  alone, so an agent named `Équipe` was unreachable as `équipe`.
- Automatic `@Display Name` resolution: buzz-cli keyed its `name → pubkey` map
  with `to_ascii_lowercase` while `extract_at_mentions_with_known` compared
  known names with `eq_ignore_ascii_case` and returned ASCII-folded keys — so a
  member named `ÉQUIPE` was not resolved from `@équipe` at either end.

`fold_name` moves to `buzz_sdk::mentions`, which is where it has to live: the
SDK returns the folded keys and the CLI looks them up, so one copy of the rule
is the only way the two can agree. buzz-cli re-exports it, leaving existing
call sites unchanged.

Matching a known name also can no longer slice the content at the known name's
byte length, because folding can change that length — `İ` is two bytes and
folds to `i̇`, which is three. `folded_prefix_len` folds the content one char
at a time and reports how many *source* bytes it consumed, and the word-
boundary check runs at that offset, so `@équipement` still does not match a
member named `ÉQUIPE`.

Signed-off-by: Taksh <takshkothari09@gmail.com>
@Chessing234

Copy link
Copy Markdown
Contributor Author

All three confirmed and fixed in 50ebc2b. (Complear/buzz 404s, so written from your description rather than cherry-picked.)

Verified each: match_profiles_by_name (messages search --author) folded with to_ascii_lowercase; owned_agent_pubkeys_from_events used eq_ignore_ascii_case, which is case-blind for ASCII letters only; and the @Display Name path was broken at both ends — buzz-cli keyed its name → pubkey map with to_ascii_lowercase while extract_at_mentions_with_known compared with eq_ignore_ascii_case and returned ASCII-folded keys.

Agreed on the placement: fold_name now lives in buzz_sdk::mentions, because the SDK produces the folded keys and the CLI looks them up — one copy of the rule is the only way those two can agree. buzz-cli re-exports it, so existing call sites are unchanged.

The byte-length expansion you flagged needed a bit more than folding both sides: matching a known name could no longer slice the content at the known name's length, since İ is two bytes and folds to three. folded_prefix_len folds the content one character at a time and reports how many source bytes it consumed, and the word-boundary check runs at that offset — so @équipement still doesn't match a member named ÉQUIPE. That case is pinned, along with the length-changing fold itself.

Verification: cargo test -p buzz-sdk --lib (265 passed), -p buzz-cli --lib (354 passed), clippy -D warnings on both, cargo fmt --all --check, git diff --check. The existing fold expectations are still checked against node's toLowerCase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants