Add unicode tips for SyntaxError suggestion

[Locked-chess-official]

Feature or enhancement

Proposal:

Now we have:

>>> impοrt abc
  File "<python-input-0>", line 1
    impοrt abc
    ^^^^^^
SyntaxError: invalid syntax. Did you mean 'import'?

The ο(U+03BF) and o(ascii) are similar in eyes. Maybe it is better to add the unicode tips when the SyntaxError caused by unicode characters.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

[StanFromIreland]

Maybe it is better to add the unicode tips when the SyntaxError caused by unicode characters.

How often does this happen in practice, I don't think I've ever heard of someone ending up in this situation (unintentionally)? I'm also curious as to how you plan to identify the similar characters?

[Locked-chess-official]

I'm also curious as to how you plan to identify the similar characters?

Just like the other exception, use NFCK to check the word.

This issue is an enextension of #143962

[StanFromIreland]

Just like the other exception, use NFCK to check the word.

Can you please give an example of you're proposed changes to the error message? Normalising will tell you there is a change, and then normalising each character will tell you which character(s) changed, but it won't tell you if ο and o look similar?

[Locked-chess-official]

Can you please give an example of you're proposed changes to the error message? Normalising will tell you there is a change, and then normalising each character will tell you which character(s) changed, but it won't tell you if ο and o look similar?

Here is the message:
If there is the suggestion given for the wrong name, when the wrong name is not totally ascii, the message can be Did you mean {suggestion!r} instead of {wrong_name!r} ({wrong_name!a})
Just check the wrong_name because all keywords are totally ascii.

[StanFromIreland]

I see, I suggest you add that to the issue (and in general please do so in the future, it's hard to discuss not know what exactly it is you're discussing ;-). I'm sorry but I don't really see much benefit, we already highlight the "wrong" name, and give the suggestion. @serhiy-storchaka, what do you think?

[serhiy-storchaka]

I have realistic scenario of such kind of errors.

In most of modern Cyrillic keyboards there is one key which types similarly looking Latin and Cyrillic letters "c" and "с". So, you can accidentally switch to Cyrillic layout, type "сдфіі" (meaning "class"), notice your error, switch to Latin layout and retype the wrong letters. And then get:

>>> сlass A:
  File "<python-input-1>", line 1
    сlass A:
    ^^^^^
SyntaxError: invalid syntax. Did you mean 'class'?

This can puzzle when occurred to you the first time, but re-typing the whole word (or copying it from the error message) fixes it. All keywords are ASCII-only, so you will not make the error when re-typing the word.

This is very different from #143962, where copying the non-normalized name from sources does not work, because getattr() etc need normalized names and which usually happens with non-ASCII names.

So, I think the current error message is fine.

[Locked-chess-official]

This can puzzle when occurred to you the first time, but re-typing the whole word (or copying it from the error message) fixes it.

In some place, the "copying" is disabled (e.g. error message pop-up or in some webs). In that case, user may not find that it is the question by input method. They may just feel puzzle and then don't know what was wrong and what to do. Tiping that the word is not ASCII-only can notice them to check the input method and change to the right mode.