bpo-37751: Document the change in What's New in Python 3.9 #17997
Conversation
|
cc @malemburg |
|
Oops, the change was done in Python 3.9, not in Python 3.8! PR updated. |
| @@ -439,6 +439,12 @@ Changes in the Python API | |||
| :data:`~errno.EBADF` error. | |||
| (Contributed by Victor Stinner in :issue:`39239`.) | |||
|
|
|||
| * :func:`codecs.lookup` now normalizes the encoding name the same way than | |||
There was a problem hiding this comment.
There are other differences. For example, normalize_encoding("КОИ-8") returns "кои_8", but codecs.lookup normalizes it to "8".
The comment in the sources is also not correct.
There was a problem hiding this comment.
encodings.normalize_encoding() says "Note that encoding names should be ASCII only." You're correct: "КОИ-8" is normalized to "8" by codecs.lookup() because the C function _Py_normalize_encoding() ignores non-ASCII letters.
I don't know which behavior is correct. It sounds strange to me to have a non-ASCII encoding name. Which encoding is supposed to be used to encoding the encoding name?!? :-D Maybe encodings.normalize_encoding() should also ignore non-ASCII letters, be more strict.
There was a problem hiding this comment.
I created bpo-39337: codecs.lookup() ignores non-ASCII characters, whereas encodings.normalize_encoding() copies them.
There was a problem hiding this comment.
Maybe encodings.normalize_encoding() should also ignore non-ASCII letters, be more strict.
Hm, the annotation of normalize_encoding have the words: Note that encoding names should be ASCII only.
+1 for encodings.normalize_encoding() should be similar as _Py_normalize_encoding().
And I created a PR: #22219
There was a problem hiding this comment.
There are other differences. For example, normalize_encoding("КОИ-8") returns "кои_8", but codecs.lookup normalizes it to "8".
After #22219 merged, this problem have been fixed(MAYBE enhanced will be more exact).
| @@ -439,6 +439,12 @@ Changes in the Python API | |||
| :data:`~errno.EBADF` error. | |||
| (Contributed by Victor Stinner in :issue:`39239`.) | |||
|
|
|||
| * :func:`codecs.lookup` now normalizes the encoding name the same way than | |||
There was a problem hiding this comment.
| * :func:`codecs.lookup` now normalizes the encoding name the same way than | |
| * :func:`codecs.lookup` now normalizes the encoding name the same way as |
There was a problem hiding this comment.
I create a new PR in #23096, and this word have been replaced.
|
@vstinner is this one that needs to be merged soon? |
|
@shihai1991: I failed to find time to finish this PR. Since you are wokring on bpo-39337, do you want to continue the work on this PR? You can steal it (copy/paste my change), and try to address previous comments. |
No problem, I will take a look :) |
|
A new PR in #23096 (copy from this PR). |
https://bugs.python.org/issue37751