Skip to content

gh-91904: Fix setting envvar PYTHONREGRTEST_UNICODE_GUARD #91905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 25, 2022
Merged

gh-91904: Fix setting envvar PYTHONREGRTEST_UNICODE_GUARD #91905

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a351699
gh-91904: Fix setting envvar PYTHONREGRTEST_UNICODE_GUARD
serhiy-storchaka Apr 25, 2022
f16859b
Fix markup.
serhiy-storchaka Apr 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions Lib/test/libregrtest/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import sys
import unittest
from test import support
from test.support.os_helper import TESTFN_UNDECODABLE, FS_NONASCII
try:
import gc
except ImportError:
Expand Down Expand Up @@ -104,10 +105,10 @@ def _test_audit_hook(name, args):

# Ensure there's a non-ASCII character in env vars at all times to force
# tests consider this case. See BPO-44647 for details.
os.environ.setdefault(
UNICODE_GUARD_ENV,
"\N{SMILING FACE WITH SUNGLASSES}",
)
if TESTFN_UNDECODABLE and os.supports_bytes_environ:
os.environb.setdefault(UNICODE_GUARD_ENV.encode(), TESTFN_UNDECODABLE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.environb is a wrapper to os.environ. You can use os.fsencode()/os.fsdecode() to set the key in os.environ.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather os.environ is a wrapper to os.environb.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine if you want to use os.environb. It's just that I regret that I added it: surrogateescape of PEP 383 is a better solution to use Unicode (os.environ) on all platforms. It avoids treating Unix differently. os.environb is just here to reduce suprises for Unix users. Internally, os.environ and os.environb are sharing the same Python dict.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this dict contains bytes.

If use os.environ, there will be double conversion: encode(decode(value)). I am not sure that it is lossless in all cases.

elif FS_NONASCII:
os.environ.setdefault(UNICODE_GUARD_ENV, FS_NONASCII)


def replace_stdout():
Expand Down
2 changes: 1 addition & 1 deletion Lib/test/test_regrtest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1339,7 +1339,7 @@ def test_print_warning(self):
def test_unicode_guard_env(self):
guard = os.environ.get(setup.UNICODE_GUARD_ENV)
self.assertIsNotNone(guard, f"{setup.UNICODE_GUARD_ENV} not set")
if guard != "\N{SMILING FACE WITH SUNGLASSES}":
if guard.isascii():
# Skip to signify that the env var value was changed by the user;
# possibly to something ASCII to work around Unicode issues.
self.skipTest("Modified guard")
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Tests/2022-04-25-11-16-36.gh-issue-91904.13Uvrz.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix initialization of :envvar:`PYTHONREGRTEST_UNICODE_GUARD` which prevented
running regression tests on non-UTF-8 locale.