Skip to content

gh-91904: Fix setting envvar PYTHONREGRTEST_UNICODE_GUARD #91905

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Apr 25, 2022

It always failed on non-UTF-8 locale and prevented running regrtests.

Closes #91904.

@serhiy-storchaka serhiy-storchaka added tests Tests in the Lib/test dir 🔨 test-with-buildbots Test PR w/ buildbots; report in status section needs backport to 3.10 only security fixes labels Apr 25, 2022
@serhiy-storchaka serhiy-storchaka requested a review from ambv April 25, 2022 08:22
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 83046e7e2084773e7c82ef73eb7e928d09fd91d6 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot added awaiting core review and removed 🔨 test-with-buildbots Test PR w/ buildbots; report in status section labels Apr 25, 2022
It always failed on non-UTF-8 locale and prevented running regrtests.
"\N{SMILING FACE WITH SUNGLASSES}",
)
if TESTFN_UNDECODABLE and hasattr(os, 'environb'):
os.environb.setdefault(UNICODE_GUARD_ENV.encode(), TESTFN_UNDECODABLE)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.environb is a wrapper to os.environ. You can use os.fsencode()/os.fsdecode() to set the key in os.environ.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather os.environ is a wrapper to os.environb.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine if you want to use os.environb. It's just that I regret that I added it: surrogateescape of PEP 383 is a better solution to use Unicode (os.environ) on all platforms. It avoids treating Unix differently. os.environb is just here to reduce suprises for Unix users. Internally, os.environ and os.environb are sharing the same Python dict.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this dict contains bytes.

If use os.environ, there will be double conversion: encode(decode(value)). I am not sure that it is lossless in all cases.

@serhiy-storchaka serhiy-storchaka force-pushed the PYTHONREGRTEST_UNICODE_GUARD branch from 83046e7 to a351699 Compare April 25, 2022 08:26
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@serhiy-storchaka serhiy-storchaka added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 25, 2022
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit f16859b 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 25, 2022
@serhiy-storchaka serhiy-storchaka merged commit 54d068a into python:main Apr 25, 2022
@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

@serhiy-storchaka serhiy-storchaka deleted the PYTHONREGRTEST_UNICODE_GUARD branch April 25, 2022 14:35
@bedevere-bot
Copy link

GH-91911 is a backport of this pull request to the 3.10 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label Apr 25, 2022
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Apr 25, 2022
…onGH-91905)

It always failed on non-UTF-8 locale and prevented running regrtests.
(cherry picked from commit 54d068a)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
miss-islington added a commit that referenced this pull request Apr 25, 2022
It always failed on non-UTF-8 locale and prevented running regrtests.
(cherry picked from commit 54d068a)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tests always fail on non-UTF-8 locale
4 participants