Skip to content

Check if we're using GNU sed instead of macOS #263

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 11, 2025
Merged

Conversation

hugovk
Copy link
Member

@hugovk hugovk commented Apr 11, 2025

By default, macOS comes with an old FreeBSD version of sed which doesn't have as many options as GNU sed you find elsewhere.

Default:

which sed
/usr/bin/sed
sed --version
sed: illegal option -- -
usage: sed script [-EHalnru] [-i extension] [file ...]
        sed [-EHalnu] [-i extension] [-e script] ... [-f script_file] ... [file ...]
man sed
[snip]
macOS 15.3                                    June 10, 2020                                   macOS 15.3

Instead, I use GNU sed installed via Homebrew:

brew install gsed
==> Downloading https://formulae.brew.sh/api/formula.jws.json
==> Downloading https://formulae.brew.sh/api/cask.jws.json
==> Downloading https://ghcr.io/v2/homebrew/core/gnu-sed/manifests/4.9-3
Already downloaded: /Users/hugo/Library/Caches/Homebrew/downloads/593980c8b4be8904de2f431575ef35e9088505050fa2ed84d7973fd3bffd180b--gnu-sed-4.9-3.bottle_manifest.json
==> Fetching gnu-sed
==> Downloading https://ghcr.io/v2/homebrew/core/gnu-sed/blobs/sha256:70edbfd4aa9ec24bd48e21353d18433741c13ec10c9903d5c93349eabb83bebb
Already downloaded: /Users/hugo/Library/Caches/Homebrew/downloads/198bf8e1368cf506186f607a57a32087bd7a7d1e29ea3fe2886421c82d02de5d--gnu-sed--4.9.arm64_sequoia.bottle.3.tar.gz
==> Pouring gnu-sed--4.9.arm64_sequoia.bottle.3.tar.gz
==> Caveats
GNU "sed" has been installed as "gsed".
If you need to use it as "sed", you can add a "gnubin" directory
to your PATH from your bashrc like:

    PATH="/opt/homebrew/opt/gnu-sed/libexec/gnubin:$PATH"
==> Summary
🍺  /opt/homebrew/Cellar/gnu-sed/4.9: 13 files, 616.4KB
==> Running `brew cleanup gnu-sed`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
which sed
/opt/homebrew/opt/gsed/libexec/gnubin/sed
sed --version
sed (GNU sed) 4.9
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Jay Fenlason, Tom Lord, Ken Pizzini,
Paolo Bonzini, Jim Meyering, and Assaf Gordon.

This sed program was built without SELinux support.

GNU sed home page: <https://www.gnu.org/software/sed/>.
General help using GNU software: <https://www.gnu.org/gethelp/>.
E-mail bug reports to: <bug-sed@gnu.org>.

However, we assume macOS is using default sed (#177) and [""] if sys.platform == "xdarwin" else [] fails with GNU sed:

2025-04-11 12:56:30,069 ERROR en/3.13: Run: 'sed -i 's/ *-A switchers=1//' /private/tmp/build_root/cpython-only-html-en/Doc/Makefile' KO:
    sed: 1: "/private/tmp/build_root ...": undefined label 'mp/build_root/cpython-only-html-en/Doc/Makefile'

2025-04-11 12:56:30,071 ERROR en/3.13: Badly handled exception, human, please help.
Traceback (most recent call last):
  File "/Users/hugo/github/docsbuild-scripts/./build_docs.py", line 554, in run
    self.build()
    ~~~~~~~~~~^^
  File "/Users/hugo/github/docsbuild-scripts/./build_docs.py", line 662, in build
    run(
    ~~~^
        ["sed", "-i"]
        ^^^^^^^^^^^^^
        + ([""] if sys.platform == "xdarwin" else [])
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        + ["s/ *-A switchers=1//", self.checkout / "Doc" / "Makefile"]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/hugo/github/docsbuild-scripts/./build_docs.py", line 296, in run
    result.check_returncode()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/subprocess.py", line 508, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
                             self.stderr)
subprocess.CalledProcessError: Command '['sed', '-i', 's/ *-A switchers=1//', '/private/tmp/build_root/cpython-only-html-en/Doc/Makefile']' returned non-zero exit status 1.

So this PR instead checks if we're using GNU sed by confirming if --version exists (similar to https://stackoverflow.com/a/65497543/724176).

Copy link
Member

@AA-Turner AA-Turner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a Mac, so can't test, but looks fine.

We could also do this via Python for (better) Windows support? sed is only used to remove some text. Low priority, though, as I'm really the only person affected by this.

Comment on lines +660 to +673
def is_gnu_sed() -> bool:
"""Check if we are using GNU sed."""
try:
subprocess.run(
["sed", "--version"],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=True,
)
return True
except subprocess.CalledProcessError:
return False
except FileNotFoundError:
return False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be a module-level function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be! This is the only place its used, so I think it's okay here, but happy to move if you like.

@hugovk
Copy link
Member Author

hugovk commented Apr 11, 2025

We could also do this via Python for (better) Windows support? sed is only used to remove some text. Low priority, though, as I'm really the only person affected by this.

Sure, feel free to do a followup, Windows support would be nice. There's at least two other sed calls as well.

@AA-Turner AA-Turner merged commit 4a5d2ca into python:main Apr 11, 2025
3 checks passed
@hugovk hugovk deleted the gnu-sed branch April 13, 2025 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants