Skip to content

-Werror=unguarded-availability-new in sysconfig prevents building of watchdog #210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
asottile-sentry opened this issue Feb 6, 2024 · 5 comments · Fixed by #422
Closed

-Werror=unguarded-availability-new in sysconfig prevents building of watchdog #210

asottile-sentry opened this issue Feb 6, 2024 · 5 comments · Fixed by #422
Assignees
@charliermarsh
Labels
enhancement New feature or request

Comments

@asottile-sentry
Copy link

admittedly this is pointing out a real warning in watchdog -- but it makes it not installable using specifically these pythons:

wget https://github.com/indygreg/python-build-standalone/releases/download/20240107/cpython-3.12.1+20240107-x86_64-apple-darwin-pgo+lto-full.tar.zst
tar -xf cpython-3.12.1+20240107-x86_64-apple-darwin-pgo+lto-full.tar.zst
./python/install/bin/python3 -m pip wheel watchdog --no-deps

output:

...
      running build_ext
      building '_watchdog_fsevents' extension
      creating build/temp.macosx-10.9-x86_64-cpython-312
      creating build/temp.macosx-10.9-x86_64-cpython-312/src
      clang -fno-strict-overflow -DNDEBUG -g -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -fPIC -Werror=unguarded-availability-new -DWATCHDOG_VERSION_STRING=\"3.0.0\" -DWATCHDOG_VERSION_MAJOR=3 -DWATCHDOG_VERSION_MINOR=0 -DWATCHDOG_VERSION_BUILD=0 -I/private/tmp/y/qq/python/install/include/python3.12 -c src/watchdog_fsevents.c -o build/temp.macosx-10.9-x86_64-cpython-312/src/watchdog_fsevents.o -std=c99 -pedantic -Wall -Wextra -fPIC -Wno-nullability-completeness -Wno-nullability-extension -Wno-newline-eof -Wno-error=unused-command-line-argument
      src/watchdog_fsevents.c:191:25: error: 'kFSEventStreamEventFlagItemCloned' is only available on macOS 10.13 or newer [-Werror,-Wunguarded-availability-new]
      FLAG_PROPERTY(IsCloned, kFSEventStreamEventFlagItemCloned)
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      src/watchdog_fsevents.c:163:27: note: expanded from macro 'FLAG_PROPERTY'
              if (self->flags & flag) { \
                                ^~~~
      /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/FSEvents.framework/Headers/FSEvents.h:544:3: note: 'kFSEventStreamEventFlagItemCloned' has been marked as being introduced in macOS 10.13 here, but the deployment target is macOS 10.9.0
        kFSEventStreamEventFlagItemCloned __OSX_AVAILABLE_STARTING(__MAC_10_13, __IPHONE_11_0) = 0x00400000
        ^
      src/watchdog_fsevents.c:191:25: note: enclose 'kFSEventStreamEventFlagItemCloned' in a __builtin_available check to silence this warning
      FLAG_PROPERTY(IsCloned, kFSEventStreamEventFlagItemCloned)
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      src/watchdog_fsevents.c:163:27: note: expanded from macro 'FLAG_PROPERTY'
              if (self->flags & flag) { \
                                ^~~~
      src/watchdog_fsevents.c:596:40: error: 'kFSEventStreamCreateFlagUseExtendedData' is only available on macOS 10.13 or newer [-Werror,-Wunguarded-availability-new]
                                           | kFSEventStreamCreateFlagUseExtendedData
                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/FSEvents.framework/Headers/FSEvents.h:294:3: note: 'kFSEventStreamCreateFlagUseExtendedData' has been marked as being introduced in macOS 10.13 here, but the deployment target is macOS 10.9.0
        kFSEventStreamCreateFlagUseExtendedData __OSX_AVAILABLE_STARTING(__MAC_10_13, __IPHONE_11_0) = 0x00000040,
        ^
      src/watchdog_fsevents.c:596:40: note: enclose 'kFSEventStreamCreateFlagUseExtendedData' in a __builtin_available check to silence this warning
                                           | kFSEventStreamCreateFlagUseExtendedData
                                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      2 errors generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for watchdog
Failed to build watchdog
ERROR: Failed to build one or more wheels

[notice] A new release of pip is available: 23.3.2 -> 24.0
[notice] To update, run: /private/tmp/y/qq/python/install/bin/python3 -m pip install --upgrade pip

a workaround is to remove that flag from _sysconfigdata__darwin_darwin.py after unpacking
@asottile-sentry asottile-sentry mentioned this issue Feb 6, 2024
Closed
@indygreg
Copy link
Collaborator

Assuming you are trying to build watchdog binaries that work on macOS <10.13, this compiler error is absolutely finding a bug in watchdog where it is failing to conditionally guard symbols not available in older macOS versions!

On one hand I consider this a feature: we're preventing bugs!

On the other, we are inserting -Werror=unguarded-availability-new into the compiler flags used to build CPython itself and CPython's build system exposes that as a CFLAG that should be used to compile extension modules. This propagation is definitely an unintended side-effect. (We added this compiler flag as a guardrail to help ensure our CPython builds don't reference symbols that aren't available in the target macOS version. This is a best practice when building portable macOS binaries. TBH I'm kind of surprised the CPython builds don't do this. I can make a case that extension module builds should be using this flag to prevent end-users from building wheels that don't conform to stated ABI compatibility.)

I think it is reasonable to strip this - and other custom - compiler flag from the sysconfig data to maintain compatibility with CPython.

@indygreg indygreg added the enhancement New feature or request label Feb 24, 2024
@charliermarsh
Copy link
Member

Is this roughly: add some additional replacements to hack_sysconfig.py (in build_cpython.sh) to strip out -Werror=unguarded-availability-new?

(Do we also need to strip from CPPFLAGS?)

@indygreg
Copy link
Collaborator

indygreg commented Apr 1, 2024

Yeah, this is a modification to hack_sysconfig.py to strip flags/words from variables.

It looks like the flag is present in the following:

  • CFLAGS
  • CONFIGURE_CFLAGS
  • CONFIGURE_CPPFLAGS
  • CONFIG_ARGS
  • CPPFLAGS
  • PY_BUILTIN_MODULE_CFLAGS
  • PY_CFLAGS
  • PY_CORE_CFLAGS
  • PY_CPPFLAGS
  • PY_STDMODULE_CFLAGS

I'd have to look at which variables are used by Python build tools to propagate compiler flags: those are the ones we want to modify.

I'm hesitant to modify flags that denote how the Python distribution itself was built: that would constitute lying. But if a variable is overloaded to both reflect what was used and what to use, then I suppose we must modify it.

@frou
Copy link

frou commented Dec 16, 2024

I'm encountering this error when trying to use uv-managed Python to develop Google Cloud Functions (functions-framework depends on watchdog)

@zanieb zanieb mentioned this issue Dec 16, 2024
Merged
@charliermarsh
Copy link
Member

At least in setuptools I see:

if 'CC' in os.environ:
    newcc = os.environ['CC']
    if 'LDSHARED' not in os.environ and ldshared.startswith(cc):
        # If CC is overridden, use that as the default
        #       command for LDSHARED as well
        ldshared = newcc + ldshared[len(cc) :]
    cc = newcc
if 'CXX' in os.environ:
    cxx = os.environ['CXX']
if 'LDSHARED' in os.environ:
    ldshared = os.environ['LDSHARED']
if 'CPP' in os.environ:
    cpp = os.environ['CPP']
else:
    cpp = cc + " -E"  # not always
if 'LDFLAGS' in os.environ:
    ldshared = ldshared + ' ' + os.environ['LDFLAGS']
if 'CFLAGS' in os.environ:
    cflags = cflags + ' ' + os.environ['CFLAGS']
    ldshared = ldshared + ' ' + os.environ['CFLAGS']
if 'CPPFLAGS' in os.environ:
    cpp = cpp + ' ' + os.environ['CPPFLAGS']
    cflags = cflags + ' ' + os.environ['CPPFLAGS']
    ldshared = ldshared + ' ' + os.environ['CPPFLAGS']
if 'AR' in os.environ:
    ar = os.environ['AR']
if 'ARFLAGS' in os.environ:
    archiver = ar + ' ' + os.environ['ARFLAGS']
else:
    archiver = ar + ' ' + ar_flags

So, that would be CFLAGS and CPPFLAGS?

@charliermarsh charliermarsh self-assigned this Dec 17, 2024
@charliermarsh charliermarsh mentioned this issue Dec 17, 2024
Merged
charliermarsh added a commit that referenced this issue Dec 17, 2024
@charliermarsh
Remove -Werror=unguarded-availability-new from sysconfig data (#422)
12eeb3e 
## Summary

Per #210, we now strip `-Werror=unguarded-availability-new` from `CFLAGS` and `CPPFLAGS` (but not, e.g., `CONFIGURE_CFLAGS`).

Closes #210.
This was referenced Dec 20, 2024
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@charliermarsh charliermarsh

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove -Werror=unguarded-availability-new from sysconfig data astral-sh/python-build-standalone
4 participants
@frou @indygreg @charliermarsh @asottile-sentry