Skip to content

ENH: Add version check for mac sdk version #17956

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 25, 2020

Conversation

dopplershift
Copy link
Contributor

PR Summary

This specifically lists the macOS SDK version (currently 10.11) needed to build the osx backend. It also turns on compiler warnings for when code uses features newer than the minimum version (and makes warnings errors for this file).

This should hopefully help CI catch when we start using API's past our minimum desired version. It's currently set to 10.9 specifically to see it now fail.

Questions:

  • Where in the docs should we document that we only support macOS >= 10.11?
  • By having the compiler flag set the version, we override anywhere else right now using e.g. MACOS_DEPLOYMENT_TARGET. The problem right now is that most places (default Python builds, etc.) are using 10.9, so this has to be set somewhere for things to build properly out of the box.

PR Checklist

  • Code is Flake 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/next_api_changes/* if API changed in a backward-incompatible way

This specifically lists the macOS SDK version (currently 10.11) needed
to build the osx backend. It also turns on compiler warnings for when
code uses features newer than the minimum version (and makes warnings
errors for this file).
@QuLogic
Copy link
Member

QuLogic commented Jul 18, 2020

By having the compiler flag set the version, we override anywhere else right now using e.g. MACOS_DEPLOYMENT_TARGET. The problem right now is that most places (default Python builds, etc.) are using 10.9, so this has to be set somewhere for things to build properly out of the box.

You could do this only if, e.g., CI environment variable is set.

@dopplershift
Copy link
Contributor Author

You could do this only if, e.g., CI environment variable is set.

But this would mean any Mac user trying to build would have failures unless they set the variable.

@dopplershift
Copy link
Contributor Author

Fundamentally, the problem we have right now is we have merged API calls that only work on macOS >= 10.11, but by default many places (including python.org) are targeting macOS 10.9.

@reneeotten
Copy link

I am maintaining the matplotlib port on MacPorts - with the version update to 3.3.0 we see indeed all builds for macOS < 10.11 fail because of the recent changes:

src/_macosx.m:1213:30: error: property 'maximumNumberOfLines' not found on object of type 'NSTextContainer *'
src/_macosx.m:1214:30: error: property 'lineBreakMode' not found on object of type 'NSTextContainer *'

Instead of requiring macOS 10.11 or above and documenting that (as seems the proposition of this PR), can backwards-compatibility be restored/introduced for earlier versions (it worked fine with matplotlib v3.2.2).

@dopplershift
Copy link
Contributor Author

@reneeotten The >=10.11 APIs were added in #16710 to fix a real limitation in the OSX backend. What's the argument for supporting 10.9 (seven years old) vs. 10.11 (5 years old), when the latter still supports systems 12 years old at this point?

One potential option here is to add the @available checks (as pointed out by the warnings enabled in this PR). This would allow users on 10.11 to get the benefit of the change but keep things buildable for 10.9. The downside is any package that's built with a target of 10.9, regardless of the version in the actual runtime environment, would have the broken behavior. I think anyways? I don't have ready access to any of these older OSes so I'm really trying to infer from web docs.

Don't unconditionally use the newer properties, but check the available
version. This allows us to continue to target 10.9 by default and
eliminate any hard-coded deployment version.
@dopplershift
Copy link
Contributor Author

Ok, so on the basis of:

I've gone ahead and eliminated the hard-coded version check, and instead surrounded those properties with a guard for macOS 10.11. This gives us back the flexibility of the what the deployment target is. The effect will be that on versions < 10.11, we don't use those properties; I assume then that things won't work quite as well, but won't be completely busted.

@reneeotten
Copy link

thanks for the response @dopplershift!

@reneeotten The >=10.11 APIs were added in #16710 to fix a real limitation in the OSX backend. What's the argument for supporting 10.9 (seven years old) vs. 10.11 (5 years old), when the latter still supports systems 12 years old at this point?

I do see your point, however MacPorts just tries to support older OSes as much as possible. I was kind of surprised about the breakage, especially since it wasn't documented anywhere (at least I think so).

I will add your patch and see how that goes on older versions of macOS. Even if that has the effect that the scaling of the messagebox isn't perfect, I'd think being able to compile/use the MacOSX is still beneficial.

We have other deprecated constants that flags warnings in CI.
Unfortunately, the deprecations occur in the same version when the new
names are introduced, and we can't rely on those being available yet.
@dopplershift
Copy link
Contributor Author

@reneeotten We had no idea, the APIs snuck in and for some reason by default the compilers don't warn about it. IMO I think we'll be leaving support for the older OSes for now since just guarding the access looks sufficient.

@reneeotten
Copy link

reneeotten commented Jul 19, 2020

@reneeotten We had no idea, the APIs snuck in and for some reason by default the compilers don't warn about it. IMO I think we'll be leaving support for the older OSes for now since just guarding the access looks sufficient.

the patch doesn't seem to work... now it still fails on macOS 10.11 with:

building 'matplotlib.backends._macosx' extension
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch x86_64 -isysroot/ -I/opt/local/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/_macosx.m -o build/temp.macosx-10.11-x86_64-3.7/src/_macosx.o -fvisibility=hidden -flto
src/_macosx.m:1211:9: error: unexpected '@' in program
    if (@available(macOS 10.11, *)) {
        ^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1

and below macOS 10.11 with:

building 'matplotlib.backends._macosx' extension
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch x86_64 -isysroot/ -I/opt/local/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c src/_macosx.m -o build/temp.macosx-10.8-x86_64-3.8/src/_macosx.o -fvisibility=hidden -flto
src/_macosx.m:1211:9: error: unexpected '@' in program
    if (@available(macOS 10.11, *)) {
        ^
src/_macosx.m:1212:34: error: property 'maximumNumberOfLines' not found on object of type 'NSTextContainer *'
        messagebox.textContainer.maximumNumberOfLines = 2;
                                 ^
src/_macosx.m:1213:34: error: property 'lineBreakMode' not found on object of type 'NSTextContainer *'
        messagebox.textContainer.lineBreakMode = NSLineBreakByTruncatingTail;
                                 ^
3 errors generated.
error: command '/usr/bin/clang' failed with exit status 1

@tacaswell
Copy link
Member

@reneeotten At the risk of asking a very obvious question, are you sure you patched it in the right place? The line number do not quite match up.

/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch x86_64 -isysroot/ -I/opt/local/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c src/_macosx.m -o build/temp.macosx-10.8-x86_64-3.8/src/_macosx.o -fvisibility=hidden -flto
src/_macosx.m:1211:9: error: unexpected '@' in program

reads to me like clang is saying that this is invalid syntax? It seems to be building on our CI as well..

Copy link
Member

@tacaswell tacaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anyone can merge once we are confident that this patch actually builds everywhere.

@reneeotten
Copy link

reneeotten commented Jul 19, 2020

@reneeotten At the risk of asking a very obvious question, are you sure you patched it in the right place? The line number do not quite match up.

@tacaswell Yes, I am sure. The line numbers are different compared to what is shown in this PR since that originates from the current master; it's correct if you look at the released version.

reads to me like clang is saying that this is invalid syntax? It seems to be building on our CI as well..

The CI seems to be using gcc, so it it's possible that this is only an issue with clang - but still something that should likely be fixed. If not done already (I didn't check all the CI runs), it might be good to test with clang as well.

[edit: see the comment by @ryandesign about the patch suggested here that I committed to MacPorts. I am definitely out of my comfort zone with this, but I'll probably try the suggestion he made and see if that works.]

@QuLogic
Copy link
Member

QuLogic commented Jul 20, 2020

The CI seems to be using gcc, so it it's possible that this is only an issue with clang - but still something that should likely be fixed. If not done already (I didn't check all the CI runs), it might be good to test with clang as well.

macOS always uses clang; its gcc is a fake that just aliases clang (unless you go through a lot of trouble to install it manually.)

@dopplershift
Copy link
Contributor Author

So based on the links from @reneeotten, it looks like you just can't use old compilers (XCode < 9) and have access to @available. XCode 9 was released September 2017 and is available everywhere we're building regularly. I'm inclined to merge this and call it good.

@tacaswell
Copy link
Member

@reneeotten are you happy enough to patch the code out of existence on the older systems?

@reneeotten
Copy link

@reneeotten are you happy enough to patch the code out of existence on the older systems?

thanks @tacaswell and @dopplershift - I have patched out the code for older systems already and that seems to do the trick. So yes, for now we're all good. I suppose that at some point if/when there are more changes in the MacOSX backend that are incompatible with older systems (ideally indicating the minimum supported macOS version somewhere), we'll just have to switch off compilation of that backend completely and hopefully the Agg backend -or one of the others- will still work for these older systems.

@dstansby
Copy link
Member

All looks good so I'll merge this. Have opened #18066 as it seems like we are missing minimum supported OSX version documentation.

@dstansby dstansby merged commit d53b16c into matplotlib:master Jul 25, 2020
meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request Jul 25, 2020
timhoffm added a commit that referenced this pull request Jul 25, 2020
…956-on-v3.3.x

Backport PR #17956 on branch v3.3.x (ENH: Add version check for mac sdk version)
@dopplershift dopplershift deleted the mac-version-check branch July 25, 2020 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants