Skip to content

gh-114828: Fix __class__ in class-scope inlined comprehensions #115139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 7, 2024

Conversation

carljm
Copy link
Member

@carljm carljm commented Feb 7, 2024

The behavior of __class__ as a cell / freevar is special-cased in various places in the symbol table and the compiler; it needs a bit more special-casing in the comprehension-inlining implementation. This fixes some fuzzer-discovered compilation crashes in code that would rarely if ever make sense to write (using super in a class-body comprehension.)

@ghost
Copy link

ghost commented Feb 7, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@carljm carljm added the needs backport to 3.12 only security fixes label Feb 7, 2024
Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Agree this seems rather unlikely to come up in practice.

@carljm carljm enabled auto-merge (squash) February 7, 2024 16:47
@carljm carljm merged commit fedbf77 into python:main Feb 7, 2024
@miss-islington-app
Copy link

Thanks @carljm for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Feb 7, 2024
…ythonGH-115139)

(cherry picked from commit fedbf77)

Co-authored-by: Carl Meyer <carl@oddbird.net>
@bedevere-app
Copy link

bedevere-app bot commented Feb 7, 2024

GH-115140 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Feb 7, 2024
@carljm carljm deleted the compdunderclass branch February 7, 2024 16:59
carljm added a commit that referenced this pull request Feb 7, 2024
…GH-115139) (#115140)

gh-114828: Fix __class__ in class-scope inlined comprehensions (GH-115139)
(cherry picked from commit fedbf77)

Co-authored-by: Carl Meyer <carl@oddbird.net>
@AlexWaygood
Copy link
Member

There's a bunch of compiler warnings in Include/internal/pycore_symtable.h in the "files changed" tab of this PR -- were they caused by the PR?

@JelleZijlstra
Copy link
Member

I saw that but didn't comment because they looked obviously unrelated. I don't know what made them show up though; @carljm do you have any ideas?

@carljm
Copy link
Member Author

carljm commented Feb 7, 2024

No, the warnings look very odd to me (2 is not negative!), and I don't see how they could possibly be caused by this PR, either.

@AlexWaygood
Copy link
Member

I thought they looked unrelated too, but they showed up on the 3.12 backport PR as well

@carljm
Copy link
Member Author

carljm commented Feb 7, 2024

Ok, after further exploration, this warning is related to the PR, and in fact reveals a bug in the PR. The expression ~DEF_FREE macro-expands to ~2<<4, and the precedence here is actually (~2)<<4 :(

It just happens that this doesn't cause any actual problem, because ~2<<4 is ...1010000 whereas ~(2<<4) is ...1011111 and those lower four bits correspond to DEF_GLOBAL, DEF_LOCAL, DEF_PARAM, and DEF_NONLOCAL, none of which would ever be set for a DEF_FREE __class__ anyway. So wrongly zeroing them out doesn't change anything.

This is a good reminder to always parenthesize your macro definitions! The best fix IMO would be to parenthesize all of those defines (e.g. #define DEF_FREE (2<<4)) rather than just parenthesizing my particular usage site.

The error is mostly confusing because it points to the wrong code location (the macro definition instead of usage site), but digging into the actual compiler logs, the warning points to both locations.

Thanks @AlexWaygood for pushing me to dig further into this!

I'll push a fix PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants