Skip to content

gh-109118: Make comprehensions work within annotation scopes, but without inlining #118160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 28, 2024

Conversation

JelleZijlstra
Copy link
Member

@JelleZijlstra JelleZijlstra commented Apr 22, 2024

This is what I have so far to allow comprehensions within annotation scopes. The first commit changes the tests to what I think is the right behavior, and removes the SyntaxError on comprehensions in annotation scopes. These tests fail because of comprehension inlining; you can see that analogous tests work correctly with genexps (which are not inlined) but not with listcomps (which are). The second commit is an attempt to fix the tests, inspired by #104528, but it does not work yet.

cc @carljm

@carljm
Copy link
Member

carljm commented Apr 24, 2024

I looked at this a bit, and my initial conclusions aren't promising. I think this may not really be possible to support, at least not without extensive hackery.

If we consider this test case:

            class C[T]:
                T = "class"
                class Inner[U](make_base([T for _ in (1,)]), make_base(T)):
                    pass

The problem is that in the annotation scope of Inner, there is already a reference to T (in make_base(T)). The goal of this PR is to inline the comprehension [T for _ in (1,)] into that same scope, but somehow have the inlined T symbol behave differently from the existing T symbol in the same scope. Since all symbol information is tracked by name, that would require renaming the inlined T, somehow tracking that the renamed symbol is "really" named T, and extending that knowledge into the compiler. That sounds quite painful.

I think the more feasible options are either to disable comprehension inlining in annotation scopes, or to accept that the visibility effects of comprehension inlining leak here.

Given that comprehension inlining is usually transparent, and that complex code in annotation scopes is unlikely, I think just disabling comprehension inlining in this case is reasonable.

(If I had a time machine, I think I would inline comprehensions only in function scopes and nowhere else, as I'd initially proposed in the first version of PEP 709.)

@JelleZijlstra
Copy link
Member Author

Thanks. I can't think of a tractable solution either, so I pushed the non-inlining version.

@JelleZijlstra JelleZijlstra changed the title gh-109118: Attempt to make comprehensions work within annotation scopes gh-109118: Make comprehensions work within annotation scopes, but without inlining Apr 26, 2024
@JelleZijlstra JelleZijlstra marked this pull request as ready for review April 26, 2024 04:20
Copy link
Member

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

Co-authored-by: Carl Meyer <carl@oddbird.net>
@JelleZijlstra JelleZijlstra merged commit 2326d6c into python:main Apr 28, 2024
36 checks passed
@JelleZijlstra JelleZijlstra deleted the lambdaanno branch April 28, 2024 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants