-
-
Notifications
You must be signed in to change notification settings - Fork 32.1k
[3.14] GH-135171: Revert async generator expressions behavior #135352
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 3.14
Are you sure you want to change the base?
[3.14] GH-135171: Revert async generator expressions behavior #135352
Conversation
@@ -4544,9 +4544,9 @@ codegen_async_comprehension_generator(compiler *c, location loc, | |||
else { | |||
/* Sub-iter - calculate on the fly */ | |||
VISIT(c, expr, gen->iter); | |||
ADDOP(c, LOC(gen->iter), GET_AITER); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, is this change necessary? I think the test passes without it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think so. There was a bug in previous commit. I've reverted related logic in codegen.c
.
I've reverted all generator expressions related changes in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
@markshannon Do you want to take a look at this fix? Do you have time to do? |
@efimov-mikhail are you sure that this isn't adding back too many We should have tests for generator expressions and list comprehensions, both sync and async that |
I've a had a quick look at the generated bytecode and looks like this PR is OK. @Yhg1s if you need this in ASAP, I think it's good to merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like tests to check that __iter__
/__aiter__
is only called once in inlined comprehensions, but that can wait for another PR.
Yes, changes in these two PRs are very similar. Although, there aren't exactly the same. But on 3.13 with #135390 being merged there is a different solution (from #125846). So, I have a question why we don't revert #125846? |
Was not it reverted as the result of all these changes? |
It was reverted here. |
Thank you, this is a great observation. |
I don't think I was aware of bytecode changes in 31.1. I tested the stdlib's compiled bytecode (including Lib/test) and saw no differences between 13.0, 13.3 and my rollback (and saw many changes between 13.3, 13.4 and 13.4 + the other fix-forward, but I didn't bother trying with this one applied). In any case it's too late to fix a change in bytecode that slipped into 13.1, if 13.1-13.3 are all the same we should stick with that. |
It is not only change in bytecode generation. It is a behavior change in comparison with 3.12, 3.13.0 and 3.13.4. |
To be precise, there is a bytecode change: -> % ./python
Python 3.13.0 (tags/v3.13.0:60403a5409, Jun 11 2025, 19:23:44) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dis
>>> dis.dis('(x for x in range(10))')
0 RESUME 0
1 LOAD_CONST 0 (<code object <genexpr> at 0x7fa6c309f450, file "<dis>", line 1>)
MAKE_FUNCTION
LOAD_NAME 0 (range)
PUSH_NULL
LOAD_CONST 1 (10)
CALL 1
GET_ITER
CALL 0
RETURN_VALUE
Disassembly of <code object <genexpr> at 0x7fa6c309f450, file "<dis>", line 1>:
1 RETURN_GENERATOR
POP_TOP
L1: RESUME 0
LOAD_FAST 0 (.0)
L2: FOR_ITER 6 (to L3)
STORE_FAST_LOAD_FAST 17 (x, x)
YIELD_VALUE 0
RESUME 5
POP_TOP
JUMP_BACKWARD 8 (to L2)
L3: END_FOR
POP_TOP
RETURN_CONST 0 (None)
-- L4: CALL_INTRINSIC_1 3 (INTRINSIC_STOPITERATION_ERROR)
RERAISE 1
ExceptionTable:
L1 to L4 -> L4 [0] lasti and -> % ./python
Python 3.13.1 (tags/v3.13.1:0671451779, Jun 11 2025, 19:25:28) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dis
>>> dis.dis('(x for x in range(10))')
0 RESUME 0
1 LOAD_CONST 0 (<code object <genexpr> at 0x7f4332397bc0, file "<dis>", line 1>)
MAKE_FUNCTION
LOAD_NAME 0 (range)
PUSH_NULL
LOAD_CONST 1 (10)
CALL 1
GET_ITER
CALL 0
RETURN_VALUE
Disassembly of <code object <genexpr> at 0x7f4332397bc0, file "<dis>", line 1>:
1 RETURN_GENERATOR
POP_TOP
L1: RESUME 0
LOAD_FAST 0 (.0)
GET_ITER
L2: FOR_ITER 6 (to L3)
STORE_FAST_LOAD_FAST 17 (x, x)
YIELD_VALUE 0
RESUME 5
POP_TOP
JUMP_BACKWARD 8 (to L2)
L3: END_FOR
POP_TOP
RETURN_CONST 0 (None)
-- L4: CALL_INTRINSIC_1 3 (INTRINSIC_STOPITERATION_ERROR)
RERAISE 1
ExceptionTable:
L1 to L4 -> L4 [0] lasti
>>> You can see additional GET_ITER instruction before FOR_ITER. IMO, it's better to revert this change and fix regression in #127682. |
I've created new PR for 3.13: #135403 |
There were 3 tests related to this topic, and I've added the fourth. |
This PR could be a part of #135225.
Changes in behavior of async generator expressions are also reverted.
@serhiy-storchaka @markshannon @Yhg1s