GH-132554: Fix tier2 FOR_ITER implementation and optimizations
#135137
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can you check out the Windows JIT failures please? I'll review the PR after that. |
brandtbucher
approved these changes
Jun 5, 2025
Include/internal/pycore_ceval.h
Outdated
| @@ -353,7 +353,8 @@ PyAPI_FUNC(_PyStackRef) _PyFloat_FromDouble_ConsumeInputs(_PyStackRef left, _PyS | |||
| extern int _PyRunRemoteDebugger(PyThreadState *tstate); | |||
| #endif | |||
|
|
|||
| _PyStackRef _PyForIter_NextWithIndex(PyObject *seq, _PyStackRef index); | |||
| _PyStackRef | |||
There was a problem hiding this comment.
Suggested change
| _PyStackRef | |
| PyAPI_FUNC(_PyStackRef) |
Python/optimizer_bytecodes.c
Outdated
| op(_GET_ITER, (iterable -- iter, index_or_null)) { | ||
| if (sym_matches_type(iterable, &PyTuple_Type) || sym_matches_type(iterable, &PyList_Type)) { | ||
| iter = iterable; | ||
| index_or_null = sym_new_type(ctx, &PyLong_Type); |
There was a problem hiding this comment.
Hm, this is sort of weird. We don't have a symbol for "unboxed int" in the JIT, but it really doesn't feel correct to type this as int. Maybe leave as unknown and we can update our lattice with unboxed C types later? It's not like this information is being used yet, anyways.
Suggested change
| index_or_null = sym_new_type(ctx, &PyLong_Type); | |
| index_or_null = sym_new_unknown(ctx); |
| } | ||
| next = PyStackRef_FromPyObjectSteal(next_o); | ||
| JUMPBY(oparg + 1); |
There was a problem hiding this comment.
Can you add back the comment that this is skipping the END_FOR?
Suggested change
| JUMPBY(oparg + 1); | |
| // Jump forward by oparg, then skip the following END_FOR: | |
| JUMPBY(oparg + 1); |
| } | ||
| JUMPBY(oparg + 1); |
There was a problem hiding this comment.
Suggested change
| JUMPBY(oparg + 1); | |
| // Jump forward by oparg, then skip the following END_FOR: | |
| JUMPBY(oparg + 1); |
Python/ceval.c
Outdated
| _PyErr_Clear(tstate); | ||
| } | ||
| else { | ||
| return PyStackRef_ERROR; |
There was a problem hiding this comment.
Suggested change
| return PyStackRef_ERROR; | |
| return PyStackRef_ERROR; |
|
Confirmed that this fixed the |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When adding virtual iterators, the tier 1 and tier 2 implementations of
FOR_ITERdiverged. I've already fixed a problem where the instrumentedFOR_ITERdiffered from the normal one.To prevent these problems happening again, this PR factors out the majority of
FOR_ITERinto a helper function for the 3 versions ofFOR_ITERto share.I've also added
PyStackRef_ERRORto distinguish between errors and no result and remove the need for an additional out parameter for the helper function.Also fixes a bug in the code generator where there are three or more output values, one is an unchanged input, one is a changed input and one is undefined.