Skip to content

gh-107265: Fix code_richcompare for ENTER_EXECUTOR case #108165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Aug 21, 2023
Merged

gh-107265: Fix code_richcompare for ENTER_EXECUTOR case #108165

merged 4 commits into from
Aug 21, 2023

Conversation

corona10
Copy link
Member

@corona10 corona10 commented Aug 20, 2023 •
edited
Loading

@corona10 corona10 requested a review from gvanrossum August 20, 2023 11:09
@corona10 corona10 requested a review from markshannon as a code owner August 20, 2023 11:09
@bedevere-bot bedevere-bot mentioned this pull request Aug 20, 2023
Open
10 tasks
@corona10 corona10 changed the title gh-107265: Fix code_richcompare to lookup actual opcode when ENTER_EX… gh-107265: Fix code_richcompare for ENTER_EXECUTOR case Aug 20, 2023
gvanrossum
gvanrossum reviewed Aug 20, 2023
View reviewed changes
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG, but could you add a test for this? You could add it to test_capi/test_misc.py in TestOptimizerAPI.

@corona10
Copy link
Member Author

corona10 commented Aug 21, 2023 •
edited
Loading

LG, but could you add a test for this? You could add it to test_capi/test_misc.py in TestOptimizerAPI.

Thanks, I added the test, and if I understand the intention correctly.
co_instr.op.arg and cp_instr.op.arg should also be updated.
If not, comparing co_instr.cache == cp_instr.cache will be failed anyway.

Please let me know if I understand wrongly.

@corona10
Copy link
Member Author

corona10 commented Aug 21, 2023 •
edited
Loading

pydoc test failure is unrelated to this PR, I checked it locally and asked to Discord too

gvanrossum
gvanrossum approved these changes Aug 21, 2023
View reviewed changes
Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the arg must also be restored, since it is overwritten (with the index of the executor to be used in this code object's array of executors). So it looks like writing the test was useful!

@corona10 corona10 enabled auto-merge (squash) August 21, 2023 05:03
@corona10 corona10 merged commit 4fdf3fd into python:main Aug 21, 2023
@corona10 corona10 deleted the gh-107265 branch August 21, 2023 05:50
markshannon
markshannon reviewed Aug 21, 2023
View reviewed changes
Objects/codeobject.c
if (co_instr.op.code == ENTER_EXECUTOR) {
const int exec_index = co_instr.op.arg;
_PyExecutorObject *exec = co->co_executors->executors[exec_index];
co_instr.op.code = exec->vm_data.opcode;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing two code objects must not modify the code object.

The interaction between executor, the specializer and instrumentation is subtle and likely to break without care.
This will leak executors, as it flip-flops between JUMP_BACKWARDS and ENTER_EXECUTOR, or worse if an optimizer assumes that a single instruction will only be seen once.

markshannon
markshannon reviewed Aug 21, 2023
View reviewed changes
Objects/codeobject.c
if (cp_instr.op.code == ENTER_EXECUTOR) {
const int exec_index = cp_instr.op.arg;
_PyExecutorObject *exec = cp->co_executors->executors[exec_index];
cp_instr.op.code = exec->vm_data.opcode;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise

@markshannon
Copy link
Member

Please revert this. It will leak, and may not be safe.

@markshannon
Copy link
Member

#101346

@gvanrossum
Copy link
Member

@corona10 We can fix this in the hash PR you are working on.

@corona10
Copy link
Member Author

@corona10 We can fix this in the hash PR you are working on.

Okay got it.

@JelleZijlstra JelleZijlstra mentioned this pull request Sep 8, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@markshannon markshannon markshannon left review comments

@gvanrossum gvanrossum gvanrossum approved these changes

Assignees
No one assigned
Labels
skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@corona10 @markshannon @gvanrossum @bedevere-bot