Skip to content

gh-124722: Fix leak in test_detach_materialized_dict_no_memory #124769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Sep 30, 2024
Merged

gh-124722: Fix leak in test_detach_materialized_dict_no_memory #124769

merged 2 commits into from
Sep 30, 2024

Conversation

sobolevn
Copy link
Member

@sobolevn sobolevn commented Sep 29, 2024 •
edited by bedevere-app bot
Loading

The leak was in setting _testcapi.set_nomemory(0, 1), in other places it is only used together with assert_python_ok

I also marked this test as cpython_only and I also skip it if _testcapi is not present.

Local run:

» ./python.exe -m test test_class -m test_detach_materialized_dict_no_memory -R 3:3
Using random seed: 365282797
0:00:00 load avg: 1.55 Run 1 test sequentially in a single process
0:00:00 load avg: 1.55 [1/1] test_class
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
XX. ...

== Tests result: SUCCESS ==

1 test OK.

Total duration: 403 ms
Total tests: run=1 (filtered)
Total test files: run=1/1 (filtered)
Result: SUCCESS

@bedevere-app bedevere-app bot mentioned this pull request Sep 29, 2024
Closed
@bedevere-app bedevere-app bot added the tests Tests in the Lib/test dir label Sep 29, 2024
@sobolevn sobolevn added skip news needs backport to 3.13 bugs and security fixes labels Sep 29, 2024
ZeroIntensity
ZeroIntensity approved these changes Sep 29, 2024
View reviewed changes
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks fine, but I'm worried about what problems that _testcapi.set_nomemory is causing. Could you elaborate on that? (This seems like more of a bandaid than an actual fix for that function.)

Eclips4
Eclips4 approved these changes Sep 29, 2024
View reviewed changes
Copy link
Member

@Eclips4 Eclips4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I thought of the same approach.

@sobolevn
Copy link
Member Author

@ZeroIntensity please, take a look at fm_setup_hooks, probably something there is not cleaned up.

The more complex fix would be:

  • make sure that fm_remove_hooks completely disables all modifications done in fm_setup_hooks
  • create a context manager to automatically clean up this context in tests

But, this change would be much more complex, that's why I went with the simplier (and already existing) fix instead.

@ZeroIntensity
Copy link
Member

Yeah, it's good to get this in sooner. I'll look into fixing set_nomemory, but it's odd because as far as I can see fm_setup_hooks makes no additional allocations, so I'm not exactly sure what could be getting leaked.

@zware
Copy link
Member

zware commented Sep 29, 2024

!buildbot .RHEL8 Refleak.

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @zware for commit e7a409b 🤖

The command will test the builders whose names match following regular expression: .*RHEL8 Refleak.*

The builders matched are:

  • s390x RHEL8 Refleaks PR
  • AMD64 RHEL8 Refleaks PR
  • aarch64 RHEL8 Refleaks PR
  • PPC64LE RHEL8 Refleaks PR

@zware
Copy link
Member

zware commented Sep 30, 2024

The buildbots look happy with this, so I'm going ahead with the merge to get them cleaned up. @markshannon, it could still use a post-merge OK from you to confirm that it's still testing what you wanted to test.

@zware zware merged commit 6f4d64b into python:main Sep 30, 2024
39 checks passed
@miss-islington-app
Copy link

Thanks @sobolevn for the PR, and @zware for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 30, 2024
@sobolevn @miss-islington
pythongh-124722: Fix leak in test_detach_materialized_dict_no_memory (
73d7d1b 
pythonGH-124769)

(cherry picked from commit 6f4d64b)

Co-authored-by: sobolevn <mail@sobolevn.me>
@bedevere-app
Copy link

bedevere-app bot commented Sep 30, 2024

GH-124777 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Sep 30, 2024
Yhg1s pushed a commit that referenced this pull request Sep 30, 2024
@miss-islington @sobolevn
[3.13] gh-124722: Fix leak in `test_detach_materialized_dict_no_memor…
ce0eaa6 
…y` (GH-124769) (#124777)

gh-124722: Fix leak in `test_detach_materialized_dict_no_memory` (GH-124769)
(cherry picked from commit 6f4d64b)

Co-authored-by: sobolevn <mail@sobolevn.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ZeroIntensity ZeroIntensity ZeroIntensity approved these changes

@Eclips4 Eclips4 Eclips4 approved these changes

@DinoV DinoV Awaiting requested review from DinoV

@markshannon markshannon Awaiting requested review from markshannon

Assignees
No one assigned
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sobolevn @ZeroIntensity @zware @bedevere-bot @Eclips4