Skip to content

[MNT]: macos-latest memory leak over threshold #22959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
andrew-fennell opened this issue May 2, 2022 · 4 comments · Fixed by #22961
Closed

[MNT]: macos-latest memory leak over threshold #22959

andrew-fennell opened this issue May 2, 2022 · 4 comments · Fixed by #22961
Labels
Maintenance
Milestone
v3.5.2

Comments

@andrew-fennell
Copy link
Contributor

Summary

macos-latest test is failing. Memory leakage is slightly over the threshold.

Failing test:
lib/matplotlib/tests/test_backends_interactive.py:559: AssertionError

Exact assertion error:
assert growth <= acceptable_memory_leakage
assert 30457856 <= 30000000

Example of test failing: macos-latest failing

Proposed fix

Raise threshold.
@andrew-fennell
Copy link
Contributor Author

Threshold can be raised by 1000000 (double 457856) to account for fluctuation.

@andrew-fennell andrew-fennell mentioned this issue May 2, 2022
Merged
2 tasks
@QuLogic QuLogic added this to the v3.5.2 milestone May 2, 2022
@andrew-fennell
Copy link
Contributor Author

@oscargus I think my original solution (#22961) didn't cover the whole problem. The original examples of this test failing was specifically with macosx and the threshold was less than 1000000 off.

The test that is failing:

matplotlib/lib/matplotlib/tests/test_backends_interactive.py

Lines 543 to 556 in 2e70254

@pytest.mark.parametrize("env", _get_testable_interactive_backends())
@pytest.mark.parametrize("time_mem", [(0.0, 2_000_000), (0.1, 30_000_000)])
def test_figure_leak_20490(env, time_mem):
pytest.importorskip("psutil", reason="psutil needed to run this test")
# We haven't yet directly identified the leaks so test with a memory growth
# threshold.
pause_time, acceptable_memory_leakage = time_mem
if env["MPLBACKEND"] == "macosx":
acceptable_memory_leakage += 11_000_000
result = _run_helper(
_test_figure_leak, str(pause_time), timeout=_test_timeout, **env
)

Here is an example of this tests still failing: https://github.com/matplotlib/matplotlib/runs/6246645151?check_suite_focus=true

Should I make this adjustment?
@pytest.mark.parametrize("time_mem", [(0.0, 2_000_000), (0.1, 35_000_000)])

@andrew-fennell
Copy link
Contributor Author

Talked about in dev call: https://hackmd.io/jd_7FjxNQ4y7XgNknvmvGQ#memory-issue

@tacaswell
Copy link
Member

Looking at this in detail, I was wrong about how the tests are run. They are in their own process so the inter-test interaction should not be a problem. It seems there is just a lot of variation, but we do not know why.

I think that in addition to the OSX backend on osx, we also need to increase the threshold when running with tkagg on osx. I'm going to open a PR with that stage and also write up an issue starting the discussion about how we may need to find a more stable way to benchmark these memory issues.

tacaswell added a commit to tacaswell/matplotlib that referenced this issue May 6, 2022
@tacaswell
CI: bump test limit from tkagg on osx
68553ac 
Extension of matplotlib#22959.
tacaswell added a commit to tacaswell/matplotlib that referenced this issue May 6, 2022
@tacaswell
CI: bump test limit from tkagg on osx
cfc8a99 
Extension of matplotlib#22959.
@tacaswell tacaswell mentioned this issue May 6, 2022
Merged
2 tasks
tacaswell added a commit to tacaswell/matplotlib that referenced this issue May 6, 2022
@tacaswell
CI: bump test limit from tkagg on osx
4004d1f 
Extension of matplotlib#22959.
@tacaswell tacaswell mentioned this issue May 6, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Maintenance
Projects
None yet
Milestone
v3.5.2
Development

Successfully merging a pull request may close this issue.

Raised macosx memory leak threshold andrew-fennell/matplotlib
3 participants
@tacaswell @QuLogic @andrew-fennell