Skip to content

gh-124872: Replace enter/exit events with "switched" #125532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 16, 2024

Conversation

Eclips4
Copy link
Member

@Eclips4 Eclips4 commented Oct 15, 2024

@Eclips4
Copy link
Member Author

Eclips4 commented Oct 15, 2024

With that PR, test_capi is no longer leaking:

admin@Admins-MacBook-Air ~/p/cpython (revert-of-revert)> ./python.exe -m test -R 3:3 test_capi
Using random seed: 992095454
0:00:00 load avg: 26.10 Run 1 test sequentially in a single process
0:00:00 load avg: 26.10 [1/1] test_capi
beginning 6 repetitions. Showing number of leaks (. for 0 or less, X for 10 or more)
123:456
XX. ...
test_capi passed in 43.3 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 43.4 sec
Total tests: run=985 skipped=67
Total test files: run=1/1
Result: SUCCESS

@vstinner
Copy link
Member

gh-125512: Revert #125513 and fix reference cycle #125532

It's not a revert, but a revert of a revert :-D The PR title should be gh-124872: Replace enter/exit events with "switched.

@vstinner
Copy link
Member

I suggest to wait after Python 3.14.0 alpha1 release to retry this change.

@Eclips4 Eclips4 changed the title gh-125512: Revert #125513 and fix reference cycle gh-124872: Replace enter/exit events with "switched" Oct 15, 2024
@1st1
Copy link
Member

1st1 commented Oct 15, 2024

I suggest to wait after Python 3.14.0 alpha1 release to retry this change.

Why? We can debug and fix the ref leak.

@rhansen Please build CPython with ./configure --with-pydebug and run the tests with -R3:3. That will take much longer so I suggest just running asyncio & contextvars & decimal tests with this flag.

@vstinner
Copy link
Member

vstinner commented Oct 15, 2024

me:

I suggest to wait after Python 3.14.0 alpha1 release to retry this change.

@1st1:

Why?

It's just that there are many broken buildbots, I think that this change can wait for 3.14.0a2.

@Eclips4
Copy link
Member Author

Eclips4 commented Oct 15, 2024

FYI, I ran the test suite in hunt refleaks mode locally and it succeeded:

== Tests result: SUCCESS ==

25 tests skipped:
    test.test_asyncio.test_windows_events
    test.test_asyncio.test_windows_utils test.test_gdb.test_backtrace
    test.test_gdb.test_cfunction test.test_gdb.test_cfunction_full
    test.test_gdb.test_misc test.test_gdb.test_pretty_print
    test.test_multiprocessing_fork.test_manager
    test.test_multiprocessing_fork.test_misc
    test.test_multiprocessing_fork.test_processes
    test.test_multiprocessing_fork.test_threads test_android
    test_dbm_gnu test_devpoll test_epoll test_free_threading
    test_launcher test_msvcrt test_perf_profiler test_perfmaps
    test_startfile test_winapi test_winconsoleio test_winreg test_wmi

11 tests skipped (resource denied):
    test_curses test_peg_generator test_pyrepl test_smtpnet
    test_socketserver test_tkinter test_ttk test_urllib2net
    test_urllibnet test_winsound test_zipfile64

442 tests OK.

Total duration: 20 min 5 sec
Total tests: run=44,347 skipped=2,182
Total test files: run=467/478 skipped=25 resource_denied=11
Result: SUCCESS

@1st1
Copy link
Member

1st1 commented Oct 15, 2024

@Eclips4 thanks for fixing the leak!

@vstinner

It's just that there are many broken buildbots, I think that this change can wait for 3.14.0a2.

I sympathize, really sorry about breaking them and causing pain :/ Can we try landing this PR since it appears it should solve the issue? I'm afraid if we wait on this for a month to get it merged it will go stale and we just forget. I'd try again and if it fails we wait? If however you really don't want to do it, then fine.

@ambv
Copy link
Contributor

ambv commented Oct 15, 2024

Since we're about to release a1 tonight, I'd wait with landing this until right after a1. We won't forget about it.

@ambv
Copy link
Contributor

ambv commented Oct 15, 2024

Actually, @hugovk should make the call.

@hugovk
Copy link
Member

hugovk commented Oct 15, 2024

Yeah, same call, thanks :)

Copy link
Contributor

@rhansen rhansen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the breakage, and thank you for debugging!

Is a blurb required now that there's an API change between v3.14.0a1 and v3.14.0a2?

Also, would you mind fixing up the commit message, authorship info, etc. on the first commit:

git -c sequence.editor='sed -i -e "1s/^pick/edit/"' rebase -i main &&
git commit --amend -C 843d28f59d2 &&
git rebase --continue

@1st1
Copy link
Member

1st1 commented Oct 16, 2024

@ambv @vstinner

Since we're about to release a1 tonight, I'd wait with landing this until right after a1. We won't forget about it.

Oh I was reading the schedule wrong. I didn't realize Victor wants to delay it just one day, essentially, I thought we were talking one month to merge this. Absolutely can wait a few days.

@hugovk
Copy link
Member

hugovk commented Oct 16, 2024

Thanks, a1 is now out, merge whenever you're ready.

@vstinner
Copy link
Member

I applied @rhansen's change, before that, the test did nothing :-)

Also, would you mind fixing up the commit message, authorship info, etc. on the first commit:

That sounds like a good idea :-) Or at least, make sure that @rhansen is mentioned as a co-author of the change.

@Eclips4
Copy link
Member Author

Eclips4 commented Oct 16, 2024

Apologies for the breakage, and thank you for debugging!

Is a blurb required now that there's an API change between v3.14.0a1 and v3.14.0a2?

Also, would you mind fixing up the commit message, authorship info, etc. on the first commit:

git -c sequence.editor='sed -i -e "1s/^pick/edit/"' rebase -i main &&
git commit --amend -C 843d28f59d2 &&
git rebase --continue

Sorry about that! I definitely will do it.

rhansen and others added 3 commits October 16, 2024 13:17
…4776)

Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for pythongh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Richard Hansen <rhansen@rhansen.org>
@Eclips4
Copy link
Member Author

Eclips4 commented Oct 16, 2024

@rhansen it seems I have done it. If you agree, I will merge it.

(I'm actually do not understand why for f0b3aa862b629690496d0336904a9bb1ada6dd47 github is treating me as co-author...)

@vstinner
Copy link
Member

(I'm actually do not understand why for f0b3aa8 github is treating me as co-author...)

GitHub is correct when saying:

rhansen authored and Eclips4 committed

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ZeroIntensity
Copy link
Member

Should this get tested with the refleak buildbots?

@vstinner
Copy link
Member

Should this get tested with the refleak buildbots?

I don't think so. I tested test_capi manually, and @Eclips4 ran the whole test suite with -R 3:3.

@ambv ambv merged commit bee112a into python:main Oct 16, 2024
37 checks passed
@Eclips4 Eclips4 deleted the revert-of-revert branch October 16, 2024 12:07
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…5532)

Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for pythongh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants