Skip to content

bpo-37421: Fix multiprocessing get_temp_dir() finalizer #14572

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 4, 2019
Merged

bpo-37421: Fix multiprocessing get_temp_dir() finalizer #14572

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7613509
bpo-37421: Fix multiprocessing get_temp_dir() finalizer
vstinner Jul 3, 2019
08ad64c
Fix if current_process() returs None
vstinner Jul 3, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion Lib/multiprocessing/util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,14 +106,26 @@ def log_to_stderr(level=None):
# Function returning a temp directory which will be removed on exit
#

def _remove_temp_dir(rmtree, tempdir):
rmtree(tempdir)

current_process = process.current_process()
# current_process() can be None if the finalizer is called
# late during Python finalization
if current_process is not None:
current_process._config['tempdir'] = None

def get_temp_dir():
# get name of a temp directory which will be automatically cleaned up
tempdir = process.current_process()._config.get('tempdir')
if tempdir is None:
import shutil, tempfile
tempdir = tempfile.mkdtemp(prefix='pymp-')
info('created temp directory %s', tempdir)
Finalize(None, shutil.rmtree, args=[tempdir], exitpriority=-100)
# keep a strong reference to shutil.rmtree(), since the finalizer
# can be called late during Python shutdown
Finalize(None, _remove_temp_dir, args=(shutil.rmtree, tempdir),
exitpriority=-100)
process.current_process()._config['tempdir'] = tempdir
return tempdir

Expand Down
4 changes: 4 additions & 0 deletions Misc/NEWS.d/next/Library/2019-07-03-12-47-52.bpo-37421.gR5hC8.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Fix :func:`multiprocessing.util.get_temp_dir` finalizer: clear also the
'tempdir' configuration of the current process, so next call to
``get_temp_dir()`` will create a new temporary directory, rather than
reusing the removed temporary directory.