Skip to content

[3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_cache (GH-29384) (GH-30922) #31076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 2, 2022
Merged

[3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_cache (GH-29384) (GH-30922) #31076

merged 1 commit into from
Feb 2, 2022

Conversation

encukou
Copy link
Member

@encukou encukou commented Feb 2, 2022 •
edited by miss-islington
Loading

Consider the following directory structure:

.
└── PATH1
    └── namespace
        └── sub1
            └── __init__.py

And both PATH1 and PATH2 in sys path:

$ PYTHONPATH=PATH1:PATH2 python3.11
>>> import namespace
>>> import namespace.sub1
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace'])
>>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

.
├── PATH1
│   └── namespace
│       └── sub1
│           └── __init__.py
└── PATH2
    └── namespace
        └── sub2
            └── __init__.py

The newly created module cannot be imported:

>>> ...
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace'])
>>> import namespace.sub2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

>>> import importlib
>>> importlib.invalidate_caches()
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace'])
>>> import namespace.sub2
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.

https://bugs.python.org/issue45703

Co-Authored-By: Miro Hrončok miro@hroncok.cz

Automerge-Triggered-By: GH:encukou

@encukou
[3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalid…
0359bdb 
…ate_cache (pythonGH-29384) (pythonGH-30922)

Consider the following directory structure:

    .
    └── PATH1
        └── namespace
            └── sub1
                └── __init__.py

And both PATH1 and PATH2 in sys path:

    $ PYTHONPATH=PATH1:PATH2 python3.11
    >>> import namespace
    >>> import namespace.sub1
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

    .
    ├── PATH1
    │   └── namespace
    │       └── sub1
    │           └── __init__.py
    └── PATH2
        └── namespace
            └── sub2
                └── __init__.py

The newly created module cannot be imported:

    >>> ...
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

    >>> import importlib
    >>> importlib.invalidate_caches()
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.
vstinner
vstinner approved these changes Feb 2, 2022
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. It looks the same than the 3.10 change.

@miss-islington
Copy link
Contributor

@encukou: Status check is done, and it's a success ✅ .

@miss-islington miss-islington merged commit 8d239bf into python:3.9 Feb 2, 2022
@encukou encukou deleted the invalidate_namespacepath_cache_3.9 branch February 2, 2022 14:11
@miss-islington
Copy link
Contributor

@hroncok: Status check is done, and it's a success ✅ .

hello-adam pushed a commit to hello-adam/cpython that referenced this pull request Jun 2, 2022
@encukou @hello-adam
[3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalid…
877a37b 
…ate_cache (pythonGH-29384) (pythonGH-30922) (pythonGH-31076)

Consider the following directory structure:

    .
    └── PATH1
        └── namespace
            └── sub1
                └── __init__.py

And both PATH1 and PATH2 in sys path:

    $ PYTHONPATH=PATH1:PATH2 python3.11
    >>> import namespace
    >>> import namespace.sub1
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

    .
    ├── PATH1
    │   └── namespace
    │       └── sub1
    │           └── __init__.py
    └── PATH2
        └── namespace
            └── sub2
                └── __init__.py

The newly created module cannot be imported:

    >>> ...
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

    >>> import importlib
    >>> importlib.invalidate_caches()
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.








Co-Authored-By: Miro Hrončok <miro@hroncok.cz>

Automerge-Triggered-By: GH:encukou
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vstinner vstinner vstinner approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@encukou @miss-islington @vstinner @the-knights-who-say-ni @bedevere-bot