Skip to content

[3.9] bpo-45703: Invalidate _NamespacePath cache on importlib.invalidate_ca… (GH-29384) #29964

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Dec 7, 2021

Consider the following directory structure:

.
└── PATH1
    └── namespace
        └── sub1
            └── __init__.py

And both PATH1 and PATH2 in sys path:

$ PYTHONPATH=PATH1:PATH2 python3.11
>>> import namespace
>>> import namespace.sub1
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace'])
>>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

.
├── PATH1
│   └── namespace
│       └── sub1
│           └── __init__.py
└── PATH2
    └── namespace
        └── sub2
            └── __init__.py

The newly created module cannot be imported:

>>> ...
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace'])
>>> import namespace.sub2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

>>> import importlib
>>> importlib.invalidate_caches()
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace'])
>>> import namespace.sub2
>>> namespace.__path__
_NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.
(cherry picked from commit ae1965c)

Co-authored-by: Miro Hrončok miro@hroncok.cz

https://bugs.python.org/issue45703

pythonGH-29384)

Consider the following directory structure:

    .
    └── PATH1
        └── namespace
            └── sub1
                └── __init__.py

And both PATH1 and PATH2 in sys path:

    $ PYTHONPATH=PATH1:PATH2 python3.11
    >>> import namespace
    >>> import namespace.sub1
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> ...

While this interpreter still runs, PATH2/namespace/sub2 is created:

    .
    ├── PATH1
    │   └── namespace
    │       └── sub1
    │           └── __init__.py
    └── PATH2
        └── namespace
            └── sub2
                └── __init__.py

The newly created module cannot be imported:

    >>> ...
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ModuleNotFoundError: No module named 'namespace.sub2'

Calling importlib.invalidate_caches() now newly allows to import it:

    >>> import importlib
    >>> importlib.invalidate_caches()
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace'])
    >>> import namespace.sub2
    >>> namespace.__path__
    _NamespacePath(['.../PATH1/namespace', '.../PATH2/namespace'])

This was not previously possible.
(cherry picked from commit ae1965c)

Co-authored-by: Miro Hrončok <miro@hroncok.cz>
@miss-islington
Copy link
Contributor Author

@hroncok: Status check is done, and it's a success ✅ .

@miss-islington
Copy link
Contributor Author

@hroncok: Status check is done, and it's a success ✅ .

@miss-islington
Copy link
Contributor Author

@hroncok: Status check is done, and it's a success ❌ .

@github-actions
Copy link

github-actions bot commented Jan 7, 2022

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale Stale PR or inactive for long period of time. label Jan 7, 2022
@vstinner
Copy link
Member

The test failed on Ubuntu:

ERROR: test_invalidate_caches (test.test_importlib.test_namespace_pkgs.SeparatedNamespacePackagesCreatedWhileRunning)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/cpython/cpython/Lib/test/test_importlib/test_namespace_pkgs.py", line 158, in test_invalidate_caches
    import foo.just_created
ModuleNotFoundError: No module named 'foo.just_created'

@hroncok @encukou: Any idea what's going on?

@hroncok
Copy link
Contributor

hroncok commented Jan 23, 2022

Yes. This automatic backport will not work. Things are different here. Please close.

@vstinner vstinner closed this Jan 23, 2022
@miss-islington miss-islington deleted the backport-ae1965c-3.9 branch January 23, 2022 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting review stale Stale PR or inactive for long period of time.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants