Skip to content

New semantic analyzer: store import deps generated during analysis #6582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 1, 2019

Conversation

JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented Mar 22, 2019

Previously some module dependencies genererated from from m import *
went missing, and this could result in crashes when deserialization,
since the target module wasn't serialized yet.

This fixes one incremental mode test case.

Previously some module dependencies genererated from `from m import *`
went missing, and this could result in crashes when deserialization,
since the target module wasn't serialized yet.

This fixes one incremental mode test case.
@JukkaL JukkaL requested review from ilevkivskyi and msullivan and removed request for ilevkivskyi March 22, 2019 18:36
Copy link
Collaborator

@msullivan msullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea looks good, but I have a question

@@ -278,6 +279,9 @@ def semantic_analyze_target(target: str,
analyzer.refresh_partial(refresh_node, patches, final_iteration)
if isinstance(node, Decorator):
infer_decorator_signature_if_simple(node, analyzer)
for dep in analyzer.imports:
state.dependencies.append(dep)
state.priorities[dep] = mypy.build.PRI_LOW
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no checking against what is already a dependency, so can this result in things ending up in the dependency list multiple times or having the priority clobbered?
(Though I suppose we hope that the priority does not matter with the new semantic analyzer, right?)

@JukkaL JukkaL merged commit ffa65a4 into master Apr 1, 2019
@gvanrossum gvanrossum deleted the semanal-import-deps branch April 29, 2019 15:56
hauntsaninja added a commit that referenced this pull request Apr 11, 2025
Fixes #12664

A root cause is there is this stateful `_update_ns_ancestors` thing in
`modulefinder`, so if things get called in the wrong order, you can get
incorrect results.

See also the logic in `all_imported_modules_in_file` where we've fixed
several bugs like this previously, like #13124 and #10937

As a result of (seemingly accidentally) reusing imports across modules,
we can end up in a situation where the namespace gets added as a
dependency to all other modules and so on the cached run we attempt to
find namespace before package, which does not work

I am not sure this `imports` code path is even needed, so I will open an
alternate PR, see #18908.

Relevant history:
- #6582
- #6179

I can't write a good test for this because it requires something in
site_packages, but here's a minimal repro:
```
set -eux
rm -rf repro
mkdir repro
cd repro

SITEPACK=env/site-packages
mkdir -p $SITEPACK

mkdir $SITEPACK/ruamel
mkdir $SITEPACK/ruamel/yaml

printf 'from ruamel.yaml.main import *' > $SITEPACK/ruamel/yaml/__init__.py
printf 'import ruamel.yaml' > $SITEPACK/ruamel/yaml/main.py
printf '' > $SITEPACK/ruamel/yaml/py.typed

printf 'import ruamel.yaml' > a.py
printf 'import a' > main.py

rm -rf .mypy_cache
PYTHONPATH=$SITEPACK mypy main.py
PYTHONPATH=$SITEPACK mypy main.py
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants