Skip to content

New semantic analyzer: only create each SymbolNode once #6300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JukkaL opened this issue Feb 1, 2019 · 5 comments
Closed

New semantic analyzer: only create each SymbolNode once #6300

JukkaL opened this issue Feb 1, 2019 · 5 comments
Labels
priority-1-normal semantic-analyzer Problems that happen during semantic analysis

Comments

@JukkaL
Copy link
Collaborator

JukkaL commented Feb 1, 2019

Currently we can create a symbol node multiple times (once per iteration), which is risky since we don't want to risk having different references to a particular name pointing to different nodes. For example, class-based TypedDict definitions may create multiple TypeInfo objects per definition.

I think that for most things we can treat the definition as complete once we've successfully created a symbol table node, and further iterations would skip the definition, as there's nothing more to do.

Normal class definitions should not have this problem, but we should verify that these work as expected:

  • Var nodes
  • TypedDict (as discussed above)
  • Named tuples
  • Type variables
  • Type aliases
  • NewTypes
  • Enums
@JukkaL JukkaL added semantic-analyzer Problems that happen during semantic analysis priority-0-high labels Feb 1, 2019
@ilevkivskyi
Copy link
Member

@JukkaL the problem with AnyStr I told you about was caused by this. I fixed this for type vars, but we still need to check other node kinds in this list.

@ilevkivskyi ilevkivskyi self-assigned this Mar 14, 2019
@ilevkivskyi
Copy link
Member

Just a note to self that we need to fix TypeVar redefinition by preserving a link to defining assignment (I will make a PR).

@JukkaL
Copy link
Collaborator Author

JukkaL commented Jun 12, 2019

Lowering priority since I believe that we've addresses most issues (but not necessarily all).

@JukkaL
Copy link
Collaborator Author

JukkaL commented Jun 12, 2019

We currently allow submodules to be redefined as various different things (see #6828) which breaks these assumptions. It looks like we need another approach for dealing with storing submodules. The current approach is problematic in other ways as well.

ilevkivskyi added a commit that referenced this issue Jun 26, 2019
…nd fix stale upper bounds and values (#6563)

This is a follow up for #6527.

Fixes #7037
I think this also essentially concludes #6300

This PR prohibits duplicate type variable definitions, previously the second one was silently winning. Also now the first one wins. Note that it didn't require additional attribute on `TypeVarExpr` as I initially though, it looks like we can simple use `s.rvalue.analyzed` for the purpose of linking assignments to variables.

This PR also naturally fixes an issue where stale upper bound or value appears in a class that uses the type variable.
@ilevkivskyi
Copy link
Member

I think we have fixed all the cases I am aware. We still do some patching/updating of the existing nodes (replace placeholder types in base classes, alias targets, and variable upper bounds), but we don't re-add symbol table nodes. If we will discover more, we can open dedicated issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority-1-normal semantic-analyzer Problems that happen during semantic analysis
Projects
None yet
Development

No branches or pull requests

2 participants