Skip to content

incremental: Crash after deleting a function. #3052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
timabbott opened this issue Mar 24, 2017 · 13 comments
Closed

incremental: Crash after deleting a function. #3052

timabbott opened this issue Mar 24, 2017 · 13 comments

Comments

@timabbott
Copy link

timabbott commented Mar 24, 2017

I ran into this crash with 0.501 today on the Zulip codebase when running mypy with this version of the code zulip/zulip@09f66b5 (which deleted the below function).

I've attached the cache state tarball in case it's helpful: mypy.tar.gz

Traceback (most recent call last):
  File "/srv/zulip-py3-venv/bin/mypy", line 6, in <module>
    main(__file__)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/main.py", line 42, in main
    res = type_check_only(sources, bin_dir, options)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/main.py", line 86, in type_check_only
    options=options)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 183, in build
    dispatch(sources, manager)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1531, in dispatch
    process_graph(graph, manager)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1761, in process_graph
    process_fresh_scc(graph, prev_scc)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1830, in process_fresh_scc
    graph[id].fix_cross_refs()
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/build.py", line 1311, in fix_cross_refs
    fixup_module_pass_one(self.tree, self.manager.modules)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 21, in fixup_module_pass_one
    node_fixer.visit_symbol_table(tree.names)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 79, in visit_symbol_table
    stnode = lookup_qualified_stnode(self.modules, cross_ref)
  File "/srv/zulip-venv-cache/015153c2b7c837e69f696afeaa7df4d23e89fed1/zulip-py3-venv/lib/python3.5/site-packages/mypy/fixup.py", line 255, in lookup_qualified_stnode
    assert key in names, "Cannot find %s for %s" % (key, name)
AssertionError: Cannot find do_set_realm_description for zerver.lib.actions.do_set_realm_description
@refi64
Copy link
Contributor

refi64 commented Mar 27, 2017

FWIW, I couldn't actually reproduce this with (inside the Zulip directory):

rm -rf var/mypy_cache
git checkout 09f66b5c6dfc5c8bf5d52fb51259189ade0f336e^
tools/run-mypy
git checkout 09f66b5c6dfc5c8bf5d52fb51259189ade0f336e
tools/run-mypy

@gvanrossum
Copy link
Member

Yeah the missing info is all the mtimes of the files in Tim's working directory when it failed. :-(

@gvanrossum
Copy link
Member

(FWIW if this occurs again on an open source package, as of mypy 0.521 the mtimes in the .mypy_cache directory are no longer a problem, as we now use a hash of the source file.)

@timabbott
Copy link
Author

I think @hackerkid saw this earlier today in the Zulip project; but we may be a bit behind 0.521 (haven't updated since PyCon).

@hackerkid
Copy link

hackerkid commented Aug 3, 2017

I got the error after I switched from a branch invite-link which had the model MultiUserInvitation to the branch get-user-2g which didn't had the model. But interestingly I was not able to reproduce the error when I switched from invite-link to master. I tried the switching multiple times and was able to obtain the same behavior. All the branches were rebased on top of upstream/master.

During each switch I did

  • ./tools/provision
  • ./tools/run-mypy

I have attached the tar of my Zulip directory including the cache here.

Update: fixed the get-user-2g-link. Thanks ethanhs

@emmatyping
Copy link
Member

(the correct get-user-2g link)

@hackerkid did this happen consistently? Could you try updating the version of Mypy that you use? It appears your version is old.

@gvanrossum
Copy link
Member

I downloaded the tarball but could not repro the issue with mypy 0.521, so I think it's not worth looking into further. Let us know whether you still see this with master.

@timabbott
Copy link
Author

Hmm, I've definitely seen this since 0.521. I just upgraded to the latest mypy master in case that helps; will let you know when I next see it (it's about once a week, usually after a series of switches between branches including one that deletes things).

@gvanrossum
Copy link
Member

If and when you see it, tarring up your entire zulip tree (.git and all) like @hackerkid did would be very helpful.

@gvanrossum
Copy link
Member

We've seen a similar crash internally, where the assert referenced a class that was deleted by the change. Sadly it was also impossible to repro. Which means that at least the work-around of deleting the cache shouldn't be too painful.

@gvanrossum
Copy link
Member

An internal user claims this can be reproduced as follows:

  1. Start with a clean slate
  2. Run mypy
  3. Make an edit
  4. Run it again, with -i
  5. Revert the edit (e.g. git stash)
  6. Run it again, with -i -- BOOM

The edit in step 2 could be a whitespace edit, as long as it's a type-checked file.

@ilevkivskyi
Copy link
Member

@gvanrossum I now have a simple repro for this crash inspired by #4043. Here is the recipe:

Initial setup:

# a.py
from b import x

# b.py
from c import x

# c.py
x = 1

with any previous mypy caches removed. Steps to reproduce:

  • run mypy a.py
  • delete (or comment out) definition of x in c.py
  • run mypy -i a.py (it will show an obvious error)
  • add random whitespace to a.py
  • run mypy -i a.py again -- AssertionError: Cannot find x for c.x

@gvanrossum
Copy link
Member

Thanks! Adding an unused variable definition to a.py in the 4th step will work too. I've also confirmed that this crash was introduced by the same PR (#2014) that caused #4043. I haven't played with fixes yet.

gvanrossum pushed a commit that referenced this issue Oct 4, 2017
Fixes #4043
Fixes #3852
Fixes #3052

This is a simple-minded solution for inconsistent cache state problem discovered in #4043.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants