Skip to content

Bad type inference with a partial None type and a for loop. #8637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
barisione opened this issue Apr 6, 2020 · 2 comments · Fixed by #18631
Closed

Bad type inference with a partial None type and a for loop. #8637

barisione opened this issue Apr 6, 2020 · 2 comments · Fixed by #18631
Labels
bug mypy got something wrong priority-0-high topic-runtime-semantics mypy doesn't model runtime semantics correctly

Comments

@barisione
Copy link

barisione commented Apr 6, 2020

I believe there's a bug in mypy (from master) which makes it not detect that a variabe used in a loop may be uninitialised (or use the value assigned before the loop) unless you are iterating over a literal []/().

This code:

def test() -> str:
    c = None
    for c in "":
        pass

    if c is None:
        return None

    return "foo"


print(test())

Prints:

$ python3 opt.py
None

But mypy says:

$ mypy opt.py
Success: no issues found in 1 source file

This is with mypy just updated from git and Python 3.7.4 but also happens with mypy 0.720 and Python 2.7:

$ mypy --version
mypy 0.770+dev.d54fc8e78a8dd5be3c97c1aa0da357e313113dec

If you replace "" with a [] or () I get:

$ mypy opt.py
opt.py:10: error: Incompatible return value type (got "None", expected "str")
Found 1 error in 1 file (checked 1 source file)

It's also interesting that, if the loop iterates over a variable, then nothing potentially wrong is detected:

import typing


def test(l : typing.Sequence[str]) -> str:
    c = None
    for c in l:
        pass

    if c is None:
        return None

    return "foo"


print(test([]))
print(test(""))
print(test(["hello", "world"]))
@JukkaL
Copy link
Collaborator

JukkaL commented Apr 9, 2020

I narrowed this down to this example:

c = None
a = [1]
for c in a:
    pass

reveal_type(c)  # int (should be Optional[int])

I believe that this is not about tracking whether iterables are empty, but an issue with type inference with a partial None type and a for loop.

Marking as high priority since this can lead to false negatives and cause issues with mypyc.

@JukkaL JukkaL added bug mypy got something wrong priority-0-high labels Apr 9, 2020
@TH3CHARLie
Copy link
Collaborator

TH3CHARLie commented Apr 15, 2020

Any hints on how to fix this? I'd like to take a shot.

Updates: I did some tracing and summarize below:

  • In checker.py -> visit_for_stmt -> analyze_index_variables, it introduces a type assignment in the conditional binder's frames, which I believe serves the purpose of correct type inference inside the loop body.

  • When calling reveal_type outside the function body, checkexpr.py -> visit_name_expr -> narrow_type_from_binder gets a restriction from the conditional binder(in @JukkaL 's case, a int) and therefore narrows the type to a int

I am still going through the details but I feel that this is caused by the incorrect conditional binder+name lookup behavior as a modified version of @JukkaL's example presented:

c = "foo"
a = [1]
for c in a:
    reveal_type(c) #  Revealed type is 'builtins.str', but I expect it should be int
    pass
reveal_type(c) #  Revealed type is 'builtins.str'

@AlexWaygood AlexWaygood added the topic-runtime-semantics mypy doesn't model runtime semantics correctly label Apr 14, 2022
@JelleZijlstra JelleZijlstra changed the title Iterating over an empty string makes mypy think the iteration variable is assigned Bad type inference with a partial None type and a for loop. Apr 14, 2022
@JukkaL JukkaL closed this as completed in f946af4 Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug mypy got something wrong priority-0-high topic-runtime-semantics mypy doesn't model runtime semantics correctly
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants