Diff from mypy_primer, showing the effect of this PR on open source code:

pytest (https://github.com/pytest-dev/pytest)
+ src/_pytest/_code/code.py:140: error: Unsupported operand types for - ("None" and "int")  [operator]
+ src/_pytest/_code/code.py:140: note: Left operand is of type "Optional[int]"
+ testing/deprecated_test.py:125: error: Unsupported operand types for > ("int" and "None")  [operator]
+ testing/deprecated_test.py:125: note: Left operand is of type "Optional[int]"
+ testing/deprecated_test.py:125: error: Unsupported operand types for < ("int" and "None")  [operator]
+ testing/deprecated_test.py:125: note: Right operand is of type "Optional[int]"

rich (https://github.com/willmcgugan/rich)
+ rich/console.py:1823: error: Incompatible return value type (got "Tuple[str, Optional[int], Dict[str, Any]]", expected "Tuple[str, int, Dict[str, Any]]")

Do you have a test case? The frame object is implemented in C, and has type int, so if it is int, something must have been gone severly wrong:

https://github.com/python/cpython/blob/60929576e40038ec71d896230f69e4411c82be4b/Include/cpython/frameobject.h#L12

There's also some code in CPython that assumes it not to be None.

This getter returns None if that int is negative: https://github.com/python/cpython/blob/60929576e40038ec71d896230f69e4411c82be4b/Objects/frameobject.c#L55-L65

Edit: It also returns None when the int in struct _frame is zero and PyCode_Addr2Line fails.

Good catch, and a bit unfortunate that the Python docs and implementation disagree.

I'm still curious if we have a test case. When would int be negative / when would PyCode_Addr2Line fail? This has non negligible primer impact, so would be useful to be able to explain the issue in the not unlikely case that users complain.

That's a couple thumbs ups, but no replies :-)

@jd Any luck with a test case? It sucks to tell users "this can be None, but we don't know when or why". There's also a chance you've run into a CPython bug.

Sorry, I don't have a good test case. I've seen this happening randomly while using the profiler project I work on, but it was reported by a user, not something I saw firsthand.

If None bothers people, it might be good to "fix" CPython to return, e.g., 0 instead.

Here's a test case:

def f():
    1/0

f.__code__ = f.__code__.replace(co_linetable=b'\x04\x80\xff\x80')

try:
    f()
except Exception as ex:
    t = ex.__traceback__
    print(t.tb_next.tb_frame.f_lineno)

I created this by adding a printf statement to CPython where it returns None, running ./python -m test, and reducing the part of the test suite that made the printf run.

That said, the test case seems contrived and not something that would come up in practice, so I'm not sure if it's very helpful.

Here's a case where lineno is None (but note that it's been reported as a 3.11 regression): https://bugs.python.org/issue46389

I think we should just use int | Any. This way you can assume that lineno is an int, or you can check if it's None, and mypy won't complain either way. We already do this for regex groups, which return None much more commonly.

I think we should just use int | Any.

Here's a PR doing that: #6935