Skip to content

Python 3.14 REPL sometimes breaks when editing code blocks #135185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
treyhunner opened this issue Jun 5, 2025 · 3 comments
Open

Python 3.14 REPL sometimes breaks when editing code blocks #135185

treyhunner opened this issue Jun 5, 2025 · 3 comments
Labels
stdlib Python modules in the Lib dir topic-repl Related to the interactive shell type-bug An unexpected behavior, bug, or error

Comments

@treyhunner
Copy link
Member

treyhunner commented Jun 5, 2025 •
edited by github-actions bot
Loading

Bug report

Bug description:

This has happened to me three times now over the past few weeks and I have not yet been able to reliable reproduce the issue.

The most recent time, I had entered this function into the REPL (typing it all out manually):

>>> def upper_based_on_length(fruit):
...     if len(fruit) < 6:
...         return fruit
...     else:
...         return fruit.upper()
...

I hit the up arrow on my keyboard and I tried to delete the if line with Ctrl+K at the start of the if.

That then immediately showed this error:

>>> def upper_based_on_length(fruit):
...     Traceback (most recent call last):
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/readline.py", line 393, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/reader.py", line 748, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/reader.py", line 731, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/reader.py", line 661, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/reader.py", line 638, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/reader.py", line 315, in calc_screen
    colors = list(gen_colors(self.get_unicode()))
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/utils.py", line 102, in gen_colors
    for color in gen_colors_from_token_stream(gen, line_lengths):
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/utils.py", line 160, in gen_colors_from_token_stream
    for prev_token, token, next_token in token_window:
                                         ^^^^^^^^^^^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/_pyrepl/utils.py", line 357, in prev_next_window
    for x in iterator:
             ^^^^^^^^
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/tokenize.py", line 586, in _generate_tokens_from_c_tokenizer
    raise e from None
  File "/home/trey/.pyenv/versions/3.14.0b1/lib/python3.14/tokenize.py", line 582, in _generate_tokens_from_c_tokenizer
    for info in it:
                ^^
  File "<string>", line 4
    else:
         ^
IndentationError: unindent does not match any outer indentation level

Note that I did not try to run this block of code. I was simply editing it. The syntax simply became temporarily invalid while editing.

I have tried to reproduce this in the same REPL environment using a different function name and I can't seem to do so. If I edit the exact code block in history that produced the error, I see the same exception again. However, if I try to rewrite the same function with a different name and then attempt to edit it, I don't see the error again.

CPython versions tested on:

3.14

Operating systems tested on:

Linux
@treyhunner treyhunner added the type-bug An unexpected behavior, bug, or error label Jun 5, 2025
@hugovk hugovk added the topic-repl Related to the interactive shell label Jun 5, 2025
@terryjreedy
Copy link
Member

I tried the same in IDLE and when I hit ^K with the cursor just before if, the line disappeared. When I moved the cursor to the end and hit , I got correctly got SyntaxError with the unindent message as above. (For better or worse, IDLE seems to display all syntax errors as SyntaxError.) So the error does not seem to be in the code or codeop modules, which I believe _pyrepl also uses.

@picnixz picnixz added the stdlib Python modules in the Lib dir label Jun 7, 2025
@ronaldoussoren
Copy link
Contributor

This could be similar to #133541. Could you check if the error still occurs in the current beta (beta 2)?

@ivanbelenky
Copy link

ivanbelenky commented Jun 9, 2025 •
edited
Loading

@ronaldoussoren the issue is precisely the same as the cited issue.

If you are using uv and pinning to 3.14 -> 0b1 is the micro installed. I just built 0b2 from source and the bug is impossible to reproduce. If you want to double check you can always go to lib/python3.14/_pyrepl/utils.py and spot that gen_color function is only capturing tokenize.TokenError. You should see

def gen_colors(buffer: str):
    ...
    try:
        for color in gen_colors_from_token_stream(gen, line_lengths):
            yield color
            last_emitted = color
    # this SyntaxError except should be misssing
    except SyntaxError:
        return
    except tokenize.TokenError as te:
        yield from recover_unterminated_string(
            te, line_lengths, last_emitted, buffer
        )

Probably @treyhunner this is worhtless knowing it is fixed in beta 2, but in case you want to reproduce, you can always do something like the cited issue, essentially:

>>> def foo():
...      return 0
... a = 3

Place your cursor delete the a and the error should be triggered. This or whatever construct that involves an indentation token will raise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stdlib Python modules in the Lib dir topic-repl Related to the interactive shell type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@treyhunner @ronaldoussoren @hugovk @picnixz @terryjreedy @ivanbelenky