Skip to content

IndentationError in new repl #133541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ronaldoussoren opened this issue May 6, 2025 · 5 comments
Open

IndentationError in new repl #133541

ronaldoussoren opened this issue May 6, 2025 · 5 comments
Assignees
@ambv
Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir topic-repl Related to the interactive shell type-bug An unexpected behavior, bug, or error

Comments

@ronaldoussoren
Copy link
Contributor

ronaldoussoren commented May 6, 2025 •
edited by bedevere-app bot
Loading

Bug report

Bug description:

Python version: Python 3.14.0a7+ (heads/main-dirty:8d0e07eb899.

To reproduce type the following class definition in the REPL:

class C:
    def
    m

Then set the cursor to the space before m and press backspace to correct the incorrect newline.

Behaviour: the REPL exits with an IndentationError.

Typing backspace works correctly when the cursor is on m.

The exit/crash does not happen in 3.14a7.

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

macOS

Linked PRs
@ronaldoussoren ronaldoussoren added the type-bug An unexpected behavior, bug, or error label May 6, 2025
@terryjreedy
Copy link
Member

Win 10: with a space between the cursor and 'm', so 3 spaces to left of cursor
IndentationError: unindent does not match any outer indentation level

@ronaldoussoren
Copy link
Contributor Author

Full traceback:

Python 3.14.0a7+ (heads/main-dirty:8d0e07eb899, May  6 2025, 18:45:02) [Clang 17.0.0 (clang-1700.0.13.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...     def
...    Traceback (most recent call last):
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/__main__.py", line 10, in <module>
    __pyrepl_interactive_console()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/main.py", line 59, in interactive_console
    run_multiline_interactive_console(console)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/simple_interact.py", line 138, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/readline.py", line 393, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 751, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 734, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 664, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 641, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/reader.py", line 315, in calc_screen
    colors = list(gen_colors(self.get_unicode()))
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/utils.py", line 97, in gen_colors
    for color in gen_colors_from_token_stream(gen, line_lengths):
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/utils.py", line 155, in gen_colors_from_token_stream
    for prev_token, token, next_token in token_window:
                                         ^^^^^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/_pyrepl/utils.py", line 348, in prev_next_window
    for x in iterator:
             ^^^^^^^^
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/tokenize.py", line 586, in _generate_tokens_from_c_tokenizer
    raise e from None
  File "/Library/Frameworks/PythonDev.framework/Versions/3.14/lib/python3.14/tokenize.py", line 582, in _generate_tokens_from_c_tokenizer
    for info in it:
                ^^
  File "<string>", line 3
    method(self): pass
                      ^
IndentationError: unindent does not match any outer indentation level

@ronaldoussoren ronaldoussoren added the topic-repl Related to the interactive shell label May 7, 2025
@picnixz picnixz added stdlib Python modules in the Lib dir 3.14 bugs and security fixes labels May 7, 2025
@StanFromIreland
Copy link
Contributor

I can reproduce but with a varied traceback.

Python 3.14.0a7+ (heads/main:3dfed230928, May  6 2025, 17:44:25) [GCC 15.1.1 20250425 (Red Hat 15.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class C:
...     def
...    Traceback (most recent call last):
  File "/usr/local/lib/python3.14/_pyrepl/readline.py", line 393, in multiline_input
    return reader.readline()
           ~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 748, in readline
    self.handle1()
    ~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 731, in handle1
    self.do_cmd(cmd)
    ~~~~~~~~~~~^^^^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 661, in do_cmd
    self.refresh()
    ~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 638, in refresh
    self.screen = self.calc_screen()
                  ~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/_pyrepl/completing_reader.py", line 261, in calc_screen
    screen = super().calc_screen()
  File "/usr/local/lib/python3.14/_pyrepl/reader.py", line 315, in calc_screen
    colors = list(gen_colors(self.get_unicode()))
  File "/usr/local/lib/python3.14/_pyrepl/utils.py", line 102, in gen_colors
    for color in gen_colors_from_token_stream(gen, line_lengths):
                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/_pyrepl/utils.py", line 160, in gen_colors_from_token_stream
    for prev_token, token, next_token in token_window:
                                         ^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/_pyrepl/utils.py", line 357, in prev_next_window
    for x in iterator:
             ^^^^^^^^
  File "/usr/local/lib/python3.14/tokenize.py", line 586, in _generate_tokens_from_c_tokenizer
    raise e from None
  File "/usr/local/lib/python3.14/tokenize.py", line 582, in _generate_tokens_from_c_tokenizer
    for info in it:
                ^^
  File "<string>", line 3
    m
     ^
IndentationError: unindent does not match any outer indentation level

@danielhollas
Copy link
Contributor

cc @ambv based on the stacktrace it looks like it's coming from the new syntax highlighting code.

@ambv ambv self-assigned this May 7, 2025
@ambv
Copy link
Contributor

ambv commented May 7, 2025

Oof. OK, yes, indeed. Sorry about that omission.

ambv added a commit to ambv/cpython that referenced this issue May 7, 2025
@ambv
pythongh-133541: Handle SyntaxError raised by the tokenizer on user i…
431b928 
…nput
@bedevere-app bedevere-app bot mentioned this issue May 7, 2025
Merged
pablogsal pushed a commit that referenced this issue May 8, 2025
@ambv
gh-133541: Handle SyntaxError raised by the tokenizer on user input (#…
b2fabce 
…133606)
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 8, 2025
@ambv @miss-islington
pythongh-133541: Handle SyntaxError raised by the tokenizer on user i…
1971073 
…nput (pythonGH-133606)

(cherry picked from commit b2fabce)

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
@bedevere-app bedevere-app bot mentioned this issue May 8, 2025
Merged
pablogsal pushed a commit that referenced this issue May 8, 2025
@miss-islington
[3.14] gh-133541: Handle SyntaxError raised by the tokenizer on user …
0c340d1 
…input (GH-133606) (#133709)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ambv ambv

Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir topic-repl Related to the interactive shell type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@ambv @ronaldoussoren @danielhollas @picnixz @terryjreedy @StanFromIreland