Skip to content

Span for invalid escape sequence in multiline strings is wrong #116042

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
konstin opened this issue Feb 28, 2024 · 4 comments
Open

Span for invalid escape sequence in multiline strings is wrong #116042

konstin opened this issue Feb 28, 2024 · 4 comments
Labels
3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)

Comments

@konstin
Copy link

konstin commented Feb 28, 2024

Bug report

Bug description:

a = """
Invalid\ Escape
"""

When running with PYTHONWARNINGS=error python3.13 example.py, i get the correct error that there is an invalid escape sequence, but the error span is located at the beginning of the string, not at the location of the actual error:

  File "/home/konsti/example.py", line 1
    a = """
        ^
SyntaxError: invalid escape sequence '\ '

Similarly, for docstrings, the opening quotes are marked, not the actual location:

def f():
    """This function computes f.
    Invalid\ Escape
    """
$ PYTHONWARNINGS=error python3.13 example.py 
  File "/home/konsti/example.py", line 2
    """This function computes f.
    ^^^
SyntaxError: invalid escape sequence '\ '

This makes it look like the file is somehow corrupted or there is an encoding error rather than checking the actual docstring (astral-sh/uv#1928).

Python 3.13.0a1+, installed with pyenv.

I'd expected this to have been reported before, but searching for "invalid escape sequence strings", "escape sequence span" and "SyntaxWarning location" i didn't find anything matching.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

@konstin konstin added the type-bug An unexpected behavior, bug, or error label Feb 28, 2024
@hugovk hugovk added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.13 bugs and security fixes labels Feb 28, 2024
@hugovk
Copy link
Member

hugovk commented Feb 28, 2024

cc @pablogsal

@hugovk
Copy link
Member

hugovk commented Feb 28, 2024

For comparison with Python 3.12.2:

❯ PYTHONWARNINGS=error python3.12 example.py
  File "/private/tmp/example.py", line 1
    a = """
        ^^^
SyntaxError: invalid escape sequence '\ '

And:

❯ PYTHONWARNINGS=error python3.12 example.py
  File "/private/tmp/example.py", line 2
    """This function computes f.
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid escape sequence '\ '

@pablogsal
Copy link
Member

The problem here is that the parser and the tokeniser raise errors with granularity of tokens, and the whole string here is a token, so the parser cannot see inside the string to correctly point to it. I will try to see how hard is to point to the invalid escape and not the full token...

pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 28, 2024
pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 28, 2024
pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 28, 2024
@terryjreedy
Copy link
Member

Python 3.13.0a1+ Thanks for reporting. We are now at .a4+, with many bugfixes and additions and likely a few new uncaught bugs and regressions. So better for testing if possible.

@pablogsal pablogsal removed the type-bug An unexpected behavior, bug, or error label Feb 29, 2024
@pablogsal pablogsal reopened this Mar 2, 2024
pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 13, 2025
pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 13, 2025
pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 13, 2025
…pes in the tokenizer (pythonGH-116049)

(cherry picked from commit 56eda25)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
pablogsal added a commit to pablogsal/cpython that referenced this issue Feb 13, 2025
…pes in the tokenizer (pythonGH-116049)

(cherry picked from commit 56eda25)
(cherry picked from commit 369704b)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
pablogsal added a commit that referenced this issue Feb 13, 2025
pablogsal added a commit that referenced this issue Feb 13, 2025
… the tokenizer (GH-116049) (#130066)

(cherry picked from commit 56eda25)
(cherry picked from commit 369704b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs)
Projects
None yet
Development

No branches or pull requests

4 participants