Skip to content

pathlib.Path.rglob("") raises IndexError in Python 3.11b1 #92550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
bluetech opened this issue May 9, 2022 · 2 comments · Fixed by #92604
Closed

pathlib.Path.rglob("") raises IndexError in Python 3.11b1 #92550

bluetech opened this issue May 9, 2022 · 2 comments · Fixed by #92604
Assignees
@serhiy-storchaka
Labels
3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@bluetech
Copy link

bluetech commented May 9, 2022

Bug report

Python 3.11b1 raises with the following given path.rglob("") where path is a pathlib.Path. This worked in previous versions (and also in whichever previous alphas GitHub Actions used).

    def rglob(self, pattern):
        """Recursively yield all existing files (of any kind, including
        directories) matching the given relative pattern, anywhere in
        this subtree.
        """
        sys.audit("pathlib.Path.rglob", self, pattern)
        drv, root, pattern_parts = self._flavour.parse_parts((pattern,))
        if drv or root:
            raise NotImplementedError("Non-relative patterns are unsupported")
>       if pattern[-1] in (self._flavour.sep, self._flavour.altsep):
E       IndexError: string index out of range

rglob is documented as

This is like calling Path.glob() with “**/” added in front of the given relative pattern

so I think "" is a valid input to rglob().

Pretty sure this was introduced in ea2f5bc/#10349.

Your environment

  • CPython versions tested on: 3.11b1
  • Operating system and architecture: Both Linux and Windows.

Cross-ref: pytest-dev/pytest#9930
@bluetech bluetech added the type-bug An unexpected behavior, bug, or error label May 9, 2022
@AlexWaygood
Copy link
Member

AlexWaygood commented May 9, 2022 •
edited
Loading

Pretty sure this was introduced in ea2f5bc/https://github.com/python/cpython/pull/10349.

Yes, looks like it:

C:\Users\alexw\coding\cpython>git checkout ea2f5bcda1a392804487e6883be89fbad38a01a5
HEAD is now at ea2f5bcda1 bpo-22276: Change pathlib.Path.glob not to ignore trailing path separator (GH-10349)

C:\Users\alexw\coding\cpython>python
Running Debug|x64 interpreter...
Python 3.12.0a0 (main, May  9 2022, 11:32:16) [MSC v.1931 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> next(pathlib.Path().rglob(""))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\alexw\coding\cpython\Lib\pathlib.py", line 963, in rglob
    if pattern[-1] in (self._flavour.sep, self._flavour.altsep):
       ~~~~~~~^^^^
IndexError: string index out of range
>>> exit()

C:\Users\alexw\coding\cpython>git checkout HEAD~1
Previous HEAD position was ea2f5bcda1 bpo-22276: Change pathlib.Path.glob not to ignore trailing path separator (GH-10349)
HEAD is now at 0ef8d921f5 gh-91603: Speed up isinstance/issubclass on union types (GH-91631)

C:\Users\alexw\coding\cpython>python
Running Debug|x64 interpreter...
Python 3.12.0a0 (main, May  9 2022, 11:32:16) [MSC v.1931 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> next(pathlib.Path().rglob(""))
WindowsPath('.')
>>>

cc. @brettcannon, @barneygale

@AlexWaygood AlexWaygood added stdlib Python modules in the Lib dir 3.11 only security fixes 3.12 only security fixes labels May 9, 2022
barneygale added a commit to barneygale/cpython that referenced this issue May 9, 2022
@barneygale
pythongh-92550 - Fix regression in pathlib.Path.rglob()
0cf0eed 
We could try to remedy this by taking a slice, but we then run
into an issue where the empty string will match altsep on POSIX.
That rabbit hole could keep getting deeper.

A proper fix involves making pathlib's path normalisation more
configurable - in this case we want to retain trailing slashes,
but in other we might want to preserve `./` prefixes, or elide
`../` entries when we're sure we won't encounter symlinks.

This reverts commit ea2f5bc.
miss-islington pushed a commit that referenced this issue May 10, 2022
@barneygale
gh-92550 - Fix regression in pathlib.Path.rglob() (GH-92583)
dcdf250 
We could try to remedy this by taking a slice, but we then run into an issue where the empty string will match altsep on POSIX. That rabbit hole could keep getting deeper.

A proper fix for the original issue involves making pathlib's path normalisation more configurable - in this case we want to retain trailing slashes, but in other we might want to preserve `./` prefixes, or elide `../` segments when we're sure we won't encounter symlinks.

This reverts commit ea2f5bc.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 10, 2022
@barneygale @miss-islington
pythongh-92550 - Fix regression in pathlib.Path.rglob() (pythonGH-9…
8d8e823 
…2583)

We could try to remedy this by taking a slice, but we then run into an issue where the empty string will match altsep on POSIX. That rabbit hole could keep getting deeper.

A proper fix for the original issue involves making pathlib's path normalisation more configurable - in this case we want to retain trailing slashes, but in other we might want to preserve `./` prefixes, or elide `../` segments when we're sure we won't encounter symlinks.

This reverts commit ea2f5bc.
(cherry picked from commit dcdf250)

Co-authored-by: Barney Gale <barney.gale@gmail.com>
miss-islington added a commit that referenced this issue May 10, 2022
@miss-islington @barneygale
gh-92550 - Fix regression in pathlib.Path.rglob() (GH-92583)
a51baec 
We could try to remedy this by taking a slice, but we then run into an issue where the empty string will match altsep on POSIX. That rabbit hole could keep getting deeper.

A proper fix for the original issue involves making pathlib's path normalisation more configurable - in this case we want to retain trailing slashes, but in other we might want to preserve `./` prefixes, or elide `../` segments when we're sure we won't encounter symlinks.

This reverts commit ea2f5bc.
(cherry picked from commit dcdf250)

Co-authored-by: Barney Gale <barney.gale@gmail.com>
@serhiy-storchaka
Copy link
Member

Why the whole feature was removed instead of applying a simple fix? @brettcannon

serhiy-storchaka added a commit that referenced this issue May 10, 2022
@serhiy-storchaka
Revert "gh-92550 - Fix regression in pathlib.Path.rglob() (GH-92583)"
ea84fe7 
This reverts commit dcdf250.
serhiy-storchaka added a commit that referenced this issue May 10, 2022
@serhiy-storchaka
Revert "gh-92550 - Fix regression in pathlib.Path.rglob() (GH-92583)"
d01f1e3 
This reverts commit a51baec.
@serhiy-storchaka serhiy-storchaka self-assigned this May 10, 2022
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue May 10, 2022
@serhiy-storchaka
pythongh-92550: Fix pathlib.Path.rglob() for empty pattern
c4d86f9
@serhiy-storchaka serhiy-storchaka mentioned this issue May 10, 2022
Merged
serhiy-storchaka added a commit that referenced this issue May 11, 2022
@serhiy-storchaka
Revert "gh-92550 - Fix regression in pathlib.Path.rglob() (GH-92583)…
b1c4368 
…" (GH-92598)

This reverts commit dcdf250.
serhiy-storchaka added a commit that referenced this issue May 11, 2022
@serhiy-storchaka
Revert "gh-92550 - Fix regression in pathlib.Path.rglob() (GH-92583)…
5197134 
…" (GH-92599)

This reverts commit a51baec.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 11, 2022
@serhiy-storchaka @miss-islington
pythongh-92550: Fix pathlib.Path.rglob() for empty pattern (pythonGH-…
2ea36b3 
…92604)

(cherry picked from commit 87f849c)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this issue May 11, 2022
@serhiy-storchaka
gh-92550: Fix pathlib.Path.rglob() for empty pattern (GH-92604)
87f849c
miss-islington added a commit that referenced this issue May 11, 2022
@miss-islington @serhiy-storchaka
gh-92550: Fix pathlib.Path.rglob() for empty pattern (GH-92604)
5135b6e 
(cherry picked from commit 87f849c)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@hroncok hroncok mentioned this issue May 11, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@serhiy-storchaka serhiy-storchaka

Labels
3.11 only security fixes 3.12 only security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

gh-92550: Fix pathlib.Path.rglob() for empty pattern serhiy-storchaka/cpython
3 participants
@bluetech @serhiy-storchaka @AlexWaygood