-
-
Notifications
You must be signed in to change notification settings - Fork 32.1k
GH-116380: Speed up glob.glob()
by removing some system calls
#116392
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
101 commits
Select commit
Hold shift + click to select a range
db3c620
GH-116380: Make `glob.glob()` twice as fast
barneygale 9e1f059
Use `os.listdir()` if we don't need to check entry type.
barneygale 10432df
A few small speedups.
barneygale 7e389e2
Simplify prefix removal
barneygale 8680a0a
Re-implement `glob0()`, `glob1()`, and `has_magic()`.
barneygale 3bf3124
Fix errant `StopIteration`.
barneygale f8fb992
Skip compiling pattern for consecutive `**` segments.
barneygale 50ef080
Clarify regex/path building in literal and recursive selectors.
barneygale ccefacd
Simplify code to ignore root_dir.
barneygale fa951f6
Fix possible Windows separator issue.
barneygale 0aec12c
Address some review feedback.
barneygale 72691ba
Use assignment expressions in a couple of places
barneygale c58dd21
Replace lambda with `operator.not_`.
barneygale c361ec9
Merge branch 'main' into gh-116380
barneygale 22b30db
Speed up `_add_trailing_slash()`
barneygale 83b70bd
Speed up `select_literal()`
barneygale 1d32d14
Speed up `select_recursive()`
barneygale 1e5aacc
Merge branch 'main' into gh-116380
barneygale a038bb8
Merge branch 'main' into gh-116380
barneygale f1440a9
Cache compiled patterns rather than selectors.
barneygale 9c64643
Remove a bit of code duplication.
barneygale b0e8ba6
Fix stray newline
barneygale 1b1233e
Merge branch 'main' into gh-116380
barneygale 0e02ec5
Remove tests for glob0 and glob1
barneygale be4865e
Add a bunch of comments explaining the more subtle parts.
barneygale 203e8ef
Merge branch 'main' into gh-116380
barneygale 13355a0
Clarify variable naming in iglob()
barneygale 2e5cebd
Use keyword arguments to pass True/False/None literals, for clarity.
barneygale 5eba2eb
Speed up recursive globbing very slightly
barneygale b0a99b7
Merge branch 'main' into gh-116380
barneygale ad0ece8
Implement recursive wildcards with a stack
barneygale cafe9be
Add argument defaults, simplify code slightly.
barneygale 301d922
Also make rel_path optional
barneygale beb2507
Optimise _add_trailing_slash
barneygale 312c73a
Remove use of os.listdir() -- doesn't generalise
barneygale ae820e2
Add `_Globber` class; prepare for merger with pathlib globbing.
barneygale dcfe11d
Unify with pathlib implementation \o/
barneygale 123a0f6
Use literal selector only if no case sensitivity preference is given.
barneygale 0ed7b9c
Fix a few tests
barneygale aceb85f
Fix a few more tests.
barneygale b04de9d
Merge commit '689ada79150f28b0053fa6c1fb646b75ab2cc200' into gh-116380
barneygale 3eb2d19
Merge branch 'main' into gh-116380
barneygale 8a15db0
Fix select() argument order.
barneygale 7eb3e61
Merge branch 'main' into gh-116380
barneygale 316ea56
Merge branch 'main' into gh-116380
barneygale 2018027
Support `include_hidden` and `dir_fd` in `pathlib._glob`.
barneygale 2f21626
Fix stray newline
barneygale 339df68
Update Lib/pathlib/_glob.py
barneygale 28aa95f
Fix docs
barneygale abcb1f8
Test for unique results
barneygale 71387a6
Spacing
barneygale de22de6
Merge branch 'main' into gh-116380
barneygale 8b08374
Merge branch 'main' into gh-116380
barneygale 54efa7c
Merge branch 'main' into gh-116380
barneygale cf11922
Update whatsnew
barneygale 6710924
Merge branch 'main' into gh-116380
barneygale a547cd2
Merge branch 'main' into gh-116380
barneygale 14ae438
Close file descriptors when `recursive_selector` is finalized.
barneygale 69d7a86
Make `iglob()` a generator.
barneygale 3b84a1d
Make `_iglob()` a generator.
barneygale f9f9a8d
Make `_relative_glob()` a generator.
barneygale 24a9ee4
Simplify skipping empty string
barneygale d05d58d
Merge branch 'main' into gh-116380
barneygale 27c463e
Merge branch 'main' into gh-116380
barneygale a94f2a7
Make `_GlobberBase` fully abstract.
barneygale d19bb89
Address review feedback
barneygale 1677588
Typo fix
barneygale 539f044
Speed up pattern parsing.
barneygale 70a1b42
Add test for globbing above recursion limit.
barneygale 1560712
Merge branch 'main' into gh-116380
barneygale 099e86e
Apply suggestions from code review
barneygale ee76faf
Test that `iglob().close()` closes file descriptors.
barneygale 4cf8a4d
Address some review feedback
barneygale 8a118a7
Merge branch 'main' into gh-116380
barneygale 3ad9367
Address more review comments
barneygale 66af33d
Drop parse_entry
barneygale ce74ef1
Address review feedback
barneygale a69a060
Add comment.
barneygale a10a1e0
Merge branch 'main' into gh-116380
barneygale e2e0a2e
Merge branch 'main' into gh-116380
barneygale be260c1
Merge branch 'main' into gh-116380
barneygale 0d1dc7e
Merge branch 'main' into gh-116380
barneygale f1c5326
Merge branch 'main' into gh-116380
barneygale e113457
Merge branch 'main' into gh-116380
barneygale d8ec2d4
Merge branch 'main' into gh-116380
barneygale 203b78b
Merge branch 'main' into gh-116380
barneygale ffeb81a
Windows test fixes
barneygale ec157a1
Speed up suppression of initial empty path.
barneygale 7a620db
Anticipate the removal of glob0() and glob1()
barneygale e9fa3ac
Merge branch 'main' into gh-116380
barneygale 46429ad
Handle _initial_path_exists more explicitly in select_special()
barneygale f160e8c
Add comment, remove redundant check
barneygale 47846ca
Avoid slicing prefix when root_dir is None
barneygale e55d946
Replace `_initial_path_exists` sentinel with `empty` boolean arg
barneygale 47499c7
Slightly simplify recursive globbing
barneygale 849de87
Revert "Slightly simplify recursive globbing"
barneygale 10a6cf4
Split out method for scanning cwd
barneygale 413b12e
Merge branch 'main' into gh-116380
barneygale acf0f9b
Merge branch 'main' into gh-116380
barneygale a89211f
Merge branch 'main' into gh-116380
barneygale e4c519c
Merge branch 'main' into gh-116380
barneygale File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.