Skip to content

Implement some minor performance optimizations #4283

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 21, 2022

Conversation

charliermarsh
Copy link
Collaborator

@charliermarsh charliermarsh commented Nov 20, 2022

I tend to benchmark the parser by parsing all of CPython in Ruff. This involves (1) disabling Ruff's linting pass, then (2) running cargo build --release && hyperfine --warmup 10 --runs 50 -i "./target/release/ruff resources/test/cpython --no-cache".

Prior to this change:

Benchmark 1: ./target/release/ruff resources/test/cpython --no-cache
  Time (mean ± σ):     265.2 ms ±   4.6 ms    [User: 2260.7 ms, System: 83.1 ms]
  Range (min … max):   259.0 ms … 281.1 ms    50 runs

Benchmark 1: ./target/release/ruff resources/test/cpython --no-cache
  Time (mean ± σ):     268.4 ms ±   4.2 ms    [User: 2268.1 ms, System: 83.5 ms]
  Range (min … max):   262.5 ms … 283.6 ms    50 runs

Benchmark 1: ./target/release/ruff resources/test/cpython --no-cache
  Time (mean ± σ):     269.7 ms ±   5.5 ms    [User: 2277.9 ms, System: 83.5 ms]
  Range (min … max):   260.1 ms … 280.7 ms    50 runs

After this change:

Benchmark 1: ./target/release/ruff resources/test/cpython --no-cache
  Time (mean ± σ):     255.6 ms ±   5.1 ms    [User: 2156.0 ms, System: 83.4 ms]
  Range (min … max):   247.2 ms … 271.8 ms    50 runs

Benchmark 1: ./target/release/ruff resources/test/cpython --no-cache
  Time (mean ± σ):     255.4 ms ±   4.8 ms    [User: 2152.8 ms, System: 83.0 ms]
  Range (min … max):   246.9 ms … 269.7 ms    50 runs

Benchmark 1: ./target/release/ruff resources/test/cpython --no-cache
  Time (mean ± σ):     256.3 ms ±   4.8 ms    [User: 2156.3 ms, System: 82.8 ms]
  Range (min … max):   248.5 ms … 270.6 ms    50 runs

So it's a ~4.8% improvement, if you take the median results.

Note that there's about ~58ms of overhead associated with this benchmark (e.g., to locate all the Python files on-disk, read their contents, etc.), so that's really a lower-bound on improvement.

Copy link
Member

@coolreader18 coolreader18 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thank you! I had no idea ruff used rustpython-parser, that's so cool!!

@@ -14,6 +14,19 @@ pub fn parse_strings(
let initial_end = values[0].2;
let initial_kind = (values[0].1 .1 == StringKind::U).then(|| "u".to_owned());

// Optimization: fast-track the common case of a single string.
if values.len() == 1 && matches!(&values[0].1 .1, StringKind::Normal | StringKind::U) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you wanted, you could do matches!(&*values, [(_, (_, StringKind::Normal | StringKind::U), _)]) for this whole thing (or the equivalent with if-let). I find that using patterns is generally clearer for understanding the structures of tuples over using indices.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice suggestion, thanks! (I still can't figure out how to take ownership of the String itself without the weird values.into_iter(), so I couldn't if-let.)

@charliermarsh
Copy link
Collaborator Author

@coolreader18 - Definitely! We've used it from the very beginnings of Ruff. I've tried to contribute back as many compatibility improvements as I can :)

@youknowone youknowone merged commit 42b304a into RustPython:main Nov 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants