Skip to content

Documentation feature request - Rule performance benchmarks #2749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
CreativeTechGuy opened this issue Nov 8, 2020 · 1 comment
Closed
Labels
documentation Documentation ("docs") that needs adding/updating locked due to age Please open a new issue if you'd like to say more. See https://typescript-eslint.io/contributing. package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin wontfix This will not be worked on

Comments

@CreativeTechGuy
Copy link

Add performance profiling benchmarks for each rule and display these benchmark numbers on the rule summary page. (Maybe a relative score or a green/yellow/red indicator.)

Motivation:

It may not be obvious when first enabling a rule the performance impact it'll have in the long run. It'd help users make decisions about which rules are worth adding depending on how slow the rules are to evaluate. For example: if a rule is a very fringe case that may never apply to your codebase but has a very high performance impact, you would probably choose to not enable it. It'd hopefully also help reduce the number of "it's too slow" Issues being opened.

From a Rule Developer's perspective it'd be a great way to see if the rule you've created is performant or not. There's already not great documentation on how to write rules and it can be easy to find something that functions as you intended but turns out it's extremely slow and you wouldn't realize it until you try it on a massive codebase.

Implementation:

There's already debugging and profiling information, the missing piece is the large and varied codebase to run it on and automatically collect the stats. We might be able to find several other large open-source projects that use TypeScript heavily and run each rule against that. Having enough different codebases can produce a range of values for the performance of each rule. As a step of generating documentation, the command could automatically run the rule against all of those massive projects and collect the stats.

@CreativeTechGuy CreativeTechGuy added package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin triage Waiting for team members to take a look labels Nov 8, 2020
@bradzacher bradzacher added documentation Documentation ("docs") that needs adding/updating and removed triage Waiting for team members to take a look labels Nov 13, 2020
@JoshuaKGoldberg JoshuaKGoldberg added the accepting prs Go ahead, send a pull request that resolves this issue label Oct 25, 2021
@JoshuaKGoldberg JoshuaKGoldberg moved this to Todo in Documentation Nov 9, 2021
@Josh-Cena
Copy link
Member

Josh-Cena commented Jun 2, 2024

This is quite hard because it's practically impossible to benchmark rules in isolation. See for example #5495, #8189 (comment). It doesn't mean we are not interested in what rules are spending time; we will pursue #8087 instead where we can examine things more holistically.

@Josh-Cena Josh-Cena closed this as not planned Won't fix, can't repro, duplicate, stale Jun 2, 2024
@github-project-automation github-project-automation bot moved this from Todo to Done in Documentation Jun 2, 2024
@Josh-Cena Josh-Cena added wontfix This will not be worked on and removed accepting prs Go ahead, send a pull request that resolves this issue labels Jun 2, 2024
@github-actions github-actions bot added the locked due to age Please open a new issue if you'd like to say more. See https://typescript-eslint.io/contributing. label Jun 10, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Documentation ("docs") that needs adding/updating locked due to age Please open a new issue if you'd like to say more. See https://typescript-eslint.io/contributing. package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin wontfix This will not be worked on
Projects
No open projects
Status: Done
Development

No branches or pull requests

4 participants