Skip to content

⚡ Performance: parserOptions.projectService sometimes no longer outperforms parserOptions.project #9571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JoshuaKGoldberg opened this issue Jul 17, 2024 · 11 comments
Labels
accepting prs Go ahead, send a pull request that resolves this issue bug Something isn't working performance Issues regarding performance team assigned A member of the typescript-eslint team should work on this.

Comments

@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Jul 17, 2024

Overview

When we first started working with the new parserOptions.projectService (formerly parserOptions.EXPERIMENTAL_useProjectService), it outperformed equivalent parserOptions.project setups by ~10-15%. We still sometimes have users report to us that it improved their linting speed. trpc/trpc#5868 is an example of one observation I've made.

However, at some point since then, we've started to notice worse performance of parserOptions.projectService in our own measurements. https://github.com/typescript-eslint/performance contains tooling to test this. From https://github.com/typescript-eslint/performance/blob/45f77c00c6d4384d466c39e8eb633a76f33dd019/README.md#results:

┌───────┬──────────────────────┬──────────────────────┬──────────────────────┬──────────────────────┐
│ files │ project (even)       │ project (references) │ service (even)       │ service (references) │
┼───────┼──────────────────────┼──────────────────────┼──────────────────────┼──────────────────────┤
│ 128   │ '1.149 s ±  0.030 s' │ '1.135 s ±  0.008 s' │ '1.178 s ±  0.010 s' │ '1.736 s ±  0.012 s' │
│ 512   │ '1.636 s ±  0.009 s' │ '1.656 s ±  0.004 s' │ '1.895 s ±  0.007 s' │ '2.613 s ±  0.020 s' │
│ 1024  │ '2.353 s ±  0.013 s' │ '2.399 s ±  0.016 s' │ '3.130 s ±  0.017 s' │ '4.034 s ±  0.061 s' │
┴───────┴──────────────────────┴──────────────────────┴──────────────────────┴──────────────────────┘

Summarizing those results, there seem to be two areas of slowness. Comparing project to projectService at ~1024 files:

  • Overhead per-file ((even)), jumping from ~2.35s to ~3.13s
  • Overhead switching to project references ((references)): jumping from ~2.4s to ~4.0s

⚠️ These measurements might be flawed. We are not sure there's not some external data polluting the results. This issue might just be tracking that we're measuring them incorrectly.


Also note that parserOptions.project has been in production for years, with many performance optimizations around it. parserOptions.projectService is very new and not yet stable. There is likely much more "low-hanging fruit" to optimize around parserOptions.projectService.

We don't yet know whether this is an issue in TypeScript or typescript-eslint. It's probably typescript-eslint.

Relevant past issues:

@JoshuaKGoldberg JoshuaKGoldberg added bug Something isn't working performance Issues regarding performance team assigned A member of the typescript-eslint team should work on this. labels Jul 17, 2024
@JoshuaKGoldberg JoshuaKGoldberg self-assigned this Jul 17, 2024
@JoshuaKGoldberg JoshuaKGoldberg changed the title Performance: parserOptions.projectService no longer outperforms parserOptions.project ⚡ Performance: parserOptions.projectService no longer outperforms parserOptions.project Jul 17, 2024
@JoshuaKGoldberg
Copy link
Member Author

JoshuaKGoldberg commented Jul 18, 2024

Thanks to a very helpful pairing with @jakebailey, I filed two three issues on TypeScript:

Those seem to be most of the performance discrepancy. ⚡

Variant Measurement User Time
project 2.166 s ± 0.021 s 3.381 s
projectService with fixes 2.318 s ± 0.014 s 3.627 s

@JoshuaKGoldberg JoshuaKGoldberg added this to the 8.0.0 milestone Jul 18, 2024
@JoshuaKGoldberg JoshuaKGoldberg added the accepting prs Go ahead, send a pull request that resolves this issue label Jul 18, 2024
@JoshuaKGoldberg JoshuaKGoldberg changed the title ⚡ Performance: parserOptions.projectService no longer outperforms parserOptions.project ⚡ Performance: parserOptions.projectService sometimes no longer outperforms parserOptions.project Jul 29, 2024
@JoshuaKGoldberg JoshuaKGoldberg removed this from the 8.0.0 milestone Jul 31, 2024
@JoshuaKGoldberg
Copy link
Member Author

JoshuaKGoldberg commented Jul 31, 2024

Removing from the v8 milestone. There are multiple issues filed over on TypeScript and our usage in community projects (e.g. t3-oss/create-t3-app#1936, trpc/trpc#5868) has shown real-world cases tend to be at least as fast.

@JoshuaKGoldberg
Copy link
Member Author

JoshuaKGoldberg commented Aug 20, 2024

Status update: microsoft/TypeScript#59335 resolved something like >60-70% of the discrepency. From microsoft/TypeScript#59689:

┌──────────┬───────┬───────────────────────┬───────────────────────┐
│ version  │ files │ project (even layout) │ service (even layout) │
┼──────────┼───────┼───────────────────────┼───────────────────────┤
│ baseline │ 1024  │ '2.524 s ±  0.016 s'  │ '3.338 s ±  0.013 s'  │
┼──────────┼───────┼───────────────────────┼───────────────────────┤
│ modified │ 1024  │ '2.531 s ±  0.010 s'  │ '2.846 s ±  0.010 s'  │
┴──────────┴───────┴───────────────────────┴───────────────────────┘

@shadow-identity
Copy link

I'm on @eslint/js": "^9.12.0, just migrated to 9th version and also wanted to compare project vs projectService performance and got these results:

=== project ===
pnpm exec eslint .  15.32s user 1.35s system 158% cpu 10.532 total
pnpm exec eslint .  14.35s user 1.18s system 161% cpu 9.629 total
pnpm exec eslint .  14.21s user 1.02s system 160% cpu 9.475 total
pnpm exec eslint .  15.01s user 1.02s system 164% cpu 9.757 total
pnpm exec eslint .  14.12s user 1.20s system 156% cpu 9.772 total
=== projectService ===
pnpm exec eslint .  18.82s user 1.49s system 146% cpu 13.882 total
pnpm exec eslint .  18.92s user 1.46s system 148% cpu 13.701 total
pnpm exec eslint .  18.98s user 1.48s system 147% cpu 13.883 total
pnpm exec eslint .  19.12s user 1.58s system 147% cpu 14.062 total
pnpm exec eslint .  19.04s user 1.40s system 147% cpu 13.819 total

I checked the Performance document, but we don't have reloads.
I tested it on TS 5.5, then after reading your last comment I updated TS to 5.6, and now it 1-2 seconds slower >_<
All packages related to TS or eslint are up to date.

@oriSomething
Copy link

I notice, that when we added references the tsconfig.json of non-composite projects that add some overlapped we yet able to fix caused a big slowdown. I quite sure about it, since I needed to add a new tsconfig.json of this kind today. If there are some steps I can make to help give more details, tell me what to do (This is a private repo, so it has some limitations)

@JoshuaKGoldberg JoshuaKGoldberg removed their assignment May 21, 2025
@JoshuaKGoldberg
Copy link
Member Author

Oh! Sorry @oriSomething I missed your message.

Without an actual reproduction it's hard to help. The normal steps we recommend are:

  1. Read through https://typescript-eslint.io/troubleshooting/typed-linting/performance and try the techniques in there
  2. If nothing in that page helps, try to make a minimum reproducible project or playground without any proprietary/secret stuff that you can share publicly

In general, if you have get partial results that aren't already noted in this issue, it can be helpful to post them. But something we can look at ourselves is really the best thing.

@oriSomething
Copy link

  1. I already read https://typescript-eslint.io/troubleshooting/typed-linting/performance and didn't help
  2. Technically, it will be and issue to reproduce to a small repo, because it's a big monorepo, and I cannot exposed the source anyway - this is why I asked if there are steps I can debug it myself to source of the slowness. (Worth saying, there is no issue using TSC specifically, only typed lint rules with TSESLint)

I can understand if you can't help more than this.

Thanks for your time!

@olegafx
Copy link

olegafx commented Jun 10, 2025

@oriSomething we have the same problem. Changing:

project: ['./tsconfig.json'],

to:

projectService: true,

increases ESLint's run time from 1m17s to 1m46s.

@oriSomething
Copy link

@olegafx If it wasn't clear. projectService: true was slower in my case

@olegafx
Copy link

olegafx commented Jun 10, 2025

@olegafx If it wasn't clear. projectService: true was slower in my case

That’s exactly what I said. Using projectService: true is slower in our case as well – adding +37% (+29s) to the total run time.

@oriSomething
Copy link

@olegafx I'm so sorry. I miss read 🙈

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepting prs Go ahead, send a pull request that resolves this issue bug Something isn't working performance Issues regarding performance team assigned A member of the typescript-eslint team should work on this.
Projects
None yet
Development

No branches or pull requests

4 participants