Bug: ~1.5x slowdown in Sentry codebase from project service #8424

JoshuaKGoldberg · 2024-02-10T15:39:18Z

Before You File a Bug Report Please Confirm You Have Done The Following...

I have tried restarting my IDE and the issue persists.
I have updated to the latest version of the packages.
I have searched for related issues and found none that matched my issue.
I have read the FAQ and my problem is not listed.

Issue Description

Switching from parserOptions.project: true to parserOptions.EXPERIMENTAL_useProjectService should in the worst case scenario should be roughly the same speed as before. The best case scenario should be faster. But checking it out on the getsentry/sentry project, I'm seeing a raise from ~50s/65s to ~80s/110s. 😬

Baseline:

 $ time yarn lint:js                    
yarn run v1.22.19
$ eslint static/app tests/js --ext .js,.jsx,.ts,.tsx
✨  Done in 49.47s.
yarn lint:js  65.43s user 3.26s system 138% cpu 49.607 total

With the project service:

$ time yarn lint:js
yarn run v1.22.19
$ eslint static/app tests/js --ext .js,.jsx,.ts,.tsx
✨  Done in 81.42s.
yarn lint:js  108.11s user 8.10s system 142% cpu 1:21.55 total

Thanks @yagiz for reporting!

Reproduction Repository Link

https://github.com/getsentry/sentry/pull/64967/files#diff-e2954b558f2aa82baff0e30964490d12942e0e251c1aa56c3294de6ec67b7cf5R28-R30

Repro Steps

Clone getsentry/sentry
time yarn lint:js
Enable parserOptions.EXPERIMENTAL_useProjectService
time yarn lint:js

Versions

package	version
`@typescript-eslint/eslint-plugin`	`6.19.0`
`@typescript-eslint/parser`	`6.19.0`
`TypeScript`	`5.3.2`
`ESLint`	`8.49.0`
`node`	`20`

The text was updated successfully, but these errors were encountered:

JoshuaKGoldberg · 2024-02-10T16:44:11Z

First finding: this is an example of an existing performance issue with eslint-plugin-import. @anonrig we recommend disabling the eslint-plugin-import rules mentioned in that page, as many of them duplicate work TypeScript already does and/or don't play well with our tooling.

Disabling the import/* rules as recommended improves lint time for both:

Baseline: ✨ Done in 36.57s. \n yarn lint:js 51.59s user 1.82s system 145% cpu 36.695 total
Project service: ✨ Done in 58.75s. \n yarn lint:js 80.06s user 5.38s system 145% cpu 58.866 total

More investigation to be done for the discrepancy, still. And maybe issue(s) to be filed on better detecting/handling this common scenario and/or playing better with eslint-plugin-import.

Full investigation for the curious...

Looking at logs generated with DEBUG=typescript-eslint:* yarn lint:js in 3b2040a: Sentry Debug Logs.txt...

node_modules/@types .d.ts files are having the project service called for them too. Not just source .ts/.tsx files.

$ eslint static/app tests/js --ext .js,.jsx,.ts,.tsx
  typescript-eslint:typescript-estree:useProgramFromProjectService Opening project service file for: /users/josh/repos/sentry/static/app/__mocks__/api.tsx +0ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opened project service file: { configFileName: '/users/josh/repos/sentry/tsconfig.json', configFileErrors: [] } +5s
  typescript-eslint:typescript-estree:useProgramFromProjectService Found project service program for: /users/josh/repos/sentry/static/app/__mocks__/api.tsx +1ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/josh/repos/sentry/static/app/__mocks__/api.tsx +0ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'esnext', 'dom' ] +0ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opening project service file for: /users/josh/repos/sentry/node_modules/@types/lodash/isequal.d.ts +234ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opened project service file: { configFileName: undefined, configFileErrors: undefined } +2ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Found project service program for: /users/josh/repos/sentry/node_modules/@types/lodash/isequal.d.ts +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/josh/repos/sentry/node_modules/@types/lodash/isEqual.d.ts +236ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'esnext', 'dom' ] +216ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opening project service file for: /users/josh/repos/sentry/static/app/__mocks__/prismjs.tsx +32ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opened project service file: { configFileName: '/users/josh/repos/sentry/tsconfig.json', configFileErrors: undefined } +2ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Found project service program for: /users/josh/repos/sentry/static/app/__mocks__/prismjs.tsx +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/josh/repos/sentry/static/app/__mocks__/prismjs.tsx +34ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'esnext', 'dom' ] +34ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opening project service file for: /users/josh/repos/sentry/node_modules/@types/prismjs/components.d.ts +10ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Opened project service file: { configFileName: undefined, configFileErrors: undefined } +2ms
  typescript-eslint:typescript-estree:useProgramFromProjectService Found project service program for: /users/josh/repos/sentry/node_modules/@types/prismjs/components.d.ts +0ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/josh/repos/sentry/node_modules/@types/prismjs/components.d.ts +12ms
  typescript-eslint:parser:parser Resolved libs from program: [ 'esnext', 'dom' ] +11ms
...

sentry/static/app/__mocks__/api.tsx +0ms: expected
sentry/node_modules/@types/lodash/isequal.d.ts +234ms: unexpected 🤔
sentry/static/app/__mocks__/prismjs.tsx +32ms: expected
sentry/node_modules/@types/prismjs/components.d.ts +10ms: unexpected 🤔

Running DEBUG=eslint:cli-engine yarn lint:js shows that ESLint isn't trying to load those files. From Sentry Debug Logs - eslint-cli-engine.txt:

$ eslint static/app tests/js --ext .js,.jsx,.ts,.tsx
  eslint:cli-engine Lint /Users/josh/repos/sentry/static/app/__mocks__/api.tsx +0ms
  eslint:cli-engine Lint /Users/josh/repos/sentry/static/app/__mocks__/prismjs.tsx +5s
  eslint:cli-engine Lint /Users/josh/repos/sentry/static/app/__mocks__/react-date-range.tsx +16ms
...

Why is the parser being called for these files? With a console.log(new Error().stack):

Error
    at useProgramFromProjectService (/Users/josh/repos/typescript-eslint/packages/typescript-estree/dist/useProgramFromProjectService.js:16:21)
    at getProgramAndAST (/Users/josh/repos/typescript-eslint/packages/typescript-estree/dist/parser.js:38:100)
    at parseAndGenerateServices (/Users/josh/repos/typescript-eslint/packages/typescript-estree/dist/parser.js:162:11)
    at Object.parseForESLint (/Users/josh/repos/typescript-eslint/packages/parser/dist/parser.js:93:80)
    at parse (/Users/josh/repos/sentry/node_modules/eslint-module-utils/parse.js:97:32)
    at ExportMap.parse (/Users/josh/repos/sentry/node_modules/eslint-plugin-import/lib/ExportMap.js:808:420)
    at ExportMap.for (/Users/josh/repos/sentry/node_modules/eslint-plugin-import/lib/ExportMap.js:807:201)
    at ExportMap.get (/Users/josh/repos/sentry/node_modules/eslint-plugin-import/lib/ExportMap.js:801:465)
    at checkDefault (/Users/josh/repos/sentry/node_modules/eslint-plugin-import/lib/rules/default.js:22:46)
    at ruleErrorHandler (/Users/josh/repos/sentry/node_modules/eslint/lib/linter/linter.js:1051:28)

😱 eslint-plugin-import! Specifically, import/default.

What's happening is:

The sentry codebase has eslint-plugin-import's import/default rule enabled to ensure that when an imported module has a default export, that export is default-imported
In order to determine whether an imported module has a default export, the rule does an out-of-band parse using eslint-module-utils/parse
That parse calls to the same parser, languageOptions, etc. that ESLint is normally called with
Our parser doesn't know this call is coming from not-ESLint and type information doesn't need to be generated

...and thus our parser tries to create type information on .d.ts files.

bradzacher · 2024-02-10T20:48:14Z

Oh @JoshuaKGoldberg we can fix that!
https://github.com/import-js/eslint-plugin-import/blob/7a21f7e10f18c04473faadca94928af6b8e28009/utils%2Fparse.js#L80-L85

I'd forgotten that I'd added this to their setup.
We just need to add more deletes.
Perhaps we should add a util function to our project so we can control this without touching their code??

JoshuaKGoldberg · 2024-02-10T22:27:29Z

Aha! I didn't know about that - awesome! Filed #8428.

JoshuaKGoldberg · 2024-07-18T12:00:57Z

This bug is mostly info specific to Sentry and the import plugin+rules. I filed #9571 to track general performance concerns with projectService. Closing this one for now. Thanks everyone for helping us know of -> investigate these issues!

#9571 (comment) mentions two TypeScript issues tracking performance regressions.

JoshuaKGoldberg added bug Something isn't working triage Waiting for team members to take a look performance Issues regarding performance labels Feb 10, 2024

JoshuaKGoldberg added this to the 8.0.0 milestone Feb 10, 2024

JoshuaKGoldberg self-assigned this Feb 10, 2024

JoshuaKGoldberg mentioned this issue Feb 10, 2024

Enhancement: Add more debug logs to project service code paths #8425

Closed

4 tasks

JoshuaKGoldberg changed the title ~~Bug: 2x slowdown in Sentry codebase from project service~~ Bug: ~1.5x slowdown in Sentry codebase from project service Feb 10, 2024

This was referenced Feb 10, 2024

feat: speed up non-type-aware linting with project service #8322

Merged

⚡️ Performance: Don't open client files unnecessarily for project service #8427

Closed

JoshuaKGoldberg mentioned this issue Feb 10, 2024

Enhancement: Add an exported utility to remove type info from parserOptions #8428

Closed

4 tasks

JoshuaKGoldberg mentioned this issue Feb 10, 2024

When clearing typed linting options, also delete parserOptions.EXPERIMENTAL_useProjectService import-js/eslint-plugin-import#2962

Closed

anonrig mentioned this issue Feb 12, 2024

perf: get rid of eslint-plugin-import rules getsentry/eslint-config-sentry#202

Merged

JoshuaKGoldberg added the team assigned A member of the typescript-eslint team should work on this. label Jun 2, 2024

Josh-Cena removed the triage Waiting for team members to take a look label Jun 4, 2024

JoshuaKGoldberg mentioned this issue Jul 17, 2024

⚡ Performance: parserOptions.projectService sometimes no longer outperforms parserOptions.project #9571

Open

JoshuaKGoldberg changed the title ~~Bug: ~1.5x slowdown in Sentry codebase from project service~~ ⚡️ Performance: Slowdowns switching from project to projectService Jul 18, 2024

JoshuaKGoldberg changed the title ~~⚡️ Performance: Slowdowns switching from project to projectService~~ Bug: ~1.5x slowdown in Sentry codebase from project service Jul 18, 2024

JoshuaKGoldberg closed this as completed Jul 18, 2024

github-actions bot added the locked due to age Please open a new issue if you'd like to say more. See https://typescript-eslint.io/contributing. label Jul 27, 2024

github-actions bot locked as resolved and limited conversation to collaborators Jul 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Bug: ~1.5x slowdown in Sentry codebase from project service #8424

Bug: ~1.5x slowdown in Sentry codebase from project service #8424

JoshuaKGoldberg commented Feb 10, 2024 •

edited

Loading

JoshuaKGoldberg commented Feb 10, 2024 •

edited

Loading

Uh oh!

bradzacher commented Feb 10, 2024

Uh oh!

JoshuaKGoldberg commented Feb 10, 2024

Uh oh!

JoshuaKGoldberg commented Jul 18, 2024

Uh oh!

Uh oh!

Bug: ~1.5x slowdown in Sentry codebase from project service #8424

Bug: ~1.5x slowdown in Sentry codebase from project service #8424

Comments

JoshuaKGoldberg commented Feb 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Before You File a Bug Report Please Confirm You Have Done The Following...

Issue Description

Reproduction Repository Link

Repro Steps

Versions

JoshuaKGoldberg commented Feb 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Full investigation for the curious...

Uh oh!

bradzacher commented Feb 10, 2024

Uh oh!

JoshuaKGoldberg commented Feb 10, 2024

Uh oh!

JoshuaKGoldberg commented Jul 18, 2024

Uh oh!

JoshuaKGoldberg commented Feb 10, 2024 •

edited

Loading

JoshuaKGoldberg commented Feb 10, 2024 •

edited

Loading