Skip to content

Massive performance penalty when shouldProvideParserServices is true #243

Closed
@dsgkirkby

Description

@dsgkirkby

What code were you trying to parse?
I first noticed this against my company's repo (which is private), but I've replicated a similar performance penalty on this repo itself.

What happened?
When run normally, I get a pretty fast lint on my machine (2018 Macbook Pro 15-inch, i7/16GB), about 4.5s:

✔ ~/code/typescript-eslint [master|✔] 
16:34 $ time yarn lint
yarn run v1.13.0
$ eslint . --ext .js,.ts
✨  Done in 3.04s.

real    0m3.229s
user    0m4.476s
sys     0m0.260s

However, if I add "project": "./tsconfig.json" to the parser options, I notice a 10x performance decrease (which is roughly consistent with what we noticed on our own repo).

✔ ~/code/typescript-eslint [master|✚ 1] 
16:39 $ time yarn lint
yarn run v1.13.0
$ eslint . --ext .js,.ts
✨  Done in 34.64s.

real    0m34.807s
user    0m43.377s
sys     0m2.933s

Why does this happen?

I've dug into this a little bit, and it seems to come entirely from the time taken to generate the AST and ts.Program for each file; in parser.ts when shouldProvideParserServices is true (which is only when project is set), getProgramAndAST takes ~300ms/file, compared to ~1.5ms/file when it's false.

Within getProgramAndAST, the source of the slowness is almost entirely from calls to ts.createProgram. I wrapped createProgramAndAST as well as the call to createProgram with console.time/console.timeEnd and got the following (this is a small but representative sample):

createProgram: 264.493ms
getProgramAndAST: 264.883ms
createProgram: 267.681ms
getProgramAndAST: 268.112ms
createProgram: 397.579ms
getProgramAndAST: 397.939ms
createProgram: 252.260ms
getProgramAndAST: 252.598ms

How could this be fixed?

In comparison, tslint (which lints our internal codebase about 8x faster than typescript-eslint with project set) offers an idea for what performance here could look like, only calls createProgram once per lint run (https://github.com/palantir/tslint/blob/master/src/runner.ts#L204), and gives it the names of all the files that will be linted (https://github.com/palantir/tslint/blob/master/src/linter.ts#L97). This isn't possible for typescript-estree as it stands, as eslint only provides a single filename to a call to the parser.

Some possible approaches for a solution:

  • Try to make ts.createProgram faster
  • Change eslint to pass down all filenames to the parser each time, allowing typescript-estree to create one ts.Program and reuse it

I'm happy to undertake one of these approaches (my thinking is that the second is more promising) and file a PR myself, but I wanted to check in with you, the package maintainers, and get feedback on this before investing a lot more time into it.

Versions

package version
@typescript-eslint/typescript-estree master
TypeScript 3.3.1
node 8.12.0
yarn 1.13.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    package: typescript-estreeIssues related to @typescript-eslint/typescript-estreequestionQuestions! (i.e. not a bug / enhancment / documentation)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions