Purge programs once they are done #1718
Comments
If this happens during a CLI run, it would add lot of runtime, so it's definitely a worst-case scenario. This seems reasonable, and would save a bunch of memory provided that ESLint is deterministic in how it asks us to parse files. We'll have to have a look into how ESLint globs for files and traverses the directory tree. Worst case here we could create a thin wrapper around ESLint to give us the deterministic order, though I would prefer to avoid introducing extra tools into the ecosystem unless absolutely required. I've thought about this before, but I didn't explore it because there's some complexities around handling the persistent IDE use case. Well, I guess that's more of a question - would there be complexities? There's one I can see: if the user opened a bunch of random files across every single program, then we wouldn't know to clean up the programs, so you'd quickly get into the same high-memory/OOM state. Though I think VSCode has a higher OOM threshold than A super duper rare edge-case would be if a user opens every single file in a program within their IDE, we'd detect the program is "complete", and then clean it up. Then if they save one of those files, we'd have to recreate the program. Do you know how VSCode works in these instances? Do they create and destroy programs as required, or do they keep them all around in memory? |
|
Marking this as |
What code were you trying to parse?
// Put your code hereWhat did you expect to happen?
What actually happened?
Versions
@typescript-eslint/typescript-estreeX.Y.ZTypeScriptX.Y.ZnodeX.Y.ZnpmX.Y.Zhave chatted with @uniqueiniquity offline about this but once the files from the program are done, it is ok to close the watch so we can release program and its resources.. The common scenario is to have files in one tsconfig to be in a folder so likely to be iterated over together so it is possible to remoe the programA when files from programB are being queried and all the files (except say external files and lib files) are already iterated over.. Just need to keep track of files iterated.. The worst thing that can happen is to recreate program. (we would also keep flag if the program is recreated so that it is not purged after that so editing scenario keeps programs for the files being edited if that's how eslint works in editor scenario.. Needs some experimenting)
But this is the way we can work around the OOMs when there are multiple tsconfigs to be linted..
This is what will help issue reported at microsoft/TypeScript#35676 as just sharing source file wont be enough as that fix would delay OOM from say 5 projects to 6/7 but still not solve the main problem of having to create so many programs and create types
The text was updated successfully, but these errors were encountered: