Skip to content

OOM when many (>10) project configs passed to the parser #1192

Closed
nextauthjs/next-auth
#11749
@screendriver

Description

@screendriver

@bradzacher and @JoshuaKGoldberg - hijacking OP to say:

We are currently investigating solutions to this with the TypeScript team: microsoft/TypeScript#35676, #1718, #2094, #6754

If you run into an OOM, please provide as much information as you can, including answering the following questions:

  • Is your project open-source? (if yes - link please)
  • Is your package a monorepo? If yes:
    • Are you using lerna, or some other method?
    • How many packages are in your monorepo?
    • Do you have one tsconfig per package, one at the root, or a mish-mash of both?
    • How interdependent is your codebase (i.e. are there lots of utility packages that are used by every package, or is it mainly packages with clear separations?)
  • How do you build your project(s)?
    • Is it via tsc, webpack/rollup/etc, or a mish-mash of both?
  • How many TS files are in your codebase?
  • Approx how many lines of code are in your codebase?
  • How many NPM dependencies do you have?

Also, please provide the following (you can leave the last two out if you're open source):

  • A complete lint run output with the DEBUG=* env variable set? DEBUG=* yarn lint
    • Feel free to censor the output filenames if you'd prefer, but try not to delete any lines.
  • The tsconfig(s) that you pass via the parserOptions.project option
  • Your .eslintrc

The more information you provide, the easier it is for us to look into it and figure out root causes.


What code were you trying to parse?

A relatively huge code base with a lot of TypeScript (using project references), JavaScript and React. Unfortunately I can't give you a sample project because in my small demo projects it works. It only happens in that larger project that isn't open source. I also tried to disable @typescript-eslint/recommended-requiring-type-checking but that didn't work.

What did you expect to happen?

Not to run out of memory

What actually happened?

It runs out of memory.

<--- Last few GCs --->
 start of marking 1708 ms) (average mu = 0.196, current mu = 0.093) all[49915:0x102883000]   112800 ms: Mark-sweep 2040.1 (2058.3) -> 2038.9 (2056.5) MB, 1388.1 / 0.0 ms  (+ 110.5 ms in 27 steps since start of marking, biggest step 7.2 ms, walltime since start of marking 1508 ms) (average mu = 0.104, current mu = 0.007) allo[49915:0x102883000]   114242 ms: Mark-sweep 2039.9 (2056.5) -> 2039.0 (2056.0) MB, 1436.4 / 0.0 ms  (average mu = 0.056, current mu = 0.004) allocation failure GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x10092fbd9]
Security context: 0x295dc96c08a1 <JSObject>
    1: getDeclarationName(aka getDeclarationName) [0x295d351b21f9] [/Users/me/project/node_modules/typescript/lib/typescript.js:~29254] [pc=0x1c6ba57836fc](this=0x295db0c004a9 <undefined>,0x295dbd513309 <NodeObject map = 0x295d58d4a4d1>)
    2: declareSymbol(aka declareSymbol) [0x295d351b2279] [/Users/me/project...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20191111.151654.49915.0.001.json
Node.js report completed
 1: 0x10007e743 node::Abort() [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 2: 0x10007e8c7 node::OnFatalError(char const*, char const*) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 3: 0x100176267 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 4: 0x100176203 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 5: 0x1002fa2b5 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 6: 0x1002fb984 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 7: 0x1002f8857 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 8: 0x1002f683d v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
 9: 0x100301f54 v8::internal::Heap::AllocateRawWithLightRetry(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
10: 0x100301fcf v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
11: 0x1002cebc7 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
12: 0x1005f7725 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
13: 0x10092fbd9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/me/.nvm/versions/node/v12.13.0/bin/node]
14: 0x1c6ba57836fc 	

Versions

package version
@typescript-eslint/typescript-estree 2.6.1
TypeScript 3.7.2
node v12.13.0
npm 6.12.0

Config

{
  "parser": "@typescript-eslint/parser",
  "parserOptions": {
    "sourceType": "module",
    "ecmaVersion": 2015,
    "ecmaFeatures": {
      "jsx": true
    },
    "project": "./packages/**/tsconfig.json"
  },
  "plugins": ["@typescript-eslint", "react", "react-hooks"],
  "extends": [
    "eslint:recommended",
    "plugin:@typescript-eslint/eslint-recommended",
    "plugin:@typescript-eslint/recommended",
    "plugin:@typescript-eslint/recommended-requiring-type-checking",
    "plugin:react/recommended",
    "plugin:prettier/recommended",
    "prettier/@typescript-eslint"
  ],
  "rules": {
    "@typescript-eslint/explicit-function-return-type": "off",
    "@typescript-eslint/no-explicit-any": "off",
    "@typescript-eslint/camelcase": "off",
    "@typescript-eslint/no-use-before-define": "off",
    "@typescript-eslint/unbound-method": "off",
    "@typescript-eslint/no-var-requires": "off",
    "@typescript-eslint/prefer-regexp-exec": "off",
    "require-atomic-updates": "off",
    "react/prop-types": "off",
    "react/display-name": "off",
    "react-hooks/rules-of-hooks": "error",
    "react-hooks/exhaustive-deps": "warn",
    "prefer-promise-reject-errors": "off",
    "react/no-unescaped-entities": "off",
    "react/no-find-dom-node": "off"
  },
  "settings": {
    "react": {
      "version": "detect"
    }
  },
  "env": {
    "es6": true,
    "node": true,
    "browser": true,
    "jest": true
  }
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    blocked by another issueIssues which are not ready because another issue needs to be resolved firstbugSomething isn't workinglocked due to agePlease open a new issue if you'd like to say more. See https://typescript-eslint.io/contributing.metameta-issues which consolidate many issues togetherpackage: typescript-estreeIssues related to @typescript-eslint/typescript-estreeperformanceIssues regarding performance

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions