Skip to content

Server error "Could not find source file" on opening second file in a project with large amount of code #803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
amagee opened this issue Nov 14, 2023 · 4 comments · Fixed by #805
Closed

Server error "Could not find source file" on opening second file in a project with large amount of code #803

amagee opened this issue Nov 14, 2023 · 4 comments · Fixed by #805

Comments

@amagee
Copy link

amagee commented Nov 14, 2023

In a project with about 20MB of Javascript / Typescript code, opening the second file will reliably trigger an error looking like this:

Error: Could not find source file: '/path/file1.ts'.
    at getValidSourceFile (/path/node_modules/typescript/lib/tsserver.js:143801:22)
    at Object.getEncodedSemanticClassifications3 [as getEncodedSemanticClassifications] (/path/node_modules/typescript/lib/tsserver.js:144328:77)
    at IpcIOSession.getEncodedSemanticClassifications (/path/node_modules/typescript/lib/tsserver.js:183203:41)
    at encodedSemanticClassifications-full (/path/node_modules/typescript/lib/tsserver.js:182505:43)
    at /path/node_modules/typescript/lib/tsserver.js:184838:69
    at IpcIOSession.executeWithRequestId (/path/node_modules/typescript/lib/tsserver.js:184830:14)
    at IpcIOSession.executeCommand (/path/node_modules/typescript/lib/tsserver.js:184838:29)
    at IpcIOSession.onMessage (/path/node_modules/typescript/lib/tsserver.js:184880:51)
    at process.<anonymous> (/path/node_modules/typescript/lib/tsserver.js:186461:14)
    at process.emit (node:events:517:28)
    at emit (node:internal/child_process:944:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)

even though the file exists.

In my case this size of code existed because we had a large amount of vendor code in dist directories which we had not excluded via the tsconfig. I'm mostly interested to see if the error reporting can be improved.

I put together a standalone script which generates two "legit" files, file1.ts and file2.ts, and a huge file garbage-0.js (39MB). It then opens typescript-language-server and plays back the commands I recorded when I was experiencing this problem from my editor (Neovim).

Saving test.js and tsconfig.json to an empty directory and running node test.js should be sufficient to see the error and stack trace in the output.

Here is the script test.js.

const {spawn} = require("child_process");
const fs = require("fs");

async function main() {
  const commands = [
    {
      method: "initialize",
      jsonrpc: "2.0",
      params: {
        trace: "off",
        rootPath: __dirname,
        rootUri: `file://${__dirname}`,
        workspaceFolders: [
          {
            name: __dirname,
            uri: `file://${__dirname}`,
          },
        ],
        initializationOptions: { hostInfo: "neovim" },
        capabilities: {
          workspace: {
            symbol: {
              symbolKind: {
                valueSet: [
                  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
                  19, 20, 21, 22, 23, 24, 25, 26,
                ],
              },
              dynamicRegistration: false,
            },
            semanticTokens: { refreshSupport: true },
            configuration: true,
            applyEdit: true,
            inlayHint: { refreshSupport: true },
            workspaceFolders: true,
            workspaceEdit: {
              resourceOperations: ["rename", "create", "delete"],
            },
            didChangeWatchedFiles: {
              relativePatternSupport: true,
              dynamicRegistration: true,
            },
          },
          general: { positionEncodings: ["utf-16"] },
          window: {
            showMessage: {
              messageActionItem: { additionalPropertiesSupport: false },
            },
            showDocument: { support: true },
            workDoneProgress: true,
          },
          textDocument: {
            documentHighlight: { dynamicRegistration: false },
            documentSymbol: {
              hierarchicalDocumentSymbolSupport: true,
              symbolKind: {
                valueSet: [
                  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
                  19, 20, 21, 22, 23, 24, 25, 26,
                ],
              },
              dynamicRegistration: false,
            },
            synchronization: {
              willSave: true,
              willSaveWaitUntil: true,
              didSave: true,
              dynamicRegistration: false,
            },
            codeAction: {
              isPreferredSupport: true,
              dataSupport: true,
              resolveSupport: { properties: ["edit"] },
              codeActionLiteralSupport: {
                codeActionKind: {
                  valueSet: [
                    "",
                    "quickfix",
                    "refactor",
                    "refactor.extract",
                    "refactor.inline",
                    "refactor.rewrite",
                    "source",
                    "source.organizeImports",
                  ],
                },
              },
              dynamicRegistration: true,
            },
            callHierarchy: { dynamicRegistration: false },
            hover: {
              contentFormat: ["markdown", "plaintext"],
              dynamicRegistration: true,
            },
            formatting: { dynamicRegistration: true },
            rangeFormatting: { dynamicRegistration: true },
            rename: { prepareSupport: true, dynamicRegistration: true },
            declaration: { linkSupport: true },
            definition: { linkSupport: true, dynamicRegistration: true },
            implementation: { linkSupport: true },
            typeDefinition: { linkSupport: true },
            publishDiagnostics: {
              dataSupport: true,
              relatedInformation: true,
              tagSupport: { valueSet: [1, 2] },
            },
            signatureHelp: {
              signatureInformation: {
                activeParameterSupport: true,
                parameterInformation: { labelOffsetSupport: true },
                documentationFormat: ["markdown", "plaintext"],
              },
              dynamicRegistration: false,
            },
            semanticTokens: {
              tokenTypes: [
                "namespace",
                "type",
                "class",
                "enum",
                "interface",
                "struct",
                "typeParameter",
                "parameter",
                "variable",
                "property",
                "enumMember",
                "event",
                "function",
                "method",
                "macro",
                "keyword",
                "modifier",
                "comment",
                "string",
                "number",
                "regexp",
                "operator",
                "decorator",
              ],
              serverCancelSupport: false,
              tokenModifiers: [
                "declaration",
                "definition",
                "readonly",
                "static",
                "deprecated",
                "abstract",
                "async",
                "modification",
                "documentation",
                "defaultLibrary",
              ],
              requests: { range: false, full: { delta: true } },
              dynamicRegistration: false,
              augmentsSyntaxTokens: true,
              formats: ["relative"],
              multilineTokenSupport: false,
              overlappingTokenSupport: true,
            },
            inlayHint: {
              resolveSupport: { properties: [] },
              dynamicRegistration: true,
            },
            completion: {
              completionItemKind: {
                valueSet: [
                  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
                  19, 20, 21, 22, 23, 24, 25,
                ],
              },
              contextSupport: false,
              completionItem: {
                commitCharactersSupport: false,
                preselectSupport: false,
                deprecatedSupport: false,
                documentationFormat: ["markdown", "plaintext"],
                snippetSupport: false,
              },
              dynamicRegistration: false,
            },
            diagnostic: { dynamicRegistration: false },
            references: { dynamicRegistration: false },
          },
        },
        processId: 40770,
        clientInfo: { name: "Neovim", version: "0.10.0-dev+gd7359a874" },
      },
      id: 1,
    },
    { jsonrpc: "2.0", result: null, id: 0 },
    { method: "initialized", jsonrpc: "2.0", params: {} },
    {
      method: "textDocument/didOpen",
      jsonrpc: "2.0",
      params: {
        textDocument: {
          version: 0,
          text: 'console.log("lol")\n',
          languageId: "typescript",
          uri: `file://${__dirname}/file1.ts`,
        },
      },
    },
    {
      method: "textDocument/semanticTokens/full",
      jsonrpc: "2.0",
      params: {
        textDocument: {
          uri: `file://${__dirname}/file1.ts`,
        },
      },
      id: 2,
    },
    { jsonrpc: "2.0", result: null, id: 1 },
    {
      method: "textDocument/didOpen",
      jsonrpc: "2.0",
      params: {
        textDocument: {
          version: 0,
          text: 'console.log("lol 2")\n',
          languageId: "typescript",
          uri: `file://${__dirname}/file2.ts`,
        },
      },
    },
    {
      method: "textDocument/semanticTokens/full",
      jsonrpc: "2.0",
      params: {
        textDocument: {
          uri: `file://${__dirname}/file2.ts`,
        },
      },
      id: 3,
    },
  ];

  fs.writeFileSync("file1.ts", `console.log("lol")\n`);
  fs.writeFileSync("file2.ts", `console.log("lol 2")\n`);

  // Delete any existing garbage-* files.
  const files = fs.readdirSync('.');
  const garbageFiles = files.filter(file => file.startsWith('garbage-'));
  for (const f of garbageFiles) {
    fs.unlinkSync(f);
  }

  for (let i = 0; i < 400; i++) {
    // For me the error happens somewhere between 1 million (20MB)
    // and 2 million (40MB). It's also reproducible when there are many
    // small files adding up to roughly the same amount, hence the (now-redundant)
    // loop.
    fs.writeFileSync(`garbage-${i}.js`, "console.log('lol');\n".repeat(4000));
  }

  const process = spawn(
    "typescript-language-server",
    ['--stdio'],
    {
      shell: true,
      stdio: 'pipe'
    }
  );

  process.stdout.on("data", (chunk) => {
    console.log(chunk.toString());
  });

  process.stderr.on("data", (chunk) => {
    console.error(chunk.toString());
  });

  let i = 1;
  for (const cmd of commands) {
    console.log("command", i++, "of", commands.length);
    const cmdString = JSON.stringify(cmd);
    process.stdin.write(`Content-Length: ${cmdString.length}\r\n\r\n${cmdString}`);

    // Some small delay is necessary to reproduce the problem. Increasing
    // this to 10 seconds did not fix the problem for me.
    await new Promise(r => setTimeout(r, 1000));
  }
}

main()

Here is the tsconfig.json:

{
  "compilerOptions": {
    "lib": ["ESNext", "DOM"],
    "module": "esnext",
    "target": "esnext",
    "moduleResolution": "bundler",
    "moduleDetection": "force",
    "allowImportingTsExtensions": true,
    "noEmit": true,
    "composite": true,
    "strict": true,
    "downlevelIteration": true,
    "skipLibCheck": true,
    "jsx": "react-jsx",
    "allowSyntheticDefaultImports": true,
    "forceConsistentCasingInFileNames": true,
    "allowJs": true
  }
}
@rchl
Copy link
Member

rchl commented Nov 14, 2023 •
edited
Loading

Can you post the output that you get when running your script? I'd like to see when the responses arrive in your case.

(note that I haven't been able to reproduce on doing a quick test)

@amagee
Copy link
Author

amagee commented Nov 14, 2023

Ah yep, sure, here is the full output of a run:

command 1 of 8
Content-Length: 212

{"jsonrpc":"2.0","method":"window/logMessage","params":{"type":3,"message":"Using Typescript version (bundled) 5.2.2 from path \"/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js\""}}Content-Length: 124

{"jsonrpc":"2.0","id":0,"method":"window/workDoneProgress/create","params":{"token":"92dc87ed-4022-4e69-a8da-e3816759612a"}}Content-Length: 1739

{"jsonrpc":"2.0","id":2,"result":{"capabilities":{"textDocumentSync":2,"completionProvider":{"triggerCharacters":[".","\"","'","/","@","<"],"resolveProvider":true},"codeActionProvider":{"codeActionKinds":["source.fixAll.ts","source.removeUnused.ts","source.addMissingImports.ts","source.organizeImports.ts","source.removeUnusedImports.ts","source.sortImports.ts","quickfix","refactor"]},"codeLensProvider":{"resolveProvider":true},"definitionProvider":true,"documentFormattingProvider":true,"documentRangeFormattingProvider":true,"documentHighlightProvider":true,"documentSymbolProvider":true,"executeCommandProvider":{"commands":["_typescript.applyWorkspaceEdit","_typescript.applyCodeAction","_typescript.applyRefactoring","_typescript.configurePlugin","_typescript.organizeImports","_typescript.applyRenameFile","_typescript.goToSourceDefinition"]},"hoverProvider":true,"inlayHintProvider":true,"linkedEditingRangeProvider":false,"renameProvider":{"prepareProvider":true},"referencesProvider":true,"selectionRangeProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",",","<"],"retriggerCharacters":[")"]},"workspaceSymbolProvider":true,"implementationProvider":true,"typeDefinitionProvider":true,"foldingRangeProvider":true,"semanticTokensProvider":{"documentSelector":null,"legend":{"tokenTypes":["class","enum","interface","namespace","typeParameter","type","parameter","variable","enumMember","property","function","member"],"tokenModifiers":["declaration","static","async","readonly","defaultLibrary","local"]},"full":true,"range":true},"workspace":{"fileOperations":{"willRename":{"filters":[{"scheme":"file","pattern":{"glob":"**/*.{ts,js,jsx,tsx,mjs,mts,cjs,cts}","matches":"file"}}]}}},"callHierarchyProvider":true}}}
command 2 of 8
command 3 of 8
Content-Length: 96


{"jsonrpc":"2.0","method":"$/typescriptVersion","params":{"version":"5.2.2","source":"bundled"}}
command 4 of 8
Content-Length: 124


{"jsonrpc":"2.0","id":1,"method":"window/workDoneProgress/create","params":{"token":"a3c3560d-70f1-4d66-87f9-d085d38d1ee0"}}
Content-Length: 135


{"jsonrpc":"2.0","method":"textDocument/publishDiagnostics","params":{"uri":"file:///home/amagee/t/harness/file1.ts","diagnostics":[]}}
command 5 of 8
Content-Length: 67


{"jsonrpc":"2.0","id":2,"result":{"data":[0,0,7,7,16,0,8,3,11,16]}}
command 6 of 8
Content-Length: 172


{"jsonrpc":"2.0","method":"$/progress","params":{"token":"a3c3560d-70f1-4d66-87f9-d085d38d1ee0","value":{"kind":"begin","title":"Initializing JS/TS language features…"}}}Content-Length: 120

{"jsonrpc":"2.0","method":"$/progress","params":{"token":"a3c3560d-70f1-4d66-87f9-d085d38d1ee0","value":{"kind":"end"}}}
command 7 of 8
command 8 of 8
file:///home/amagee/home/.global-npm-packages/node_modules/typescript-language-server/lib/cli.mjs:17802
    return new TypeScriptServerError(serverId, version, response, parsedResult?.message, parsedResult?.stack);
           ^

TypeScriptServerError: <semantic> TypeScript Server Error (5.2.2)
Could not find source file: '/home/amagee/t/harness/file2.ts'.
Error: Could not find source file: '/home/amagee/t/harness/file2.ts'.
    at getValidSourceFile (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:143801:22)
    at Object.getEncodedSemanticClassifications3 [as getEncodedSemanticClassifications] (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:144328:77)
    at IpcIOSession.getEncodedSemanticClassifications (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:183203:41)
    at encodedSemanticClassifications-full (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:182505:43)
    at /home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184838:69
    at IpcIOSession.executeWithRequestId (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184830:14)
    at IpcIOSession.executeCommand (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184838:29)
    at IpcIOSession.onMessage (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184880:51)
    at process.<anonymous> (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:186461:14)
    at process.emit (node:events:517:28)
    at emit (node:internal/child_process:944:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
    at TypeScriptServerError.create (file:///home/amagee/home/.global-npm-packages/node_modules/typescript-language-server/lib/cli.mjs:17802:12)
    at SingleTsServer.dispatchResponse (file:///home/amagee/home/.global-npm-packages/node_modules/typescript-language-server/lib/cli.mjs:18228:46)
    at SingleTsServer.dispatchMessage (file:///home/amagee/home/.global-npm-packages/node_modules/typescript-language-server/lib/cli.mjs:18169:16)
    at ChildProcess.<anonymous> (file:///home/amagee/home/.global-npm-packages/node_modules/typescript-language-server/lib/cli.mjs:18118:12)
    at ChildProcess.emit (node:events:517:28)
    at emit (node:internal/child_process:944:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21) {
  serverId: 'semantic',
  version: TypeScriptVersion {
    source: 'bundled',
    path: '/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js',
    logger: PrefixingLogger {
      logger: LspClientLogger {
        client: LspClientImpl {
          connection: {
            listen: [Function: listen],
            sendRequest: [Function: sendRequest],
            onRequest: [Function: onRequest],
            sendNotification: [Function: sendNotification],
            onNotification: [Function: onNotification],
            onProgress: [Function: onProgress],
            sendProgress: [Function: sendProgress],
            onInitialize: [Function: onInitialize],
            onInitialized: [Function: onInitialized],
            onShutdown: [Function: onShutdown],
            onExit: [Function: onExit],
            console: [Getter],
            telemetry: [Getter],
            tracer: [Getter],
            client: [Getter],
            window: [Getter],
            workspace: [Getter],
            languages: [Getter],
            notebooks: [Getter],
            onDidChangeConfiguration: [Function: onDidChangeConfiguration],
            onDidChangeWatchedFiles: [Function: onDidChangeWatchedFiles],
            __textDocumentSync: undefined,
            onDidOpenTextDocument: [Function: onDidOpenTextDocument],
            onDidChangeTextDocument: [Function: onDidChangeTextDocument],
            onDidCloseTextDocument: [Function: onDidCloseTextDocument],
            onWillSaveTextDocument: [Function: onWillSaveTextDocument],
            onWillSaveTextDocumentWaitUntil: [Function: onWillSaveTextDocumentWaitUntil],
            onDidSaveTextDocument: [Function: onDidSaveTextDocument],
            sendDiagnostics: [Function: sendDiagnostics],
            onHover: [Function: onHover],
            onCompletion: [Function: onCompletion],
            onCompletionResolve: [Function: onCompletionResolve],
            onSignatureHelp: [Function: onSignatureHelp],
            onDeclaration: [Function: onDeclaration],
            onDefinition: [Function: onDefinition],
            onTypeDefinition: [Function: onTypeDefinition],
            onImplementation: [Function: onImplementation],
            onReferences: [Function: onReferences],
            onDocumentHighlight: [Function: onDocumentHighlight],
            onDocumentSymbol: [Function: onDocumentSymbol],
            onWorkspaceSymbol: [Function: onWorkspaceSymbol],
            onWorkspaceSymbolResolve: [Function: onWorkspaceSymbolResolve],
            onCodeAction: [Function: onCodeAction],
            onCodeActionResolve: [Function: onCodeActionResolve],
            onCodeLens: [Function: onCodeLens],
            onCodeLensResolve: [Function: onCodeLensResolve],
            onDocumentFormatting: [Function: onDocumentFormatting],
            onDocumentRangeFormatting: [Function: onDocumentRangeFormatting],
            onDocumentOnTypeFormatting: [Function: onDocumentOnTypeFormatting],
            onRenameRequest: [Function: onRenameRequest],
            onPrepareRename: [Function: onPrepareRename],
            onDocumentLinks: [Function: onDocumentLinks],
            onDocumentLinkResolve: [Function: onDocumentLinkResolve],
            onDocumentColor: [Function: onDocumentColor],
            onColorPresentation: [Function: onColorPresentation],
            onFoldingRanges: [Function: onFoldingRanges],
            onSelectionRanges: [Function: onSelectionRanges],
            onExecuteCommand: [Function: onExecuteCommand],
            dispose: [Function: dispose]
          }
        },
        level: 3
      },
      prefix: '[lspserver]'
    },
    _api: API {
      displayName: '5.2.2',
      version: '5.2.2',
      fullVersionString: '5.2.2'
    }
  },
  response: {
    seq: 0,
    type: 'response',
    command: 'encodedSemanticClassifications-full',
    request_seq: 7,
    success: false,
    message: "Error processing request. Could not find source file: '/home/amagee/t/harness/file2.ts'.\n" +
      "Error: Could not find source file: '/home/amagee/t/harness/file2.ts'.\n" +
      '    at getValidSourceFile (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:143801:22)\n' +
      '    at Object.getEncodedSemanticClassifications3 [as getEncodedSemanticClassifications] (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:144328:77)\n' +
      '    at IpcIOSession.getEncodedSemanticClassifications (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:183203:41)\n' +
      '    at encodedSemanticClassifications-full (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:182505:43)\n' +
      '    at /home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184838:69\n' +
      '    at IpcIOSession.executeWithRequestId (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184830:14)\n' +
      '    at IpcIOSession.executeCommand (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184838:29)\n' +
      '    at IpcIOSession.onMessage (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184880:51)\n' +
      '    at process.<anonymous> (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:186461:14)\n' +
      '    at process.emit (node:events:517:28)\n' +
      '    at emit (node:internal/child_process:944:14)\n' +
      '    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)'
  },
  serverMessage: "Could not find source file: '/home/amagee/t/harness/file2.ts'.",
  serverStack: "Error: Could not find source file: '/home/amagee/t/harness/file2.ts'.\n" +
    '    at getValidSourceFile (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:143801:22)\n' +
    '    at Object.getEncodedSemanticClassifications3 [as getEncodedSemanticClassifications] (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:144328:77)\n' +
    '    at IpcIOSession.getEncodedSemanticClassifications (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:183203:41)\n' +
    '    at encodedSemanticClassifications-full (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:182505:43)\n' +
    '    at /home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184838:69\n' +
    '    at IpcIOSession.executeWithRequestId (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184830:14)\n' +
    '    at IpcIOSession.executeCommand (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184838:29)\n' +
    '    at IpcIOSession.onMessage (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:184880:51)\n' +
    '    at process.<anonymous> (/home/amagee/home/.global-npm-packages/node_modules/typescript/lib/tsserver.js:186461:14)\n' +
    '    at process.emit (node:events:517:28)\n' +
    '    at emit (node:internal/child_process:944:14)\n' +
    '    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)'
}

Node.js v18.18.0

Note I also had trouble reproducing it with the original script after a restart. I fiddled a bit with some of the numbers and re-copied the commands from my editor session and now I'm able to reproduce it again, the script is now this (apart from changing the timeout and the number & sizes of the files, I suspect the only other change is the ordering of the JSON dictionary parameters but I am not certain)

It's now this:

const {spawn} = require("child_process");
const fs = require("fs");

async function main() {
  const commands = [
    {
      id: 2,
      jsonrpc: "2.0",
      params: {
        initializationOptions: { hostInfo: "neovim" },
        capabilities: {
          general: { positionEncodings: ["utf-16"] },
          workspace: {
            workspaceEdit: {
              resourceOperations: ["rename", "create", "delete"],
            },
            applyEdit: true,
            configuration: true,
            didChangeWatchedFiles: {
              dynamicRegistration: true,
              relativePatternSupport: true,
            },
            workspaceFolders: true,
            inlayHint: { refreshSupport: true },
            semanticTokens: { refreshSupport: true },
            symbol: {
              dynamicRegistration: false,
              symbolKind: {
                valueSet: [
                  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
                  19, 20, 21, 22, 23, 24, 25, 26,
                ],
              },
            },
          },
          textDocument: {
            declaration: { linkSupport: true },
            implementation: { linkSupport: true },
            typeDefinition: { linkSupport: true },
            diagnostic: { dynamicRegistration: false },
            references: { dynamicRegistration: false },
            documentHighlight: { dynamicRegistration: false },
            documentSymbol: {
              symbolKind: {
                valueSet: [
                  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
                  19, 20, 21, 22, 23, 24, 25, 26,
                ],
              },
              dynamicRegistration: false,
              hierarchicalDocumentSymbolSupport: true,
            },
            semanticTokens: {
              tokenModifiers: [
                "declaration",
                "definition",
                "readonly",
                "static",
                "deprecated",
                "abstract",
                "async",
                "modification",
                "documentation",
                "defaultLibrary",
              ],
              requests: { full: { delta: true }, range: false },
              formats: ["relative"],
              dynamicRegistration: false,
              overlappingTokenSupport: true,
              multilineTokenSupport: false,
              serverCancelSupport: false,
              augmentsSyntaxTokens: true,
              tokenTypes: [
                "namespace",
                "type",
                "class",
                "enum",
                "interface",
                "struct",
                "typeParameter",
                "parameter",
                "variable",
                "property",
                "enumMember",
                "event",
                "function",
                "method",
                "macro",
                "keyword",
                "modifier",
                "comment",
                "string",
                "number",
                "regexp",
                "operator",
                "decorator",
              ],
            },
            hover: {
              dynamicRegistration: true,
              contentFormat: ["markdown", "plaintext"],
            },
            codeAction: {
              codeActionLiteralSupport: {
                codeActionKind: {
                  valueSet: [
                    "",
                    "quickfix",
                    "refactor",
                    "refactor.extract",
                    "refactor.inline",
                    "refactor.rewrite",
                    "source",
                    "source.organizeImports",
                  ],
                },
              },
              resolveSupport: { properties: ["edit"] },
              dynamicRegistration: true,
              isPreferredSupport: true,
              dataSupport: true,
            },
            rename: { dynamicRegistration: true, prepareSupport: true },
            callHierarchy: { dynamicRegistration: false },
            formatting: { dynamicRegistration: true },
            completion: {
              completionItemKind: {
                valueSet: [
                  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
                  19, 20, 21, 22, 23, 24, 25,
                ],
              },
              contextSupport: false,
              dynamicRegistration: false,
              completionItem: {
                snippetSupport: false,
                commitCharactersSupport: false,
                preselectSupport: false,
                deprecatedSupport: false,
                documentationFormat: ["markdown", "plaintext"],
              },
            },
            publishDiagnostics: {
              dataSupport: true,
              relatedInformation: true,
              tagSupport: { valueSet: [1, 2] },
            },
            synchronization: {
              willSave: true,
              didSave: true,
              dynamicRegistration: false,
              willSaveWaitUntil: true,
            },
            rangeFormatting: { dynamicRegistration: true },
            inlayHint: {
              dynamicRegistration: true,
              resolveSupport: { properties: [] },
            },
            signatureHelp: {
              dynamicRegistration: false,
              signatureInformation: {
                activeParameterSupport: true,
                parameterInformation: { labelOffsetSupport: true },
                documentationFormat: ["markdown", "plaintext"],
              },
            },
            definition: { linkSupport: true, dynamicRegistration: true },
          },
          window: {
            showMessage: {
              messageActionItem: { additionalPropertiesSupport: false },
            },
            showDocument: { support: true },
            workDoneProgress: true,
          },
        },
        processId: 9646,
        clientInfo: { version: "0.10.0-dev+gd7359a874", name: "Neovim" },
        trace: "off",
        rootPath: `${__dirname}`,
        rootUri: `file://${__dirname}`,
        workspaceFolders: [
          {
            uri: `file://${__dirname}`,
            name: `${__dirname}`,
          },
        ],
      },
      method: "initialize",
    },
    { id: 1, jsonrpc: "2.0", result: null },
    { jsonrpc: "2.0", params: {}, method: "initialized" },
    {
      jsonrpc: "2.0",
      params: {
        textDocument: {
          text: 'console.log("lol")\n',
          languageId: "typescript",
          uri: `file://${__dirname}/file1.ts`,
          version: 0,
        },
      },
      method: "textDocument/didOpen",
    },
    {
      id: 2,
      jsonrpc: "2.0",
      params: {
        textDocument: { uri: `file://${__dirname}/file1.ts` },
      },
      method: "textDocument/semanticTokens/full",
    },
    { id: 1, jsonrpc: "2.0", result: null },
    {
      jsonrpc: "2.0",
      params: {
        textDocument: {
          text: 'console.log("lol 2")\n',
          languageId: "typescript",
          uri: `file://${__dirname}/file2.ts`,
          version: 0,
        },
      },
      method: "textDocument/didOpen",
    },
    {
      id: 3,
      jsonrpc: "2.0",
      params: {
        textDocument: { uri: `file://${__dirname}/file2.ts` },
      },
      method: "textDocument/semanticTokens/full",
    },
  ];

  fs.writeFileSync("file1.ts", `console.log("lol")\n`);
  fs.writeFileSync("file2.ts", `console.log("lol 2")\n`);

  // Delete any existing garbage-* files.
  const files = fs.readdirSync('.');
  const garbageFiles = files.filter(file => file.startsWith('garbage-'));
  for (const f of garbageFiles) {
    fs.unlinkSync(f);
  }

  for (let i = 0; i < 1; i++) {
    // For me the error happens somewhere between 1 million (20MB)
    // and 2 million (40MB). It's also reproducible when there are many
    // small files adding up to roughly the same amount, hence the (now-redundant)
    // loop.
    fs.writeFileSync(`garbage-${i}.js`, "console.log('lol');\n".repeat(2 * 1000 * 1000));
  }

  const process = spawn(
    "typescript-language-server",
    ['--stdio'],
    {
      shell: true,
      stdio: 'pipe'
    }
  );

  process.stdout.on("data", (chunk) => {
    console.log(chunk.toString());
  });

  process.stderr.on("data", (chunk) => {
    console.error(chunk.toString());
  });

  let i = 1;
  for (const cmd of commands) {
    console.log("command", i++, "of", commands.length);
    const cmdString = JSON.stringify(cmd);
    process.stdin.write(`Content-Length: ${cmdString.length}\r\n\r\n${cmdString}`);

    // Some small delay is necessary to reproduce the problem. Increasing
    // this to 10 seconds did not fix the problem for me.
    await new Promise(r => setTimeout(r, 2000));
  }
}

main()

@rchl
Copy link
Member

rchl commented Nov 14, 2023 •
edited
Loading

I could reproduce the error and while I haven't dug into the root of the problem (it looks like a Typescript issue and I'm not paid to solve its issues ;)), I've noticed that the language server is crashing due to unhandled exception on tsserver error.

This doesn't happen in VSCode because it seems to have a global handler for unhandled exceptions. Thus I've made a fix (#805) that will make it so that tsserver errors will not crash the process but the error will still be there technically.

I can also reproduce the code lens resolve request triggering a tsserver error in VSCode so it's not a problem that is specific to this project.

@rchl rchl mentioned this issue Nov 14, 2023
Merged
@rchl rchl closed this as completed in #805 Nov 14, 2023
@rchl
Copy link
Member

rchl commented Nov 14, 2023 •
edited
Loading

I'm not sure if this solution is satisfying for you but as I said, I'm not really willing to dig into Typescript unless someone pays me to do that. :)

It could be reported against TypeScript repo but since reproduction relies on a standalone script and this server, they will likely dismiss it as a third-party issue. If you can find a way to reproduce reliably in VSCode then that would be looked at with more priority by them then.

As I said, I can reproduce code lens resolving failing with "No Project" error in VSCode but that's not exactly the same issue as you've seen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@rchl @amagee