From the stack trace, it looks like this is an error in yarn.
There's nothing of ours in the stack trace. If it was an error due to our packages, I'd expect to see both ESLint and typescript-eslint in the stack trace, but there are neither.
Unless the stack trace is truncated, but given that it ends in .pnp.js, it looks like yarn functionality.
Also doing a quick search of the typescript codebase reveals no function called getPnpTypeRoots, which makes sense given the path to the typescript file is .yarn\cache\typescript-patch-fea84800fa-1.zip - implying they have patched the typescript package.

Would you be able to produce a small repro repo to help look into it?

Certainly!
https://github.com/jstaro/yarn-pnp-eslint-repro
Run yarn lint

This does not repro on my system using your example repo.
This probably implies that your cache (or your yarn2 install) is bad.

Thanks for checking.
Don't know what to say, I tried it on 3 machines now, and I can repro it on all of them (Windows 10). yarn cache clean --all does not help, and Yarn2 is installed locally in the repro repo, as per Yarn 2 guidelines.

I'll see if I can get some non-Windows coworkers to try...

Hmm, windows 10 - I tried on macos.
That makes me think even more that this is a yarn2 problem.
I don't know how easy it is to do with yarn2 caches and all that, but could you try patching in the code in #1593 to see if that fixes it?

If yes, then this is definitely a yarn2 problem.

It did not fix it (here are debug logs from before and after monkey-patching the .zip in the yarn cache: https://gist.github.com/jstaro/c73b02664993c931a30c705e9cddde03).
Before the patch I see some resolution rejections related to base.json in the log though (lines 127+) that are later gone, and the exception call stack has an additional line after the patch for some reason. I personally can't glean much from these logs.

It was a long shot! Just thought I'd put it out there, as another user using yarn 2 suggested that change to fix yarn 2 with our packages.
I'd probably consider filing a bug with yarn at this point, as it smells like something with the new tooling.

@bradzacher i will check this in few hours when i get back from work

Found the culprit. This happens because typescript-estree canonicalize file paths to lowercase. The canonical tsconfig path ends up being passed to yarn pnp's findPackageLocator API (which works case-sensitively) via the patched version of typescript.

So it is the perfect storm of the three interacting.

we lowercase the paths on case insensitive OS because TS does the same under the hood. If we don't sync with TS's implementation, then we can have cache misses, and falsely report files as not found.

so this is a bug in yarn then? Probably worth raising it with them.

I'm not sure there's anything we can do on our side - this kind of normalization goes against the very purpose of Yarn which is to ensure a deterministic behaviour as much as possible, even across platforms. If the PnP API was to be case-insensitive, it could lead to working Windows programs that would fail on Linux (unless we switched the API to be case-insensitive everywhere, which won't happen 😄).

Imo if you really want the normalization (which is reasonable), lowercasing the path should be done when computing the cache key, but not for the actual file manipulation. This would make both tools work. Does that make sense? Basically, something like this:

const getCached = p => {
  const cacheKey = getCanonicalFileName(p);
  cache[cacheKey] = cache[cacheKey] || loadFile(p);
};

Instead of something like:

const getCached = p => {
  p = getCanonicalFileName(p);
  cache[p] = cache[p] || loadFile(p);
};

Note that I don't have any knowledge of your codebase except for what's mentioned in this thread so I might be very wrong - feel free to correct me if you think this isn't possible.

If the PnP API was to be case-insensitive, it could lead to working Windows programs that would fail on Linux

Which is why it should be contextually case sensitive right? If the OS is case sensitive, shouldn't yarn also follow suit, so that the experience is predictable?
I.e. if a windows user does require('Typescript'), that should succeed, because it'll succeed without yarn pnp.
Typescript has ts.sys.useCaseSensitiveFileNames to help implementors determine exactly this - what mode typescript is working in.

lowercasing the path should be done when computing the cache key, but not for the actual file manipulation

It's not our cache, it's typescript's cache.
We are a relatively thin layer on top of typescript's compiler API. TypeScript internally canonicalizes the paths, and because we hook into the internals, it means we have to work with these paths.
(anything in this file really https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/typescript-estree/src/create-program/createWatchProgram.ts)

When we step back out of the internals, then we instead use the filepath as written:

Which is why it should be contextually case sensitive right? If the OS is case sensitive, shouldn't yarn also follow suit, so that the experience is predictable? I.e. if a windows user does require('Typescript'), that should succeed, because it'll succeed without yarn pnp.

Not really. If a project is configured to use PnP, then there is no reason to make it compatible with non-PnP setups - otherwise it'd be a bit like saying that Jest and Mocha should have the same configuration file for predictable usage.

By contrast, the OS will vary from a contributor to the other, and even during deployments (even people developing on Windows will most likely deploy on Linux). We must have a consistent behavior by following the strictest common denominator.

We are a relatively thin layer on top of typescript's compiler API. TypeScript internally canonicalizes the paths, and because we hook into the internals, it means we have to work with these paths.

Hm. I think I'll need to try it out myself to better understand if there's something we could do 🤔

@arcanis How about using fs.realpathSync.native on currentDirectory?

I know PnP patches fs to use portable paths so that needs the original node fs module instead. I don't know how easy/difficult would it be to access the unpatched fs from the patched typescript, but I think this is the cleanest way to achieve the fix since

• this fix does not require OS-specific logic within the yarn codebase. It just relies on different implementations of fs.realpathSync.native on different OSes.
• this fix is limited to one line in the typescript package, and we already need to patch typescript for it to work properly with PnP already.

Does anyone has a quick and dirty fix for now?

I have created a quick and dirty solution for now.
Create a file called plugin-pnp.js:

const origStr = `  function findPackageLocator(location) {
    let relativeLocation = normalizePath(fslib_2.ppath.relative(runtimeState.basePath, location));`;
const fixedStr = `  function findPackageLocator(location) {
    let basePath = runtimeState.basePath;
    if (location && process.platform === 'win32' && location.length >= runtimeState.basePath.length) {
      const lowerCaseBasePath = runtimeState.basePath.toLowerCase();
      if (location.startsWith(lowerCaseBasePath)){
        basePath = lowerCaseBasePath;
      }
    }
    let relativeLocation = normalizePath(fslib_2.ppath.relative(basePath, location));`;

module.exports = {
  name: `plugin-pnp`,
  factory: (require) => ({
    hooks: {
      afterAllInstalled(project) {
        const fs = require('@yarnpkg/fslib');
        const pnpFile = fs.ppath.join(project.cwd, `.pnp.js`);
        const pnpContent = fs.xfs.readFileSync(pnpFile, `utf-8`);
        const fixed = pnpContent.replace(origStr, fixedStr);
        fs.xfs.writeFileSync(pnpFile, fixed, `utf-8`);
      },
    },
  }),
};

and then add to your .yarnrc.yml:

plugins:
  - path: plugin-pnp.js

The plugin will make sure your .pnp.js file is always patched in a way that handles the case insensitivity issues caused by eslint

@clemyan do you have a way to pinpoint to the location that needs to be patched in typescript? I can verify your fix and maybe make a PR to yarn...

It's not our cache, it's typescript's cache.
We are a relatively thin layer on top of typescript's compiler API. TypeScript internally canonicalizes the paths, and because we hook into the internals, it means we have to work with these paths.
(anything in this file really https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/typescript-estree/src/create-program/createWatchProgram.ts)

@bradzacher my CI works perfectly on windows machines using yarn 2 and typescript, and the only thing failing is eslint, which means that yarn 2 is working well with typescript on windows machines. Doesn't that forces us to come to a conclusion that the issue stems from your library and not typescript/yarn2?

Yarn2 works with typescript, because (from my understanding) it patches your typescript package so that it works with typescript (https://github.com/yarnpkg/berry/blob/9076e9bf68e1a74f01b8184749f258ce8741d324/packages/plugin-compat/sources/patches/typescript.patch.ts).

You're welcome to have a look at our source code - the only thing we do to file paths is: ensure they're absolute, and ensure they match TS's casing.

Why? Because we have to ensure that ESLint interops with typescript. ESLint provides no guarantee about the casing of the file paths, nor does it guarantee that a path is absolute or relative.

TS on the other hand guarantees that all paths given to us from the program host's callback functions are both absolute, and normalised in casing. Thus, we must ensure that ESLint's paths match TS's paths by normalising their casing and ensuring they're absolute.

In this regard, our package follows typescript's internal implementation.

This is the function we use to normalise the casing of paths:
https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/typescript-estree/src/create-program/shared.ts#L40-L44

This is typescript's function to do the same thing:
https://github.com/Microsoft/TypeScript/blob/master/src/compiler/core.ts#L1925-L1927

They're exactly the same - relying upon ts.sys.useCaseSensitiveFileNames to conditionally lowercase the path when the OS is detected to be case insensitive (i.e. lowercased on windows).

I don't have a windows dev box, nor do I use yarn 2. This mirrors the majority of our userbase right now. For these reasons, I won't be looking into it myself in the near future - as a volunteer maintainer there are more pressing issues for me to focus on.

We're a community maintained project. We rely upon contributions from the community to help move the project forward. As such, you're welcome to dig into the thin layer to investigate where the potential incompatibility with yarn2 might be.

Here is a quick rundown on how the parser interops with TS:

The entrypoint to a parse triggered by ESLint is here:
parseAndGenerateServices, which prepares an options object (extra), and calls into getProgramAndAST. This function purely switches between the various styles of parsing we support:

If you do not have parserOptions.project specified, then it performs a very, very thin interop with typescript via createIsolatedProgram. This simply creates an isolated program for the single file you've given us (i.e. no typechecking or dependency resolution), and returns the AST for the one file.

If you have parserOptions.project specified, then it instead calls createProjectProgram. This function is a thin coordination layer which calls getProgramsForProjects until it finds a project that contains the required file.

getProgramsForProjects is the caching layer which keeps track of which programs we've already made TS create for what projects.

If there is a program, it does some basic caching using TS's normalised filenames - this saves us having to do an expensive lookup in the program unless we have to.

If there are no programs for a project, it calls createWatchProgram, which sets up a program host, and a subsequent program for the file.

If for some reason there are no programs containing the given file, then getProgramsForProjects calls maybeInvalidateProgram which goes through a process of elimination to figure out if it's due to one of the programs and filesystem being out of sync. This check is required for reasons I won't get into right now as it shouldn't be relevant.

@clemyan do you have a way to pinpoint to the location that needs to be patched in typescript? I can verify your fix and maybe make a PR to yarn...

No, my comment above is just throwing some ideas out here.

However, as far as I can see, the only blocker is the fact that findPnpTypeRoots (created by Yarn's typescript patch) tries to find the PackageLocator of the workspace containing the given tsconfig.json and fails because of path case-sensitivity.

If I need to use typescript-eslint with Yarn 2 now, I would just write my own local patch to use case sensitive paths for tsconfig only. Again, just only throwing ideas out here.

From my understanding, there are two ways this issue could be fixed:

• Yarn could switch to a case insensitive model on relevant platforms, similar to what TS is doing. I don't think we'll do that, because it doesn't solve the problem at core (what if there's a tool that assumes case strictness? It will be the same issue all over again).

• Or, since Yarn enforces casing anyway, maybe the isFileSystemCaseSensitive check from TS should return "true" for Windows when operating within a PnP runtime - because in this case, the filesystem is effectively case sensitive. Since Typescript-Eslint is (correctly) using the TS sys interface, it should pick up the right information and stop normalizing the paths.

I'll apply a change in microsoft/TypeScript#35206 to implement his second option, I'll appreciate if some of you can then test it in your projects.

@arcanis thanks that will be perfect! Once you implement it I will gladly verify it.
Will it be possible to add the fix to the patch logic in yarn as well? edit: I saw you already added it to the patch mechanism, I will verify it shortly and keep you posted

@clemyan thanks for your response, it seems that @arcanis will try to make a fix. In the mean while I posted here a quick fix that you can already use

@bradzacher I'm sorry if it seems like I came too strong in my comment. That was not my intention. I'm an open source maintainer my self and truly appreciate all your work! I was just trying to figure out how to fix the issue properly and I appreciate all the info you provided. Let's see if @arcanis can fix the issue and move forward from there.

@regevbr I've opened a PR, check the instruction inside it: yarnpkg/berry#1205 (comment)

PR confirmed and merged; I've tested myself, and the repository set up by @jstaro doesn't exhibit the problem anymore starting from Yarn master. I think this issue can be closed! 🙂

Thanks @arcanis! I also confirmed on my windows machine that it works great with ts 3.8

Thanks for looking into and fixing this!