Skip to content

feat(scope-manager): only populate implicit globals whose names appear in the file #10561

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

dmichon-msft
Copy link
Contributor

@dmichon-msft dmichon-msft commented Dec 27, 2024

PR Checklist

Overview

This PR optimizes populateGlobalsFromLib by deduplicating work to calculate the set of variables available in the configured libs array, and by only injecting implicit variable definitions for variables whose names appear in the file being analyzed.

As a consequence of only injecting implicit variables that are actually used in the file, we can avoid the overhead of lint rules spending time processing irrelevant injected globals, for example in typescript-eslint/no-redeclare and typescript-eslint/naming-convention.

Before

populateGlobalsFromLib took 8.556 seconds
findVariablesInScope took 4.077 seconds

After

populateGlobalsFromLib took 0.312 seconds
findVariablesInScope took 0.662 seconds

@typescript-eslint
Copy link
Contributor

Thanks for the PR, @dmichon-msft!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!


🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint.

Copy link

netlify bot commented Dec 27, 2024

Deploy Preview for typescript-eslint ready!

Name Link
🔨 Latest commit 518da6b
🔍 Latest deploy log https://app.netlify.com/sites/typescript-eslint/deploys/6780416d2feba700083d75bd
😎 Deploy Preview https://deploy-preview-10561--typescript-eslint.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 99 (🟢 up 6 from production)
Accessibility: 100 (no change from production)
Best Practices: 92 (no change from production)
SEO: 98 (no change from production)
PWA: 80 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

nx-cloud bot commented Dec 27, 2024

View your CI Pipeline Execution ↗ for commit 518da6b.

Command Status Duration Result
nx run-many --target=build --exclude website --... ✅ Succeeded 5s View ↗
nx run-many --target=clean ✅ Succeeded 10s View ↗

☁️ Nx Cloud last updated this comment at 2025-01-13 01:30:24 UTC

@bradzacher
Copy link
Member

As shown in the failed tests this doesn't work as written. I'm guessing that what's happening is that you're deferring the addition of the variables to after the scopes are closed - causing dangingling, unresolved references.

When a scope is finalised we close off the references within the scope. This means we attempt to resolve any unresolved references created in the current or child scopes to variables defined in the current scope. If we don't find the right variable the reference is left as unresolved and we continue to the parent scope.

So if you don't declare the implicit globals early enough then they can't be resolved and so the references are left dangling - which then causes the no-undef rule to error on them.

@dmichon-msft
Copy link
Contributor Author

dmichon-msft commented Dec 28, 2024

There's a reason this is a draft. One thing I know I'm doing wrong here is that if there two lib records define the nature of an implicit variable differently (type-only vs. value-only vs. value-and-type), one of them wins at effectively random.

As written this theoretically should run before the global scope is closed, but I'll need to dig into it a bit more. I just wanted to put this out there to get some eyes on the concept.

Ah, I just realized that part of the change is missing here. The invocation of populateGlobalsFromLib was supposed to have been moved to between this.visitChildren() and this.close() in Program.

Copy link

codecov bot commented Dec 28, 2024

Codecov Report

Attention: Patch coverage is 90.00000% with 7 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@290211b). Learn more about missing BASE report.
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...ackages/scope-manager/src/referencer/Referencer.ts 90.00% 7 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #10561   +/-   ##
=======================================
  Coverage        ?   86.95%           
=======================================
  Files           ?      446           
  Lines           ?    15571           
  Branches        ?     4532           
=======================================
  Hits            ?    13540           
  Misses          ?     1676           
  Partials        ?      355           
Flag Coverage Δ
unittest 86.95% <90.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...ackages/scope-manager/src/referencer/Referencer.ts 95.49% <90.00%> (ø)

@dmichon-msft dmichon-msft changed the title Performance: Only add implicit globals from lib that are observed in the file feat(scope-manager): Only populate implicit globals whose names appear in the file Dec 30, 2024
@dmichon-msft dmichon-msft changed the title feat(scope-manager): Only populate implicit globals whose names appear in the file feat(scope-manager): only populate implicit globals whose names appear in the file Dec 30, 2024
@dmichon-msft dmichon-msft marked this pull request as ready for review December 31, 2024 04:34
@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Jan 13, 2025

This is really cool, thanks for sending @dmichon-msft! I'd love to dive in soon and see if we can get this shipped. But - we're a little swamped catching up on issues after the holiday & new year break, and this is a lot of very nuanced code in a not-very-often changed area of the project. So it'll probably be a little while till we can take a deep look.

In the meantime, what would help us review more quickly:

  • I see there are some "Added ... was not covered by tests" annotations: could you either test those, or leave comments explaining that they are tested but the annotations are inaccurate (e.g. from Repo: Merge codecov report from ast-spec into typescript-estree's report #6701 or similar)?
  • The populateGlobalsFromLib function has gotten huge and IMO much less readable. maybe split out some well-named helper functions to encapsulate more of the used-only-in-one-place logic/variables?
    • Example: entriesForLib could be a const if the retrieval logic were moved to a separate function
    • Example: implicitVariablesFromLibs's creation & cache checking could be moved to a separate function
  • You mentioned local testing: could you post your reproduction(s) and how you got the measurements?

Btw, you don't need to keep merging the latest main in: https://typescript-eslint.io/contributing/pull-requests#updating-over-time. We don't want to give that burden to contributors. Especially if us taking a while to get to a PR makes it worse!

@JoshuaKGoldberg JoshuaKGoldberg added the awaiting response Issues waiting for a reply from the OP or another party label Jan 13, 2025
@dmichon-msft
Copy link
Contributor Author

I'm slicing out the easy perf win (caching of Object.entries calls) into its own PR that will be much simpler to review, then I'll refactor this a bit and add more testing to the pieces that power it.

@JoshuaKGoldberg
Copy link
Member

I'm slicing out the easy perf win (caching of Object.entries calls) into its own PR that will be much simpler to review, then I'll refactor this a bit and add more testing to the pieces that power it.

👍 in that case drafting this PR in the meantime. Let me know if it's actually ready for review and I misinterpreted. Exciting stuff! 🙂

@JoshuaKGoldberg JoshuaKGoldberg marked this pull request as draft February 24, 2025 15:58
@JoshuaKGoldberg
Copy link
Member

JoshuaKGoldberg commented Apr 7, 2025

👋 Just checking in @dmichon-msft, do you still have time for this very exciting perf work?

@dmichon-msft
Copy link
Contributor Author

The PR with the bigger part of the performance win merged, so this hasn't been high on my priority list. This wouldn't be terribly difficult to get going again, I'd just largely forgotten about it. The new structure should make this pretty simple.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting response Issues waiting for a reply from the OP or another party
Projects
None yet
Development

Successfully merging this pull request may close these issues.

⚡️ Performance: Overhead of populateGlobalsFromLib in scope-manager
3 participants