Skip to content

chore: add --changedSince to CI test runs on other Node.js versions #4661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

JoshuaKGoldberg
Copy link
Member

@JoshuaKGoldberg JoshuaKGoldberg commented Mar 9, 2022

PR Checklist

Overview

Uses Jest's --changedSince against the target branch in a PR to cut down on the number of unnecessary tests run on other Node.js versions. This means we'll get reports of unit test failures in eslint-plugin a minute or two sooner!

Doesn't use this in the primary Node.js version because the test coverage report still needs to include all files.

Not that I think this is reachable, but if the ${{ github.event.pull_request.base.sha }} variable is blank/empty then all tests will be run.

@nx-cloud
Copy link

nx-cloud bot commented Mar 9, 2022

☁️ Nx Cloud Report

CI ran the following commands for commit 8ddf512. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch


🟥 Failed Commands
Node 16.x - nx test @typescript-eslint/eslint-plugin --changedSince=bb4dadf6f48269a333cc0f046f07a7ba5f5f915c
✅ Successfully ran 30 targets

Sent with 💌 from NxCloud.

@typescript-eslint
Copy link
Contributor

Thanks for the PR, @JoshuaKGoldberg!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!


🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint. As a thank you, your profile/company logo will be added to our main README which receives thousands of unique visitors per day.

@netlify
Copy link

netlify bot commented Mar 9, 2022

✔️ Deploy Preview for typescript-eslint ready!

🔨 Explore the source changes: 8ddf512

🔍 Inspect the deploy log: https://app.netlify.com/sites/typescript-eslint/deploys/6229fc248e66e000080e81ca

😎 Browse the preview: https://deploy-preview-4661--typescript-eslint.netlify.app

@JoshuaKGoldberg JoshuaKGoldberg changed the title chore: add --changedSince to CI test runs chore: add --changedSince to CI test runs on other Node.js versions Mar 9, 2022
@JoshuaKGoldberg JoshuaKGoldberg force-pushed the ci-jest-changed-since branch from 435a4cb to cb73516 Compare March 9, 2022 23:43
@JoshuaKGoldberg JoshuaKGoldberg marked this pull request as ready for review March 9, 2022 23:54
@bradzacher bradzacher added the repo maintenance things to do with maintenance of the repo, and not with code/docs label Mar 10, 2022
@bradzacher
Copy link
Member

Do we know how good jest's change tracking is here? Can it detect changes to our snapshots correctly?
We might want to leave these packages as is:

  • parser (tests which use globs [1])
  • scope-manager (tests which use globs [1])
  • typescript-estree (tests which use globs [1] [2] [3] [4])

@JamesHenry
Copy link
Member

JamesHenry commented Mar 10, 2022

UPDATE: Sorry I initially misunderstood what this was doing, I see it is related to tests within a particular project - as you were 😅

UPDATE (2): This probably does require some broader thinking through, though, because conceptually the idea of the project graph is what is determining whether or not the test results for the project are restored from cache or not.

If the eslint-plugin project was deemed to not be affected today, the results are ready in less than one second already. So this change is targeting a case where the project graph has determined the project is affected and the tests should therefore be run... Just thinking out loud at this stage but I think this would expose us to some new edge cases.

I would strongly vouch for nx affected (https://nx.dev/using-nx/affected#affected) here instead of this feature - we can apply across all targets not just test/jest

The only reason I didn't do this initially was to slowly dip everyone's toes into the Nx waters and change things as little as possible

I'll put up an alternative PR for us to consider

@codecov
Copy link

codecov bot commented Mar 10, 2022

Codecov Report

Merging #4661 (8ddf512) into main (cf325ce) will decrease coverage by 1.58%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #4661      +/-   ##
==========================================
- Coverage   94.25%   92.67%   -1.59%     
==========================================
  Files         151      317     +166     
  Lines        7971    11299    +3328     
  Branches     2573     3337     +764     
==========================================
+ Hits         7513    10471    +2958     
- Misses        262      563     +301     
- Partials      196      265      +69     
Flag Coverage Δ
unittest 92.67% <100.00%> (-1.59%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
packages/typescript-estree/src/convert.ts 98.37% <100.00%> (ø)
...kages/scope-manager/src/lib/es2017.sharedmemory.ts 100.00% <0.00%> (ø)
packages/scope-manager/src/lib/es2021.ts 100.00% <0.00%> (ø)
packages/scope-manager/src/lib/es7.ts 100.00% <0.00%> (ø)
packages/scope-manager/src/lib/es2015.generator.ts 100.00% <0.00%> (ø)
packages/scope-manager/src/ScopeManager.ts 74.07% <0.00%> (ø)
packages/scope-manager/src/scope/ModuleScope.ts 100.00% <0.00%> (ø)
packages/scope-manager/src/lib/es2020.promise.ts 100.00% <0.00%> (ø)
packages/scope-manager/src/lib/es2015.core.ts 100.00% <0.00%> (ø)
...ckages/typescript-estree/src/ts-estree/ts-nodes.ts 100.00% <0.00%> (ø)
... and 157 more

@JoshuaKGoldberg
Copy link
Member Author

leave these packages as is

Makes sense. Since it's really just eslint-plugin that takes a while, I reduced the changes to only impact that one package.

broader thinking ... nx affected

Yes! I would love to see nx onboarded more fully -- and to learn more about adopting nx in general! Will file a followup task.

@JoshuaKGoldberg
Copy link
Member Author

Blurgh, needs more work on the Git checkout... https://github.com/typescript-eslint/typescript-eslint/runs/5495766395?check_suite_focus=true

    fatal: bb4dadf6f48269a333cc0f046f07a7ba5f5f915c...HEAD: no merge base

@JoshuaKGoldberg JoshuaKGoldberg marked this pull request as draft March 10, 2022 18:00
@JoshuaKGoldberg
Copy link
Member Author

Speaking of those edge cases: I think this is also either blocked on us completely auto-generating .md files, or means we'll need to always test those cases.

@JoshuaKGoldberg
Copy link
Member Author

(closing to keep my queue small)

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 25, 2022
@JoshuaKGoldberg JoshuaKGoldberg deleted the ci-jest-changed-since branch October 25, 2022 21:42
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
repo maintenance things to do with maintenance of the repo, and not with code/docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants