ci(napi): shard NAPI tests #12743
ci(napi): shard NAPI tests #12743
Conversation
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
CodSpeed Instrumentation Performance ReportMerging #12743 will not alter performanceComparing Summary
Footnotes |
|
@Boshen Playground doesn't seem to have any tests. I assume it's included in CI just to make sure it builds without errors. But have I missed something? |
.github/workflows/ci.yml
Outdated
| - uses: oxc-project/setup-rust@cd82e1efec7fef815e2c23d296756f31c7cdc03d # v1.0.0 | ||
| if: steps.filter.outputs.src == 'true' | ||
| with: | ||
| cache-key: napi | ||
| cache-key: napi-${{ matrix.component }} |
There was a problem hiding this comment.
This will bust CI cache https://github.com/oxc-project/oxc/actions/caches, it's already over the limit.
There was a problem hiding this comment.
Weirdly, I'm not sure that's true.
ID KEY SIZE CREATED ACCESSED
809899462 v0-rust-napi-Linux-x64-4d15c146-e92cea93 1.01 GiB about 1 day ago about 3 minutes ago
814552967 v0-rust-napi-parser-Linux-x64-4d15c146-e92cea93 111.99 MiB about 12 hours ago about 7 hours ago
814552798 v0-rust-napi-oxlint2-Linux-x64-4d15c146-e92cea93 203.75 MiB about 12 hours ago about 7 hours ago
814552499 v0-rust-napi-e2e-Linux-x64-4d15c146-e92cea93 132.53 MiB about 12 hours ago about 7 hours ago
814552909 v0-rust-napi-playground-Linux-x64-4d15c146-e92cea93 195.43 MiB about 12 hours ago about 7 hours ago
The individual package caches total 644 MiB, so roughly 400 MiB smaller than the cache from compiling all packages together. Why that would be, I don't know!
(note: napi-e2e cache covers both minify and transform)
Maybe all the caches have accumulated cruft over time, and if we nuke them all and start again, they'll get smaller?
I also noticed something odd. Every cache item has 2 copies e.g.:
Is that normal? Just there's the current cache and the previous most recent?
If so, we're not actually over the limit in practice, because we don't care if the out-of-date caches get deleted.
There was a problem hiding this comment.
To address your comment, I've pushed another commit which returns to building all packages in one job, so there's only 1 cache.
But it doesn't really work - the builds take longer than the tests for all the packages except parser, so to get CI to the finish line faster, building needs to be parallelized, as well as running the tests.
Build job is too slow:
.github/workflows/ci.yml
Outdated
| component: | ||
| - oxlint2 | ||
| - parser | ||
| - playground |
There was a problem hiding this comment.
The entire purpose of this PR is to spawn more jobs!
Could we request an increase in the concurrent jobs limit?
GitHub Support can increase job concurrency limits for GitHub Actions. To request an increase, submit a support ticket.
https://docs.github.com/en/actions/reference/limits#job-concurrency-limits-for-github-hosted-runners
To make sure it builds. |
161af48 to
3a96dc0
Compare
ce53949 to
02504b9
Compare
3a96dc0 to
cf62dcd
Compare
cf62dcd to
35e48c6
Compare
1ddff8c to
d0517b5
Compare
d0517b5 to
509287d
Compare
509287d to
b02457e
Compare
6ce3ac7 to
6ada02a
Compare
6ada02a to
a96e78b
Compare
b02457e to
1ae50b9
Compare
1ae50b9 to
bf35776
Compare
bf35776 to
5d26f1b
Compare
Fix #12435.
Shard NAPI tests, because they're the slowest task in CI.
This split seems fairly decent. Each task is now in the 1m40s - 2m40s range. Previously when they all ran in one task together, it was 7 mins+. This should get PRs through the merge queue a lot faster.