Skip to content

chore: add nx monorepo tooling #3465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jun 9, 2021
Merged

chore: add nx monorepo tooling #3465

merged 9 commits into from
Jun 9, 2021

Conversation

JamesHenry
Copy link
Member

@JamesHenry JamesHenry commented May 30, 2021

@bradzacher As discussed, starting with the most lightweight implementation possible where we just execute essentially the same "targets" (Nx term) as we do currently but going through Nx (+ Nx Cloud) gets us distributed caching for free.

As well as providing monetary contributions every month https://nrwl.io have kindly sponsored us with 10,000 hours of free savings (i.e. not usage, only hours we have truly saved by utilizing the cache) on Nx Cloud.

I have sent you an email invite to add you to the Nx Cloud organization.

NOTE: The only thing about the current setup that is not really compatible with Nx is using : in the names of "targets". This is because : is used as the separator between project names and target names when invoking targets. E.g. to run the "bar" target on the project "foo", the full command would be:

nx run foo:bar

As well as updating the targets which are immediately being utilized with Nx, I updated all remaining usage of : to use - for consistency.

We can evolve this further in future by making more use of nx affected and its knowledge of the dependency graph of the monorepo.

@typescript-eslint
Copy link
Contributor

Thanks for the PR, @JamesHenry!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!


🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint. As a thank you, your profile/company logo will be added to our main README which receives thousands of unique visitors per day.

@nx-cloud
Copy link

nx-cloud bot commented May 30, 2021

Nx Cloud Report

CI ran the following commands for commit 830930c. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

Status Command
#000000 nx run-many --target=build --all --parallel
#000000 nx run-many --target=typecheck --all --parallel

Sent with 💌 from NxCloud.

@JamesHenry
Copy link
Member Author

Something we may or may not want to address here: how come the unit tests are split out into individual invocations per project?

@JamesHenry JamesHenry marked this pull request as ready for review May 30, 2021 13:31
@JamesHenry JamesHenry requested a review from bradzacher May 30, 2021 13:32
@bradzacher bradzacher added the repo maintenance things to do with maintenance of the repo, and not with code/docs label May 30, 2021
@bradzacher
Copy link
Member

bradzacher commented May 30, 2021

Something we may or may not want to address here: how come the unit tests are split out into individual invocations per project?

I did this for a few reasons.

First because lerna streams the parallel output to the console.
So if there are two projects running tests simultaneously, then you'll have the log lines interspersed.
This makes it really hard to read the logs in GH actions if something failed.

Second because lerna prefixes the log lines with the package name.
This also makes it harder to read the logs in GH actions and just adds a bunch of visual noise.

And finally because it makes it trivial to figure out exactly what package failed the tests.
Instead of having to scroll through the test output to look for failures, you just look for the red X:
image

That last point is the same reason I split out the build from the install.
So you can tell the difference between "yarn failed to install for some transient reason" and "the build failed because your types are wrong"

@JamesHenry
Copy link
Member Author

@bradzacher Ok cool, let's punt the testing point for now then - I believe the nx parallelized logs are a little easier to scan than the lerna ones, but I'll do a dedicated evaluation in a follow up draft

@JamesHenry
Copy link
Member Author

JamesHenry commented Jun 1, 2021

Updated dependency graph for the repo:

image

(Docs on how you render this: https://nx.dev/latest/cli/dep-graph)

"check:docs": "lerna run check:docs",
"check:format": "prettier --list-different \"./**/*.{ts,js,json,md}\"",
"check:spelling": "cspell --config=.cspell.json \"**/*.{md,ts,js}\"",
"build": "npx nx prebuild @typescript-eslint/types && nx run-many --target=build --all --parallel",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, we need to include an explicit reference to the prebuild script of @typescript-eslint/types because it is unusual in that it has a side-effect on disk outside of dist/ or build/ (the common output locations Nx checks for) and it therefore does not play well with the automated caching behavior based on simplistic npm scripts (it would be configurable if we moved to full Nx targets in workspace.json).

Feel free to ping me on slack if you want to cover this in more detail and I can walk you through where this is today vs how it can be configured

@JamesHenry JamesHenry merged commit ced9b26 into master Jun 9, 2021
@JamesHenry JamesHenry deleted the nx branch June 9, 2021 07:47
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
repo maintenance things to do with maintenance of the repo, and not with code/docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants