[WIP] Add an Inside Rust blog post for stage 0 std redesign #1582

jieyouxu · 2025-04-17T07:15:30Z

Tip

There are intentionally a lot of WIP intermediate commits for review purposes, I will rebase once the content is ready.

Accompanying blog post for:

Stage 0 std redesign: redesign stage 0 std rust#119899, and
Making it possible to use stage 0 libtest for compiletest: make it possible to use stage0 libtest on compiletest rust#139386.

r? @onur-ozkan (and @Mark-Simulacrum)
cc @rust-lang/bootstrap (significant stage 0 bootstrap changes)

Rendered

jieyouxu · 2025-04-17T07:19:00Z

Hi @senekor, do you know what this zola failure is? Or am I missing sth in the front-matter?

Run zola build
Building site...
Checking all internal links with anchors.
> Successfully checked 254 internal link(s) with anchors.
-> Creating 600 pages (0 orphan) and 2 sections
Error: Failed to build the site
Error: Failed to render 'feed.xml'
Error: Reason: Filter call 'nth' failed
Error: Reason: Filter `nth` received an incorrect type for arg `n`: got `-2` but expected a usize
Error: Process completed with exit code 1.

I guess I'm missing a path = ""...? Or what am I supposed to put there?

content/inside-rust/stage0-std-redesign.md

senekor · 2025-04-17T07:33:03Z

Yes, the path key is required. The format has to be "YYYY/MM/DD/whatever-you-want".

I'll try to improve the templates so they produce better error messages in this case.

senekor · 2025-04-17T07:38:38Z

Oh if it's an inside-rust post, that should be in the path as well. So: "inside-rust/YYYY/MM/DD/whatever".

If the date of publication is not yet known, it's recommended to use the placeholder "9999/12/31" as the date, CI will prevent that from being published accidentally.

I can definitely improve docs/guidance/tooling/error msgs here.

jieyouxu · 2025-04-17T07:45:39Z

content/inside-rust/stage0-std-redesign.md

@@ -0,0 +1,54 @@
+++
+path = "inside-rust/9999/12/31/redesigning-stage-0-std"


FIXME(jieyouxu): update this date when we're ready

onur-ozkan

Great work!

Just two suggestions in addition to my minor notes:

It'd be helpful to mention the implementation PR.
Consider showing the staging diagram to illustrate how things looked before and how they look now (the updated version is in the implementation PR).

content/inside-rust/stage0-std-redesign.md

jieyouxu · 2025-04-17T07:49:37Z

It'd be helpful to mention the rust-lang/rust#119899.

I have no idea how I didn't include that when I even remember telling myself not to forget that lmao

jieyouxu · 2025-04-17T08:37:08Z

Changes since last review (in round 2 222f4fe):

Added a comparison diagram showing difference before/after the redesign.
Addressed nits.
Actually link to the stage 0 redesign PR itself.

content/inside-rust/stage0-redesign/index.md

jieyouxu · 2025-04-17T11:31:53Z

content/inside-rust/stage0-redesign/index.md

TODO: having someone on t-libs and someone on t-compiler skim it and sign off that it makes sense

content/inside-rust/stage0-redesign/index.md

jieyouxu · 2025-04-29T14:49:41Z

content/inside-rust/stage0-redesign/index.md

+
+### Caveat: `libtest` changes
+
+See the *Recommended config: have `compiletest` instead depend on precompiled stage 0 libtest* section below.


FIXME(jieyouxu) [COMPILETEST_STAGE0 1/2]: drop this if we remove the compiletest force stage0 hack

Dropped, but blocked on removing the hack

content/inside-rust/stage0-redesign/index.md

jieyouxu · 2025-04-29T14:50:38Z

content/inside-rust/stage0-redesign/index.md

+## Recommended config: have `compiletest` instead depend on precompiled stage 0 libtest
+
+`compiletest` currently depends on in-tree libtest. For workflows that involve building the compiler, this can cause `compiletest` rebuilds if stage 1 library is rebuilt as a consequence of compiler changes. If you don't intend to change libtest, you can [specify `build.compiletest-use-stage0-libtest = true` to instead have `compiletest` depend on precompiled stage 0 libtest](https://github.com/rust-lang/rust/pull/139386). That way, compiler test iterations can avoid rebuilding `compiletest` unnecessarily. This is already the default unless the `dist` profile is used.
+
+Note that some CI jobs have `build.compiletest-use-stage0-libtest = true` set while others have `build.compiletest-use-stage0-libtest = false`, meaning that libtest programmatic API changes can require adding `cfg(bootstrap)`/`cfg(not(bootstrap))` to `compiletest`'s libtest API use sites. However, in practice we expect this to be very rare.


FIXME(jieyouxu) [COMPILETEST_STAGE0 3/3]: drop this once we drop the hack

Dropped, but blocked on removing the hack

jackh726 · 2025-04-29T15:56:07Z

content/inside-rust/stage0-redesign/index.md

+team_url = "https://www.rust-lang.org/governance/teams/infra#team-bootstrap"
+++
+
+# Redesigning stage 0 std


I would maybe change the title to be something like "Redesigning stage 0 during compiler bootstrapping" (i.e. give a little bit more of a hint at who will be affected, for those not familiar with what "stage 0" is)

I might rename this to sth like "Redesigning the stage 0 compiler bootstrap sequence", not sure if that makes more sense?

Btw the heading should be removed, the title attribute in the frontmatter is authoritative. Otherwise it creates this:

I might rename this to sth like "Redesigning the stage 0 compiler bootstrap sequence", not sure if that makes more sense?

I agree to find a less dense title. Another (more verbose) proposal:

Redesigning rustc initial bootstrap stage to compile the standard library

or something like that. Titles are hard 😅

jackh726 · 2025-04-29T15:57:30Z

content/inside-rust/stage0-redesign/index.md

+
+## Summary
+
+We are reworking how the stage 0 bootstrap sequence works (the sequence used to build a stage 1 compiler).


Similarly, I would put a little bit of a lead up here on why this change is desired (which I think will naturally require a little bit of background to flow well).

Yeah, previously the motivation was up front, I'll need to balance it a bit.

jackh726 · 2025-04-29T16:04:43Z

content/inside-rust/stage0-redesign/index.md

+
+## What does this mean for a typical compiler workflow?
+
+- If unstable library features are being dogfooded, some additional `cfg(bootstrap)` usages may now be needed in compiler sources.


I think this would really benefit from some example(s). For example, what would this look like for e.g. a new trait solving feature that requires some core lang items?

(Aside, not the right place for this, but this effectively means that such features actually will need to a stage 2 compiler to be built to properly test them, right?)

jackh726 · 2025-04-29T16:05:50Z

content/inside-rust/stage0-redesign/index.md

+
+## Why are we making this change?
+
+The previous stage 0 bootstrapping sequence was a source of endless confusion for compiler, library and bootstrap contributors alike, because std had to support being built by *both* a previous beta rustc and in-tree rustc, with `cfg(bootstrap)` in std sources necessary to distinguish between them. By adjusting the stage 0 bootstrap sequence to instead use a precompiled stage 0 std instead of building the in-tree std, we hope to:


Said above, a very brief summary of this could be put at the beginning.

jackh726 · 2025-04-29T16:07:58Z

content/inside-rust/stage0-redesign/index.md

+
+The previous stage 0 bootstrapping sequence was a source of endless confusion for compiler, library and bootstrap contributors alike, because std had to support being built by *both* a previous beta rustc and in-tree rustc, with `cfg(bootstrap)` in std sources necessary to distinguish between them. By adjusting the stage 0 bootstrap sequence to instead use a precompiled stage 0 std instead of building the in-tree std, we hope to:
+
+1. Simplify library development workflow to no longer need `cfg(bootstrap)`, and


Also an aside and shouldn't really be discussed here, but I personally think cfg(bootstrap) in library code is likely easier to maintain and reason about than cfg(bootstrap) in compiler code.

Though, perhaps I'm wrong, in which case I think this blog post can do better at convincing me this isn't the case.

It may be easier, but it's much more common. Instances of cfg(bootstrap) should be really rare, while in the library it's incredibly common, especially for larger refactors that were simply not done previously because it's so bad

BoxyUwU

Some random thoughts.

In general I feel like this post doesn't do too much to explain why the new system is a clearer/simpler mental model. And I think that does a bit of a disservice to all of the (great) explanation of what that actually means in terms of changes to building the compiler/library.

It's almost like the post frames this as an arbitrary change in what depends on what because it just so happens to make std's life simpler. When, as far as I can tell, the new system is just generally more coherent and how you would expect things to work.

E.g. generally when a rust crate is built by compiler X it also depends on the standard library built and shipped with/by X. With the previous bootstrapping setup the compiler was actually depending on some weird frankenstein std that was built by the beta compiler but wasn't shipped with it.

Similarly the inconsistency between "build the stage0 compiler" and "build the stage0 std library" was weird and has now been resolved (stage0 compiler is beta compiler, stage0 library was a weird frankenstein std but is now the beta std), but doesn't really seem to be mentioned anywhere in this post.

I think it would be a lot easier to understand what has changed if the post started with the motivation and talked about the shift in mental model to something that's generally more coherent. It's hard to fully understand what has changed without talking about how the change fits in with the wider "heres how things normally work".

Generally that's what my review comments are trying to get at I think 🤔 Small changes that would help build the right mental model for readers- but realistically it should probably be all together in one upfront paragraph for the motivation.

BoxyUwU · 2025-04-29T17:05:09Z

content/inside-rust/stage0-redesign/index.md

+- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.
+- After: a stage 0 (beta by default) compiler and a precompiled stage 0 std is instead used to build the stage 1 compiler.
+


I wonder if this could do with a short summary with less of the... I hesitate to say jargon, but something like:

Suggested change

- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.

- After: a stage 0 (beta by default) compiler and a precompiled stage 0 std is instead used to build the stage 1 compiler.

- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.

- After: a stage 0 (beta by default) compiler and a precompiled stage 0 std is instead used to build the stage 1 compiler.

In other words, in the typical workflow, the stage1 compiler now depends on beta std instead of the in-tree std (built by the beta compiler).

Generally I find the current "before/after" description to be a little bit hard to parse 🤔

BoxyUwU · 2025-04-29T17:11:12Z

content/inside-rust/stage0-redesign/index.md

+Notably, this means that after [redesign stage 0 std #119899](https://github.com/rust-lang/rust/pull/119899) PR lands:
+
+- `./x {build,test,check} library --stage 0` becomes no-op, as stage 0 std is no longer the built in-tree std, and the minimum supported stage to build std is now `1`.
+    - Consequently, default (test, check, bench) stage values in the library profile are no longer `0`, but instead defaults to `1`.
+- Some additional `cfg(bootstrap)` usages may be needed in the compiler sources for dogfooding unstable library features.


I'm not sure how you'd fit in but I wonder if it'd be helpful to say something like "stage0 now always refers to things from the beta toolchain (unless otherwise customized)".

"If unstable library features are being dogfooded"

dogfooding = technical lingo, which might be ok if we assume a specific target audience. Could it be reworded to be less technical?

I can use "experimenting with unstable library features", sure.

This internal blog post is targeted towards people (compiler/library contributors) who'd already be exposed to terms like "dogfooding", but "experimenting" is less opaque, so sure.

BoxyUwU · 2025-04-29T17:23:53Z

content/inside-rust/stage0-redesign/index.md

+
+We are reworking how the stage 0 bootstrap sequence works (the sequence used to build a stage 1 compiler).
+
+- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.


I also think that having to clarify that stage0 is "technically not always the beta compiler" makes things a bit harder to understand 🤔 Almost every contributor is going to have stage0 being the beta compiler so even if its not necessarily true I do wonder if it would make sense to lie a bit with a footnote the first time and then just ignore it from then on:

Suggested change

- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.

- Before: the beta compiler[^1] is used to build the in-tree std, which in turn is used to build the stage 1 compiler.

[^1]: In practice, rarely, the stage0 compiler can be something other than the beta compiler. This doesn't matter too much for the purpose of this change though, so we've taken the liberty of conflating the two to make it easier to explain the change to bootstrapping.

Although maybe this actually doesn't matter too much if you have a motivation section upfront discussing mental models so that there's already a nice intuition built for "stage0 == beta mostly sort of"

BoxyUwU · 2025-04-29T17:26:59Z

content/inside-rust/stage0-redesign/index.md

+
+| Invocation                                 | Before stage 0 std redesign                                 | After stage 0 std redesign                               |
+|--------------------------------------------|-------------------------------------------------------------|----------------------------------------------------------|
+| `./x check library`                        | Checks in-tree library                                      | Check/build/tests stage 1 library                        |


No, that's a mistake from moving things around, good catch

BoxyUwU · 2025-04-29T17:27:09Z

content/inside-rust/stage0-redesign/index.md

+| Invocation                                 | Before stage 0 std redesign                                 | After stage 0 std redesign                               |
+|--------------------------------------------|-------------------------------------------------------------|----------------------------------------------------------|
+| `./x check library`                        | Checks in-tree library                                      | Check/build/tests stage 1 library                        |
+| `./x {build,test} library`                 | Builds in-tree library, builds/tests stage 1 library        | Check/build/tests stage 1 library                        |


Suggested change

| `./x {build,test} library` | Builds in-tree library, builds/tests stage 1 library | Check/build/tests stage 1 library |

| `./x {build,test} library` | Builds in-tree library, builds/tests stage 1 library | build/tests stage 1 library |

meithecatte · 2025-04-30T01:20:25Z

content/inside-rust/stage0-redesign/index.md

+Notably, this means that after [redesign stage 0 std #119899](https://github.com/rust-lang/rust/pull/119899) PR lands:
+
+- `./x {build,test,check} library --stage 0` becomes no-op, as stage 0 std is no longer the built in-tree std, and the minimum supported stage to build std is now `1`.
+    - Consequently, default (test, check, bench) stage values in the library profile are no longer `0`, but instead defaults to `1`.


Suggested change

- Consequently, default (test, check, bench) stage values in the library profile are no longer `0`, but instead defaults to `1`.

- Consequently, default (test, check, bench) stage values in the library profile are no longer `0`, but instead default to `1`.

Noratrieb · 2025-04-30T06:02:38Z

content/inside-rust/stage0-redesign/index.md

+
+## Why are we making this change?
+
+The previous stage 0 bootstrapping sequence was a source of endless confusion for compiler, library and bootstrap contributors alike, because std had to support being built by *both* a previous beta rustc and in-tree rustc, with `cfg(bootstrap)` in std sources necessary to distinguish between them. By adjusting the stage 0 bootstrap sequence to instead use a precompiled stage 0 std instead of building the in-tree std, we hope to:


I have an alternative description of the motivation that I think more accurately captures the effect on combined compiler/library changes. Feel free to merge and integrate or so whatever with it that you want:

The compiler and standard library are tightly coupled. Intrinsics and lang items form a broad interface between the two, and whenever it is changed, both sides need to adjust. But the standard library supports being compiled with two different compilers, the in-tree compiler and the stage0 (beta) compiler. This means that all such changes require cfg(bootstrap) to support both compilers, which results in a lot of churn of anyone wanting to improve things that touch this area.

Yeah, I think that's a way better motivation summary. I'll need to think about how to reword this section.

clubby789 · 2025-04-30T11:41:50Z

content/inside-rust/stage0-redesign/index.md

+|--------------------------------------------|-------------------------------------------------------------|----------------------------------------------------------|
+| `./x check library`                        | Checks in-tree library                                      | Check/build/tests stage 1 library                        |
+| `./x {build,test} library`                 | Builds in-tree library, builds/tests stage 1 library        | Check/build/tests stage 1 library                        |
+| `./x {check,build,test} library --stage 0` | Checks/builds/tests in-tree library                         | No-op in case of build, checks/tests precompiled library |


What does it mean to check the precompiled library?

It's a no-op (arguably not exactly that, it's "does nothing"/"unsupported"?), I'll reword that

Add a blog post for stage 0 std redesign

b412814

Kobzol reviewed Apr 17, 2025

View reviewed changes

content/inside-rust/stage0-std-redesign.md Outdated Show resolved Hide resolved

content/inside-rust/stage0-std-redesign.md Outdated Show resolved Hide resolved

[WIP] Address review comments (round 1)

c8d819a

[WIP] Fix missing path

330fecd

jieyouxu commented Apr 17, 2025

View reviewed changes

onur-ozkan reviewed Apr 17, 2025

View reviewed changes

content/inside-rust/stage0-std-redesign.md Outdated Show resolved Hide resolved

content/inside-rust/stage0-std-redesign.md Outdated Show resolved Hide resolved

senekor mentioned this pull request Apr 17, 2025

Improve error message in case of invalid path #1584

Merged

jieyouxu force-pushed the stage0-std-redesign branch from 7e0d43d to c8bc2d5 Compare April 17, 2025 08:33

[WIP] Address review feedback (round 2)

222f4fe

jieyouxu force-pushed the stage0-std-redesign branch from c8bc2d5 to 222f4fe Compare April 17, 2025 08:35

jieyouxu commented Apr 17, 2025

View reviewed changes

content/inside-rust/stage0-redesign/index.md Show resolved Hide resolved

jieyouxu mentioned this pull request Apr 29, 2025

redesign stage 0 std rust-lang/rust#119899

Open

jieyouxu commented Apr 29, 2025

View reviewed changes

jieyouxu added 2 commits April 29, 2025 23:25

[WIP] Address review feedback (round 3)

dbd84f8

[WIP] Add dedicated support thread

ad5aa2b

jackh726 reviewed Apr 29, 2025

View reviewed changes

BoxyUwU reviewed Apr 29, 2025

View reviewed changes

meithecatte reviewed Apr 30, 2025

View reviewed changes

Noratrieb reviewed Apr 30, 2025

View reviewed changes

clubby789 reviewed Apr 30, 2025

View reviewed changes

		@@ -0,0 +1,54 @@
		+++
		path = "inside-rust/9999/12/31/redesigning-stage-0-std"


		### Caveat: `libtest` changes

		See the Recommended config: have `compiletest` instead depend on precompiled stage 0 libtest section below.


		## Summary

		We are reworking how the stage 0 bootstrap sequence works (the sequence used to build a stage 1 compiler).


		## What does this mean for a typical compiler workflow?

		- If unstable library features are being dogfooded, some additional `cfg(bootstrap)` usages may now be needed in compiler sources.


		## Why are we making this change?

		The previous stage 0 bootstrapping sequence was a source of endless confusion for compiler, library and bootstrap contributors alike, because std had to support being built by both a previous beta rustc and in-tree rustc, with `cfg(bootstrap)` in std sources necessary to distinguish between them. By adjusting the stage 0 bootstrap sequence to instead use a precompiled stage 0 std instead of building the in-tree std, we hope to:


		The previous stage 0 bootstrapping sequence was a source of endless confusion for compiler, library and bootstrap contributors alike, because std had to support being built by both a previous beta rustc and in-tree rustc, with `cfg(bootstrap)` in std sources necessary to distinguish between them. By adjusting the stage 0 bootstrap sequence to instead use a precompiled stage 0 std instead of building the in-tree std, we hope to:

		1. Simplify library development workflow to no longer need `cfg(bootstrap)`, and

		- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.
		- After: a stage 0 (beta by default) compiler and a precompiled stage 0 std is instead used to build the stage 1 compiler.

-- Before: a stage 0 (beta by default) compiler is used to build the in-tree std, which in turn is used to build the stage 1 compiler.
+- Before: the beta compiler[^1] is used to build the in-tree std, which in turn is used to build the stage 1 compiler.
+[^1]: In practice, rarely, the stage0 compiler can be something other than the beta compiler. This doesn't matter too much for the purpose of this change though, so we've taken the liberty of conflating the two to make it easier to explain the change to bootstrapping.

	\| `./x check library` \| Checks in-tree library \| Check/build/tests stage 1 library \|
	\| `./x check library` \| Checks in-tree library \| Check stage 1 library \|

	\| `./x {build,test} library` \| Builds in-tree library, builds/tests stage 1 library \| Check/build/tests stage 1 library \|
	\| `./x {build,test} library` \| Builds in-tree library, builds/tests stage 1 library \| build/tests stage 1 library \|

	- Consequently, default (test, check, bench) stage values in the library profile are no longer `0`, but instead defaults to `1`.
	- Consequently, default (test, check, bench) stage values in the library profile are no longer `0`, but instead default to `1`.

[WIP] Add an Inside Rust blog post for stage 0 std redesign #1582

Are you sure you want to change the base?

[WIP] Add an Inside Rust blog post for stage 0 std redesign #1582

Conversation

jieyouxu commented Apr 17, 2025 • edited by rustbot Loading

jieyouxu commented Apr 17, 2025 • edited Loading

senekor commented Apr 17, 2025

senekor commented Apr 17, 2025

Choose a reason for hiding this comment

onur-ozkan left a comment

Choose a reason for hiding this comment

jieyouxu commented Apr 17, 2025

jieyouxu commented Apr 17, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jieyouxu Apr 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BoxyUwU left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BoxyUwU Apr 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jieyouxu Apr 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jieyouxu Apr 30, 2025 • edited Loading

Choose a reason for hiding this comment

jieyouxu commented Apr 17, 2025 •

edited by rustbot

Loading

jieyouxu commented Apr 17, 2025 •

edited

Loading

jieyouxu commented Apr 17, 2025 •

edited

Loading

jieyouxu Apr 29, 2025 •

edited

Loading

BoxyUwU Apr 29, 2025 •

edited

Loading

jieyouxu Apr 29, 2025 •

edited

Loading

jieyouxu Apr 30, 2025 •

edited

Loading