Skip to content

Go: Deal with incorrect toolchain versions #15979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Mar 28, 2024
Merged

Conversation

mbg
Copy link
Member

@mbg mbg commented Mar 19, 2024

Starting in Go 1.21, Go uses a new version format for toolchains which uses the 1.N.P syntax. This is a problem for our users when their go.mod files only contain a go directive that uses the old version format, e.g. go 1.22. 1.22 is a valid language version, but not a valid toolchain version. If there is no additional toolchain directive in the go.mod file, then go uses the language version as the toolchain version. In the example, 1.22 is not a valid toolchain version and certain go commands, particularly those which try to acquire the toolchain corresponding to that version, fail with an unhelpful error.

In this PR, we make two changes to the Go autobuilder:

  1. We detect go.mod files where the language version is 1.21 or greater, doesn't use the new toolchain version syntax, and there is no explicit toolchain directive. We emit a warning-level diagnostic in this case.
  2. We explicitly invoke go version with GOTOOLCHAIN=go1.N.P (i.e. the correct representation of the toolchain version) to get go to install that version and avoid subsequent errors despite the incorrect version format. As an optimisation for repos with a lot of go.mod files, we now keep track of toolchain versions that we know are installed and only do this if we don't yet know that the corresponding version is installed.

@mbg mbg self-assigned this Mar 19, 2024
@github-actions github-actions bot added the Go label Mar 19, 2024
@mbg mbg force-pushed the mbg/go/deal-with-incorrect-versions branch from 6a14e3f to c74d634 Compare March 20, 2024 13:55
@mbg mbg marked this pull request as ready for review March 20, 2024 13:55
@mbg mbg requested a review from a team as a code owner March 20, 2024 13:55
Copy link
Contributor

@smowton smowton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to test an actual download, consider setting GOPATH to an empty directory so it certainly won't find an existing toolchain downloaded by a prior integration test or similar.

workingDir,
)

// Run the command. If something goes wrong, report it to the log and signal failure
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider marking this as "installed" regardless to avoid repeatedly attempting to install the same thing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about this. It seems like a rare scenario where we would repeatedly attempt to install the same version and that fails for whatever reason.

If I had implemented this set specifically for the new check, then okay, but since the implementation is more general, it might lead to weird results down the line if we want to utilise the set of installed versions for something else and find that in some cases those aren't actually installed.

@mbg
Copy link
Member Author

mbg commented Mar 20, 2024

@smowton We already set a custom GOPATH for every integration test (as of #15355) and that doesn't seem to affect which toolchains go can/cannot find. I also experimented with overriding GOTOOLCHAIN to force a particular toolchain, but couldn't get that to work right because we eventually want to switch to the newer one during the test and there seems to be no good way to get that to switch at the right point.

@smowton
Copy link
Contributor

smowton commented Mar 20, 2024

seems to work locally for me (GOTOOLCHAIN=go1.22.1 go version; GOPATH=/tmp/fresh GOTOOLCHAIN=go1.22.1 go version). If you print the gopath during the test is it what you expect / is .go empty as expected?

@mbg
Copy link
Member Author

mbg commented Mar 27, 2024

@smowton weirdness seems to have been due to the awful semver behaviour that requires versions to start with a v character so the changes in toolchain.go where we kept track of installed Go versions always ended up inserting/checking for "" in the set since semver.Canonical returns "" if the version is not valid.

I have fixed those bugs and added an integration test.

@mbg mbg requested review from smowton and owen-mc March 27, 2024 15:37
owen-mc
owen-mc previously approved these changes Mar 28, 2024
Co-authored-by: Owen Mansel-Chan <62447351+owen-mc@users.noreply.github.com>
Co-authored-by: Owen Mansel-Chan <62447351+owen-mc@users.noreply.github.com>
@mbg mbg merged commit 73f71d8 into main Mar 28, 2024
12 checks passed
@mbg mbg deleted the mbg/go/deal-with-incorrect-versions branch March 28, 2024 14:16
redsun82 pushed a commit that referenced this pull request Apr 24, 2025
This diagnostic was introduced by #15979.
However in the meantime the Go team [has backtracked](golang/go#62278 (comment))
on their decision, which leads to confusing alerts for user (e.g. github/codeql-action#2868).
Even using Go toolchains from 1.21 to 1.22 we weren't immediately able
to reproduce the problem that this diagnostics was meant to guard
against. Therefore it was deemed simpler to just remove it.

_En passant_ the `Makefile` now accepts `rtjo` not being set.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants