This guide is for GopherJS contributors interested in adding support for the latest Go version into GopherJS. If you are only using GopherJS, you probably don't need to worry about it 🙂

GopherJS supports most of the standard library packages. In fact, it reuses most of their sources with only minimal modifications. Go compatibility promise assures that all public interfaces will remain backwards compatible within Go 1.x, however, that doesn't apply to internal details of standard library implementation GopherJS relies upon. Besides that, new Go releases sometimes introduce brand new language features that must be supported by the compiler itself. As a result, supporting a new minor Go release often requires non-trivial work on GopherJS side. This guide will describe the general workflow we follow, although each release is different and brings its unique challenges with it.

Before we jump into the guide itself, let's discuss a few important code locations:

build directory contains logic that's involved with findling, loading, parsing sources and applying standard library overlays. In most cases new Go versions should not change much, unless there was some major change to Go's build systems (like Go Modules or the embed package).

compiler directory is where all the Go-to-JavaScript translation logic is concentrated. If the new Go version introduces new language features, you will most likely need to make changes here. Also sometimes new tests for old features are added, which reveal bugs GopherJS compiler has, which can also be fixed here.

compiler/natives/src contains standard library overlay sources. Instead of using traditional patches, GopherJS uses AST-level overlays, which are merged into the upstream standard library sources on the fly. This has an advantage of using the regular Go syntax that can be easily edited and better resilience to upstream source code reshuffling. When building a standard package GopherJS compiler appends all overlay sources to the list of originals and builds them together. If overlay sources contain types, functions or variables of the same name as in the upstream source, the upstream one is removed from the AST. This gives us an easy way to patch parts of the standard library that are incompatible with the browser environment. The overlay sources are embedded into the compiler binary.

compiler/prelude contains low-level javascript code required by the GopherJS runtime. Things that can't be easily implemented in Go, foundations of type system, helper functions used by the generated code go there.

Of course, as any project under active development, code and structure are changing, so nothing can replace exploring the code yourself 😉

To get started, we need to install the new Go toolchain and update several places in the code to make GopherJS target it. We will use Go 1.18 as an example for this guide, make sure to substitute it with the relevant version when following the steps.

Follow official instructions to install/upgrade your system Go toolchain or install it in parallel. The latter option is particularly convenient when working with a beta or RC release.

Next, there are 4 places in the GopherJS codebase to be updated for the new Go version:

• Go version used in CI workflows, defined in the env → GO_VERSION variable. It uses semver syntax and should pin the Go version to a specific minor release, for example ~1.18.2:
  • Main CI workflow in .github/workflows/ci.yml.
  • Reference app size measurement in .github/workflows/measure-size.yml.
• Several places in compiler/version_check.go:
  • Build constraint at the top of the file, for example //go:build go1.18. This prevents building GopherJS with an earlier, incompatible version of Go toolchain.
  • Version constant that defines the GopherJS version, for example 1.18.0+go1.18.2. See GopherJS versioning convention for details.
  • GoVersion constant that defines the minor number of the Go version this GopherJS version is compatible with, for example 18.
• All references to a specific Go or GopherJS version in README.md.

See this commit as a complete example.

These changes are pretty much guaranteed to break a lot of tests in CI, so we usually do this development in a separate branch (e.g. go1.18) until the branch is stable enough to be merged into master.

NOTE: This section assumes a Linux environment. While GopherJS does support Mac OS and Windows for normal use, many standard library tests don't pass on platforms other than Linux. It would be great to fix that one day, but today is not that day 🙃

GopherJS reuses most of the standard library for the wasm architecture. In fact, internally when building standard library packages it swaps the target from GOOS=js GOARCH=ecmascript to GOOS=js GOARCH=wasm to make this easy. However, not all wasm codebase is automatically compatible with GopherJS, which leads to build or test failures. Broadly, there are a few common breakage causes:

• Upstream code changes that utilize features GopherJS doesn't support well, such as unsafe. Those are resolved with reimplementing the code in a GopherJS-compatible way using standard library overlays.
• Upstream code changes that became incompatible with pre-existing overlays (e.g. function or type renamed). Many such cases can be solved by adjusting the existing overlays to mirror upstream changes.
• New language or compiler feature that is not supported by GopherJS yet. The "Stage 4" section addresses these in more detail.

Begin by making sure that the runtime package builds:

gopherjs $ go install . && gopherjs build -v runtime

The runtime package is automatically included into every build, so no build will succeed if runtime or one of its dependencies is broken. Note that runtime is one of the few packages where only overlay sources are included into the build. This is because the low-level runtime details in GopherJS are different enough from the upstream to make the upstream sources pretty much irrelevant.

Once runtime builds successfully you can try building some of the basic std packages. fmt and log are good candidates because they include some non-trivial code, but aren't too complicated to be hard to diagnose. Finally, we need to make sure that the testing package builds and passes its own test; otherwise we can’t rely on it to run any other tests:

gopherjs $ go install . && gopherjs test -v testing
…
ok      testing 2.152s

Next, is an iterative process running the tests, triaging and fixing failures. The easiest way of doing this is to push changes into your private fork and let CI do its job. Alternatively, if you have a powerful local machine it might be faster to run tests locally:

PACKAGE_NAMES=($(GOOS=js GOARCH=wasm go list std github.com/gopherjs/gopherjs/js/... github.com/gopherjs/gopherjs/tests/... \
    | grep -v -x -f .std_test_pkg_exclusions));
gopherjs test --minify --short "${PACKAGE_NAMES[@]}";

TIP: When investigating a build error, it is often useful to look at the history of the recent commits affecting the problematic package in the upstream Go repository. Often you can easily spot the breaking change and mirror it on the GopherJS side.

When it comes to test failures, there are several possible outcomes:

• A newly introduced regression, or a bug that can be fixed in a reasonably straightforward way. This is often the case for newly introduced APIs or bugfixes in packages like reflect or syscall/js, which GopherJS heavily modifies. Here is an example of such a fix. It is, of course, the most desirable outcome 😊
• A test that relies on features GopherJS can't possibly support, for example allocation-counting tests, or tests that depend on the OS environment we can't emulate. In such cases the test function can be overridden using the overlay system and t.Skip() can be used to indicate that the test is bypassed intentionally. Please make sure to document what feature in particular makes the test inapplicable (example).
  • TIP: Sometimes only a small part of the test requires the unsupported functionality, in such cases the edited version of the test can be copied into the overlays (example).
• A test that relies on a particularly large new feature of the compiler or the build toolchain that requires a separate effort to support. In such a case it could be reasonable to file a separate tracking issue and use it as a justification to skip the test (for example TestIssue50208 in this commit, due to missing generics support). Eventually those tests will be re-enabled, but it may not be worth blocking the whole new version process on it.
• In a similar vein, sometimes a new test is added for a pre-existing issue, which is very difficult to fix for one reason or another. Filing an issue and referencing it in the skip message is a reasonable, even if not desirable option.

Although it may be tempting to skip inconvenient test failures, it is important to remember that they often reveal legitimate bugs, and ignoring them may be a disservice to our users.

Finally, you can check out this twitter thread, which has a few hands-on, illustrated examples of troubleshooting and fixing standard library.

In addition to standard library tests, GopherJS is tested against the "fixedbugs" test suite from Go the repo. It contains regression tests for compiler bugs that were found in the upstream Go compiler and is a good way to assure GopherJS's correctness as well. These tests are executed by the gorepo_tests CI workflow, or can be triggered locally:

$ go install github.com/gopherjs/gopherjs && go test github.com/gopherjs/gopherjs/tests/gorepo -v --count=1
…
ok      fixedbugs/issue9691.go  0.285s
ok      fixedbugs/issue9738.go  0.288s
  443 ok
   66 knfl
--- PASS: TestGoRepositoryCompilerTests (40.17s)
PASS
ok      github.com/gopherjs/gopherjs/tests/gorepo       40.176s

Note a few important moments about this command:

• The test expects the up-to-date gopherjs binary available in the $PATH, hence the go install part.
• The test is invoked by go test here, not gopherjs test. This is because under the hood it needs to exec other commands, which is unsupported by the gopherjs runtime.
• We pass the --count=1 flag to make sure the go command doesn't cache execution results.

If all is well, the test will report a number of "okay" and "knfl" (a.k.a "known failure") test cases and passing result. The test may fail if there are new failures that aren't already known, or any known failures are passing again, both deserve further investigation :)

Debugging an individual failure is easier when you can run the test case in isolation:

$ go install . && go run ./tests/gorepo/run.go -summary -v -- fixedbugs/issue15992.go  # With the test wrapper.
goos: "js", goarch: "ecmascript"
parallel: 12
ok      fixedbugs/issue15992.go 0.753s
    1 ok
$ go install . && gopherjs run $(go env GOROOT)/test/fixedbugs/issue15992.go  # Execute directly.
3 [97 98 99]
3 [97 98 99]
1
0

A test case fixedbugs/issueXYZ.go is considered passing if:

• fixedbugs/issueXYZ.out exists and stdout of the executed test matches its content.
• fixedbugs/issueXYZ.out does not exist and the test exit code is zero.

The triage process for these tests is similar to the standard library:

• Most failures are expected to be fixable. Surprisingly, many upstream compiler bugs are also found in GopherJS, despite completely separate implementations.
• Some of test tests target features specific to the upstream compiler, such as race detector, or certain flags, which aren't applicable to GopherJS. Such tests can be added to the knownFailures set in the tests/gorepo/run.go with a comment explaining the reason the test is irrelevant.
• Rarely, the test uncovers a legitimate bug, fixing which would require a lot of effort, or deep knowledge of the compiler. Make sure to file a gopherjs issue for such bugs, and link it when adding the test into known failures.

This Twitter thread contains an illustrated example of troubleshooting compiler test failures.

This is the most challenging, but also the most fun part of the process 😅 Since language features are different every time, there is no universal recipe for how to implement them. Packages such as go/types and go/build do a lot of heavy lifting for GopherJS, so in most cases we only need to worry about changes to code generation and runtime.

As a general rule, we try to support all new language features that were introduced into upstream Go as soon as we support it. However, some features can be large and complicated (looking at you, generics!), and could significantly delay a GopherJS release. By consensus among the maintainers we may decide to leave this feature out of the release and introduce it at a later stage.

Phew! We did it! The tests are passing, features are supported, fun has been had, time to deliver the fruits of labor to GopherJS users 🎉

This is the only part of the process that requires write access to the repository.

1. First, upgrade the tested Go version (as explained in Stage 1) to the latest patch release. This is to make sure that we pick up any late changes or bugfixes. In most cases this requires no additional effort, or occasionally addressing a couple of new failing tests. Stages 2 and 3 describe this process.
2. So far all the changes have been kept in a separate branch, which need to be merged to master. We usually do this via a formal pull request, which is merged once all maintainers agree that we are ready and no critical bugs remain unaddressed.
3. Create a new release on GitHub, make sure to use the correct format for the version tag, for example v1.18.0+go1.18.2. Write the release notes, calling out important changes and giving kudos to everyone who contributed to the release; GitHub's "generate release notes" usually provides good raw material.
4. Once the release is published, manually create and push an additional release tag without the + part, for example: v1.18.0. Even though allowed by semver, Go tooling sometimes doesn't like the suffix and doesn't pick it up ¯_(ツ)_/¯ Make sure that the tag is on the same commit as the new release.

And that's all there is to it! 😉