Skip to content

Support Go 1.13 and Go 1.14 both #964

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 50 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
6d6d029
compiler/natives/src/internal/reflectlite: add for Go 1.13 internal API
visualfc Jan 25, 2020
7c96e2e
compiler/natives/src/syscall/js: update for Go 1.13 internal API changes
visualfc Jan 25, 2020
752b670
compiler/natives/src/internal/reflectlite,compiler/natives/src/reflec…
visualfc Jan 25, 2020
587694b
update circle.yml go1.13.6
visualfc Jan 26, 2020
8a7356a
compiler/natives/src/syscall/js: update js panic error for Go 1.13 AP…
visualfc Jan 29, 2020
cfd313f
add github.com/visualfc/fastmod for build project use go.mod dep
visualfc Jan 31, 2020
3e8fc9f
add go.mod
visualfc Feb 1, 2020
e230287
build: add internal/goversion for check install go ReleaseTags
visualfc Feb 8, 2020
05591e7
build: add -f --force rebuild package flags. (if update --tags or cha…
visualfc Feb 8, 2020
ea61ac0
compiler/natives/src/sync: support go1.12 go1.13 both
visualfc Feb 8, 2020
ad4d281
compiler/natives/math: fix build for Go 1.14
visualfc Feb 8, 2020
c6d6d64
compiler/natives/src/syscall/js: support Go 1.12 1.13 1.14 three
visualfc Feb 8, 2020
254d3d2
compiler/natives/src/reflect: support Go 1.12 1.13 both
visualfc Feb 8, 2020
afb4460
compiler/natives/fs: fix importPathToDir for Go1.14
visualfc Feb 8, 2020
08185fa
vertion_check: update build for Go 1.13/1.14 and support go=>js for G…
visualfc Feb 8, 2020
07467f7
fix vendor test only Go 1.12
visualfc Feb 8, 2020
f1852a6
update fs_vfsdata.go
visualfc Feb 9, 2020
2452510
fix circle.yml CI failing
visualfc Feb 9, 2020
638a258
update circle.yml go1.12.9
visualfc Feb 9, 2020
f40b392
version_check: support Go 1.12 1.13 both
visualfc Feb 9, 2020
e115225
compiler/natives/src/internal/reflectlite: fix utils.go +build js
visualfc Feb 9, 2020
8153f60
fix pkg.Goroot not load mod
visualfc Feb 9, 2020
9a3412f
update circle.yml go1.13.6
visualfc Feb 9, 2020
edca086
update .std_test_pkg_exclusions
visualfc Feb 10, 2020
bb49db2
tests/sysclljs: add new go113_js_test for Go 1.13 syscall/js
visualfc Feb 10, 2020
0f9071d
tests/sysclljs: add new go114_js_test for Go 1.14 syscall/js
visualfc Feb 10, 2020
e3998bf
compiler/nativas/src/internal/reflectlite: fix rtype kindType convert
visualfc Feb 14, 2020
6879c3c
x
visualfc Feb 14, 2020
b836045
x
visualfc Feb 14, 2020
ff7a802
tests: update knownFails for Go 1.13
visualfc Feb 14, 2020
d15c989
tests: update knownFails:22326 for Go 1.13
visualfc Feb 14, 2020
24e88eb
tests: fix gopherjsvendored_test check GO111MODULE=off
visualfc Feb 14, 2020
8e608e5
compiler/natives/src/internal/reflectlite: simplify code with compile…
visualfc Feb 15, 2020
363e3aa
build: fix mod check skip goroot, rename buildImportPathWithSrcData t…
visualfc Apr 8, 2020
499543d
update fastmod version, fix mod root pkg lookup
visualfc Apr 11, 2020
4e9873f
fix gopherjs serve command mod lookup
visualfc Apr 11, 2020
a6745ae
build: update fastmod check
visualfc Apr 15, 2020
d644f5c
build: Session BuildFiles support GoModules
visualfc Apr 15, 2020
3643fd9
build: simplify mod check and clear code
visualfc Apr 15, 2020
cb53a78
build: fast mod lookup skip std
visualfc Apr 15, 2020
b32521c
change force rebuilding flags to -a
visualfc Apr 15, 2020
088e154
build: fix Import func support absolute path
visualfc Apr 17, 2020
5a3dd28
gopherjs serve args support custom path (absolute path or relative pa…
visualfc Apr 18, 2020
5e486ea
compiler/natives/src/vendor/golang.org/x/sys/cpu: add pkg for go 1.13…
visualfc Apr 18, 2020
f2cbc75
compiler: fix variadic args nil check when zero length
visualfc May 2, 2020
8f52a03
update fastmod dep: fix replace start with ../ prefix
visualfc May 7, 2020
9785635
update goversion deps
visualfc May 16, 2020
cc8eec7
update fastmod dep: fix gomod replace set abs path
visualfc May 28, 2020
a293d52
compiler/natives/src/reflect: add reflect.StructOf implement
visualfc Jun 3, 2020
f12463c
test: update fixedbugs
visualfc Jun 4, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
/node-syscall/build
/node_modules
package-lock.json
22 changes: 22 additions & 0 deletions .std_test_pkg_exclusions
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ crypto
crypto/internal/cipherhw
crypto/tls
crypto/x509/pkix
crypto/ed25519
crypto/ed25519/internal/edwards25519
debug/gosym
debug/plan9obj
encoding
Expand Down Expand Up @@ -63,3 +65,23 @@ testing
testing/internal/testdeps
testing/iotest
unsafe
vendor/golang.org/x/crypto/chacha20poly1305
vendor/golang.org/x/crypto/cryptobyte
vendor/golang.org/x/crypto/cryptobyte/asn1
vendor/golang.org/x/crypto/curve25519
vendor/golang.org/x/crypto/hkdf
vendor/golang.org/x/crypto/internal/chacha20
vendor/golang.org/x/crypto/internal/subtle
vendor/golang.org/x/crypto/poly1305
vendor/golang.org/x/net/dns/dnsmessage
vendor/golang.org/x/net/http/httpguts
vendor/golang.org/x/net/http/httpproxy
vendor/golang.org/x/net/http2/hpack
vendor/golang.org/x/net/idna
vendor/golang.org/x/net/nettest
vendor/golang.org/x/net/route
vendor/golang.org/x/sys/cpu
vendor/golang.org/x/text/secure/bidirule
vendor/golang.org/x/text/transform
vendor/golang.org/x/text/unicode/bidi
vendor/golang.org/x/text/unicode/norm
Comment on lines +68 to +87
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit strange. To my understanding, this list should only cover standard library, so seeing vendored packages is a bit unexpected.

96 changes: 76 additions & 20 deletions build/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,17 @@ import (
"strings"
"time"

"github.com/gopherjs/gopherjs/internal/goversion"

"github.com/fsnotify/fsnotify"
"github.com/gopherjs/gopherjs/compiler"
"github.com/gopherjs/gopherjs/compiler/gopherjspkg"
"github.com/gopherjs/gopherjs/compiler/natives"
"github.com/neelance/sourcemap"
"github.com/shurcooL/httpfs/vfsutil"
"golang.org/x/tools/go/buildutil"

"github.com/visualfc/fastmod"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a general observation, lack of comments makes understanding this PR considerably harder, in particular in the part that's related to modules support, the fastmod package godocs are empty.

This, unfortunately, can be said about GopherJS codebase in general too :-(

)

type ImportCError struct {
Expand Down Expand Up @@ -54,7 +58,7 @@ func NewBuildContext(installSuffix string, buildTags []string) *build.Context {
"netgo", // See https://godoc.org/net#hdr-Name_Resolution.
"purego", // See https://golang.org/issues/23172.
),
ReleaseTags: build.Default.ReleaseTags,
ReleaseTags: goversion.ReleaseTags(),
CgoEnabled: true, // detect `import "C"` to throw proper error

IsDir: func(path string) bool {
Expand Down Expand Up @@ -125,10 +129,10 @@ func Import(path string, mode build.ImportMode, installSuffix string, buildTags
wd = ""
}
bctx := NewBuildContext(installSuffix, buildTags)
return importWithSrcDir(*bctx, path, wd, mode, installSuffix)
return importWithSrcDir(*bctx, path, wd, mode, installSuffix, nil)
}

func importWithSrcDir(bctx build.Context, path string, srcDir string, mode build.ImportMode, installSuffix string) (*PackageData, error) {
func importWithSrcDir(bctx build.Context, path string, srcDir string, mode build.ImportMode, installSuffix string, mod *fastmod.Package) (*PackageData, error) {
// bctx is passed by value, so it can be modified here.
var isVirtual bool
switch path {
Expand All @@ -154,7 +158,24 @@ func importWithSrcDir(bctx build.Context, path string, srcDir string, mode build
mode |= build.IgnoreVendor
isVirtual = true
}
pkg, err := bctx.Import(path, srcDir, mode)
var pkg *build.Package
var err error
if mod != nil && mod.IsValid() && !mod.IsStd() {
if _, dir, typ := mod.Lookup(path); typ != fastmod.PkgTypeNil {
srcDir = dir
pkg, err = bctx.ImportDir(srcDir, mode)
if err == nil {
pkg.ImportPath = path
}
}
}
if pkg == nil {
if filepath.IsAbs(path) {
pkg, err = bctx.ImportDir(path, mode)
} else {
pkg, err = bctx.Import(path, srcDir, mode)
}
}
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -276,11 +297,12 @@ func parseAndAugment(bctx *build.Context, pkg *build.Package, isTest bool, fileS
}

nativesContext := &build.Context{
GOROOT: "/",
GOOS: build.Default.GOOS,
GOARCH: "js",
Compiler: "gc",
JoinPath: path.Join,
GOROOT: "/",
GOOS: build.Default.GOOS,
GOARCH: "js",
Compiler: "gc",
ReleaseTags: goversion.ReleaseTags(),
JoinPath: path.Join,
SplitPathList: func(list string) []string {
if list == "" {
return nil
Expand Down Expand Up @@ -443,6 +465,7 @@ type Options struct {
Minify bool
Color bool
BuildTags []string
Rebuild bool
}

func (o *Options) PrintError(format string, a ...interface{}) {
Expand Down Expand Up @@ -471,11 +494,23 @@ type PackageData struct {
type Session struct {
options *Options
bctx *build.Context
mod *fastmod.Package
Archives map[string]*compiler.Archive
Types map[string]*types.Package
Watcher *fsnotify.Watcher
}

func (s *Session) checkMod(pkg *PackageData) (err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I commented on this on the other PR… Unless I'm misunderstanding the purpose of this function it facilitates passing some module-related information through a side channel, which is a generally problematic pattern (for all the same reasons why global variables are to be avoided).

Aside from that, it creates a risk of hard to debug errors if there's a codepath in which this function is not called.

s.mod.Clear()
if pkg != nil && !pkg.Goroot {
err := s.mod.LoadModule(pkg.Dir)
if err != nil {
return err
}
}
return nil
}

func NewSession(options *Options) *Session {
if options.GOROOT == "" {
options.GOROOT = build.Default.GOROOT
Expand All @@ -490,6 +525,7 @@ func NewSession(options *Options) *Session {
Archives: make(map[string]*compiler.Archive),
}
s.bctx = NewBuildContext(s.InstallSuffix(), s.options.BuildTags)
s.mod = fastmod.NewPackage(s.bctx)
s.Types = make(map[string]*types.Package)
if options.Watch {
if out, err := exec.Command("ulimit", "-n").Output(); err == nil {
Expand Down Expand Up @@ -531,7 +567,7 @@ func (s *Session) BuildDir(packagePath string, importPath string, pkgObj string)
return err
}
pkg.JSFiles = jsFiles
archive, err := s.BuildPackage(pkg)
archive, err := s.buildPackage(pkg)
if err != nil {
return err
}
Expand All @@ -554,6 +590,10 @@ func (s *Session) BuildFiles(filenames []string, pkgObj string, packagePath stri
Dir: packagePath,
},
}
err := s.checkMod(pkg)
if err != nil {
return err
}

for _, file := range filenames {
if strings.HasSuffix(file, ".inc.js") {
Expand All @@ -563,7 +603,7 @@ func (s *Session) BuildFiles(filenames []string, pkgObj string, packagePath stri
pkg.GoFiles = append(pkg.GoFiles, file)
}

archive, err := s.BuildPackage(pkg)
archive, err := s.buildPackage(pkg)
if err != nil {
return err
}
Expand All @@ -574,20 +614,28 @@ func (s *Session) BuildFiles(filenames []string, pkgObj string, packagePath stri
}

func (s *Session) BuildImportPath(path string) (*compiler.Archive, error) {
_, archive, err := s.buildImportPathWithSrcDir(path, "")
_, archive, err := s.BuildImportPathWithPackage(path, nil)
return archive, err
}

func (s *Session) buildImportPathWithSrcDir(path string, srcDir string) (*PackageData, *compiler.Archive, error) {
pkg, err := importWithSrcDir(*s.bctx, path, srcDir, 0, s.InstallSuffix())
func (s *Session) BuildImportPathWithPackage(path string, pkgData *PackageData) (*PackageData, *compiler.Archive, error) {
var srcDir string
var mod *fastmod.Package
if pkgData != nil {
srcDir = pkgData.Dir
if !pkgData.Goroot {
mod = s.mod
}
}
pkg, err := importWithSrcDir(*s.bctx, path, srcDir, 0, s.InstallSuffix(), mod)
if s.Watcher != nil && pkg != nil { // add watch even on error
s.Watcher.Add(pkg.Dir)
}
if err != nil {
return nil, nil, err
}

archive, err := s.BuildPackage(pkg)
archive, err := s.buildPackage(pkg)
if err != nil {
return nil, nil, err
}
Expand All @@ -596,6 +644,14 @@ func (s *Session) buildImportPathWithSrcDir(path string, srcDir string) (*Packag
}

func (s *Session) BuildPackage(pkg *PackageData) (*compiler.Archive, error) {
err := s.checkMod(pkg)
if err != nil {
return nil, err
}
return s.buildPackage(pkg)
}

func (s *Session) buildPackage(pkg *PackageData) (*compiler.Archive, error) {
if archive, ok := s.Archives[pkg.ImportPath]; ok {
return archive, nil
}
Expand Down Expand Up @@ -634,7 +690,7 @@ func (s *Session) BuildPackage(pkg *PackageData) (*compiler.Archive, error) {
if importedPkgPath == "unsafe" || ignored {
continue
}
importedPkg, _, err := s.buildImportPathWithSrcDir(importedPkgPath, pkg.Dir)
importedPkg, _, err := s.BuildImportPathWithPackage(importedPkgPath, pkg)
if err != nil {
return nil, err
}
Expand All @@ -655,7 +711,7 @@ func (s *Session) BuildPackage(pkg *PackageData) (*compiler.Archive, error) {
}

pkgObjFileInfo, err := os.Stat(pkg.PkgObj)
if err == nil && !pkg.SrcModTime.After(pkgObjFileInfo.ModTime()) {
if !s.options.Rebuild && err == nil && !pkg.SrcModTime.After(pkgObjFileInfo.ModTime()) {
// package object is up to date, load from disk if library
pkg.UpToDate = true
if pkg.IsCommand() {
Expand Down Expand Up @@ -691,7 +747,7 @@ func (s *Session) BuildPackage(pkg *PackageData) (*compiler.Archive, error) {
if archive, ok := localImportPathCache[path]; ok {
return archive, nil
}
_, archive, err := s.buildImportPathWithSrcDir(path, pkg.Dir)
_, archive, err := s.BuildImportPathWithPackage(path, pkg)
if err != nil {
return nil, err
}
Expand All @@ -715,7 +771,7 @@ func (s *Session) BuildPackage(pkg *PackageData) (*compiler.Archive, error) {
}

if s.options.Verbose {
fmt.Println(pkg.ImportPath)
fmt.Println(pkg.Dir)
}

s.Archives[pkg.ImportPath] = archive
Expand Down Expand Up @@ -784,7 +840,7 @@ func (s *Session) WriteCommandPackage(archive *compiler.Archive, pkgObj string)
if archive, ok := s.Archives[path]; ok {
return archive, nil
}
_, archive, err := s.buildImportPathWithSrcDir(path, "")
_, archive, err := s.BuildImportPathWithPackage(path, nil)
return archive, err
})
if err != nil {
Expand Down
9 changes: 4 additions & 5 deletions circle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,21 @@ version: 2
jobs:
build:
docker:
- image: circleci/build-image:ubuntu-14.04-XXL-upstart-1189-5614f37
command: /sbin/init
- image: ubuntu:18.04
environment:
SOURCE_MAP_SUPPORT: false
working_directory: ~/go/src/github.com/gopherjs/gopherjs
steps:
- run: apt-get update && apt-get install -y sudo curl git python make g++
- checkout
- run: git clone https://github.com/creationix/nvm $HOME/.nvm && cd $HOME/.nvm && git checkout v0.33.9 && echo 'export NVM_DIR="$HOME/.nvm"' >> $BASH_ENV && echo '[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"' >> $BASH_ENV
- run: nvm install 10.0.0 && nvm alias default 10.0.0
- run: cd /usr/local && sudo rm -rf go && curl https://storage.googleapis.com/golang/go1.13beta1.linux-amd64.tar.gz | sudo tar -xz
- run: cd /usr/local && sudo rm -rf go && curl https://storage.googleapis.com/golang/go1.13.6.linux-amd64.tar.gz | sudo tar -xz
- run: echo 'export PATH="$PATH:/usr/local/go/bin:$HOME/go/bin"' >> $BASH_ENV
- run: go get -t -d -v ./...
- run: go install -v
- run: npm install # Install our (dev) dependencies from package.json.
- run: npm install --global node-gyp
- run: cd node-syscall && node-gyp rebuild && mkdir -p ~/.node_libraries && cp build/Release/syscall.node ~/.node_libraries/syscall.node
- run: cd node-syscall && ../node_modules/node-gyp/bin/node-gyp.js rebuild rebuild && mkdir -p ~/.node_libraries && cp build/Release/syscall.node ~/.node_libraries/syscall.node

- run: go generate github.com/gopherjs/gopherjs/compiler/prelude
- run: diff -u <(echo -n) <(git status --porcelain)
Expand Down
2 changes: 1 addition & 1 deletion compiler/compiler.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import (

var sizes32 = &types.StdSizes{WordSize: 4, MaxAlign: 8}
var reservedKeywords = make(map[string]bool)
var _ = ___GOPHERJS_REQUIRES_GO_VERSION_1_13___ // Compile error on other Go versions, because they're not supported.
var _ = ___GOPHERJS_REQUIRES_GO_VERSION_1_12___ // Compile error on other Go versions, because they're not supported.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be ___GOPHERJS_REQUIRES_GO_VERSION_1_13___ ?


func init() {
for _, keyword := range []string{"abstract", "arguments", "boolean", "break", "byte", "case", "catch", "char", "class", "const", "continue", "debugger", "default", "delete", "do", "double", "else", "enum", "eval", "export", "extends", "false", "final", "finally", "float", "for", "function", "goto", "if", "implements", "import", "in", "instanceof", "int", "interface", "let", "long", "native", "new", "null", "package", "private", "protected", "public", "return", "short", "static", "super", "switch", "synchronized", "this", "throw", "throws", "transient", "true", "try", "typeof", "undefined", "var", "void", "volatile", "while", "with", "yield"} {
Expand Down
3 changes: 2 additions & 1 deletion compiler/expressions.go
Original file line number Diff line number Diff line change
Expand Up @@ -955,7 +955,8 @@ func (c *funcContext) translateConversion(expr ast.Expr, desiredType types.Type)
return c.translateExpr(expr)
}

if c.p.Pkg.Path() == "reflect" {
pkgPath := c.p.Pkg.Path()
if pkgPath == "reflect" || pkgPath == "internal/reflectlite" {
if call, isCall := expr.(*ast.CallExpr); isCall && types.Identical(c.p.TypeOf(call.Fun), types.Typ[types.UnsafePointer]) {
if ptr, isPtr := desiredType.(*types.Pointer); isPtr {
if named, isNamed := ptr.Elem().(*types.Named); isNamed {
Expand Down
8 changes: 8 additions & 0 deletions compiler/go113_version_check.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
// +build go1.13

package compiler

const ___GOPHERJS_REQUIRES_GO_VERSION_1_12___ = true

// Version is the GopherJS compiler version string.
const Version = "1.13-1"
7 changes: 7 additions & 0 deletions compiler/gopherjspkg/fs.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"net/http"
"os"
pathpkg "path"
"path/filepath"

"github.com/shurcooL/httpfs/filter"
)
Expand All @@ -23,6 +24,12 @@ var FS = filter.Keep(
)

func importPathToDir(importPath string) string {
for _, src := range build.Default.SrcDirs() {
dir := filepath.Join(src, importPath)
if _, err := os.Stat(dir); err == nil {
return dir
}
}
Comment on lines +27 to +32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? I'd expect build.Import() to find packages in standard locations. Same below.

p, err := build.Import(importPath, "", build.FindOnly)
if err != nil {
log.Fatalln(err)
Expand Down
Loading