Skip to content

compiler: remove dependency on legacy gcimporter15 #803

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 24, 2018

Conversation

myitcv
Copy link
Member

@myitcv myitcv commented Apr 21, 2018

This is a follow up to #789 that removes the sticking plaster in that
PR, and switches the compiler to using
golang.org/x/tools/go/gcexportdata as directed here
https://github.com/golang/tools/blob/14d5b80f954f510ba9bcd241821b6f4828498e1a/go/gcimporter15/gcimporter.go#L13-L14.

This means we can also remove the temporary vendor of gcimporter15.

However we instead vendor golang.org/x/tools/go/gcexportdata and
golang.org/x/tools/go/internal/gcimporter to avoid hassle for people
upgrading to this version of GopherJS. We do this as of
https://github.com/golang/tools/tree/94b14834a20132093826ea5e2da5502a13908ad3,
minus *_test.go files and testdata directories.

@myitcv myitcv force-pushed the remove_dependency_on_gcimport15 branch 3 times, most recently from b8344cb to f473b75 Compare April 22, 2018 07:05
@myitcv myitcv changed the title [WIP] compiler: remove dependency on legacy gcimporter15 compiler: remove dependency on legacy gcimporter15 Apr 22, 2018
@myitcv
Copy link
Member Author

myitcv commented Apr 22, 2018

@shurcooL @hajimehoshi this is now ready for review.

Copy link
Member

@hajimehoshi hajimehoshi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Member

@dmitshur dmitshur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for working on this!

It's great to see the new non-deprecated package for this functionally is such a seamless fit, so we no longer have to rely on a vendored version of a deleted/deprecated package. Thanks for doing the investigation and implementing this update.

I have 3 specific change requests, see inline comments, and after that this will have my approval.

One more thing. In the past, when changing the packages used for doing the export/import of package data, I bumped up the compiler version number. While this seems like it works without issues (CI is passing), it still feels like the type of change that's more risky, so bumping up the version number would help with catching issues specific to this change.

So, what do you think of bumping up the version after merging this PR as well?

@@ -243,7 +243,8 @@ func ReadArchive(filename, path string, r io.Reader, packages map[string]*types.
}

var err error
_, packages[path], err = gcimporter.BImportData(token.NewFileSet(), packages, a.ExportData, path)
b := bytes.NewReader(a.ExportData)
packages[path], err = gcexportdata.Read(b, token.NewFileSet(), packages, path)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b is not a descriptive variable name. That's often okay in short functions, but it doesn't work well in this context. It's also typically associated with []byte type, not io.Reader. I see r is already taken and I don't think it's a good idea to reuse it.

I think it's better to either use a longer descriptive name here:

exportData := bytes.NewReader(a.ExportData)
packages[path], err = gcexportdata.Read(exportData, token.NewFileSet(), packages, path)

Which would mirror the gcexportdata.Write code below.

But I think this line is not that long, that it's even better to just inline it:

packages[path], err = gcexportdata.Read(bytes.NewReader(a.ExportData), token.NewFileSet(), packages, path)

Either of those is okay.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

exportData := new(bytes.Buffer)
if err := gcexportdata.Write(exportData, nil, typesPkg); err != nil {
return nil, fmt.Errorf("failed to write export data: %v", err)
}
encodedFileSet := bytes.NewBuffer(nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line uses a very outdated style of initializing a new bytes.Buffer, no one writes it like that in anymore. bytes.NewBuffer documentation hints at this:

In most cases, new(Buffer) (or just declaring a Buffer variable) is sufficient to initialize a Buffer.

It's not surprising because this line was written 4 years ago.

I know it's not your code, but it's the line directly following your code, and I think it's not nice how just 4 lines above it is a different style is used to do the same thing:

exportData := new(bytes.Buffer)
// ... 3 lines
encodedFileSet := bytes.NewBuffer(nil)

Let's make this better by updating it so they both use the same style.

I would slightly prefer the var buf bytes.Buffer (and using & operator) style over buf := new(bytes.Buffer):

var exportData bytes.Buffer
if err := gcexportdata.Write(&exportData, nil, typesPkg); err != nil {
	return nil, fmt.Errorf("failed to write export data: %v", err)
}
var encodedFileSet bytes.Buffer
if err := fileSet.Write(json.NewEncoder(&encodedFileSet).Encode); err != nil {

But if you feel strongly about using new(bytes.Buffer) for both variables (and not using & operator), I'm okay with that too.

Also, I know this file has yet another instance of bytes.NewBuffer(nil), but it's fine to leave that alone, it's too far from the affected code and hence out of scope of this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// Go 1.8 export data files, so they will work before and after the
// Go update. (See discussion at https://github.com/golang/go/issues/15651.)
//
package gcexportdata // import "golang.org/x/tools/go/gcexportdata"
Copy link
Member

@dmitshur dmitshur Apr 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the now-unused old gcimporter15 package from vendor directory is the right thing to do. However, we shouldn't vendor the new gcexportdata package.

This project currently follows the pattern of vendoring as few packages as possible, only the ones that have to be vendored. The new gcexportdata package is not deprecated (nor removed from upstream) and doesn't need to be vendored at this time.

Not vendoring it means when newer version of gcexportdata is released, installing the latest version of GopherJS will automatically use the latest (otherwise, it would not be updated until we manually do so). This is intentional. At this time, we prefer the reduced maintenance burden by not vendoring when it's not required.

This PR should follow the current pattern and not deviate from it.

In the PR description, you said:

we instead vendor golang.org/x/tools/go/gcexportdata and
golang.org/x/tools/go/internal/gcimporter to avoid hassle for people
upgrading to this version of GopherJS.

I don't understand what hassle you're referring to. To update to a newer version of GopherJS, per the installation instructions, people do go get -u github.com/gopherjs/gopherjs. That automatically gets the latest versions of dependencies. Please let me know if I'm misunderstanding something, but I don't believe there is an increased hassle for users. (And if there is, we should deal with this outside of this PR if it involves other non-vendored packages too.)

So, please remove the newly vendored packages from this PR, just keep the deletion of gcimporter15.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the vendor. The situation I was referring to in the commit message is for anyone, like me, who might update the GopherJS repo in isolation. But it's a minor point and not necessarily an improvement in the whole so I've removed it.

myitcv added 2 commits April 23, 2018 05:20
This is a follow up to gopherjs#789 that removes the sticking plaster in that
PR, and switches the compiler to using
golang.org/x/tools/go/gcexportdata as directed here
https://github.com/golang/tools/blob/14d5b80f954f510ba9bcd241821b6f4828498e1a/go/gcimporter15/gcimporter.go#L13-L14.

This means we can also remove the temporary vendor of gcimporter15.
@myitcv myitcv force-pushed the remove_dependency_on_gcimport15 branch from af80693 to 67e5760 Compare April 23, 2018 04:20
@myitcv
Copy link
Member Author

myitcv commented Apr 23, 2018

So, what do you think of bumping up the version after merging this PR as well?

I don't have a strong feeling either way, so if you would like to do a release after this PR please do.

@myitcv
Copy link
Member Author

myitcv commented Apr 23, 2018

@shurcooL - comments addressed, commit message updated - I think this is therefore now ready. Thanks for reviewing.

Copy link
Member

@dmitshur dmitshur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing those. LGTM.

Let's wait a few days before merging this. I'd like to have a chance to test this change locally a bit to gain more confidence. This PR is pretty orthogonal so it should be okay to merge at any time, but please let me know if you would like to get it merged sooner.

We'll also increase the compiler version after merging.

@myitcv
Copy link
Member Author

myitcv commented Apr 23, 2018

Let's wait a few days before merging this. I'd like to have a chance to test this change locally a bit to gain more confidence.

Ok, will leave it to you to merge when you're comfortable.

Incidentally, if there are tests/checks that you perform locally that help to give you confidence that this PR "works" we should look to add those tests/checks (or equivalent tests/checks) to the CI; that way we can just merge changes in the future, confident that things will just "work".

We'll also increase the compiler version after merging.

Ok.

@dmitshur
Copy link
Member

Incidentally, if there are tests/checks that you perform locally that help to give you confidence that this PR "works" we should look to add those tests/checks (or equivalent tests/checks) to the CI; that way we can just merge changes in the future, confident that things will just "work".

Nope, what I had in mind is just casual testing by trying it locally, and seeing if anything unexpected comes up (e.g., when upgrading from the older version, etc.). I think we have complete test coverage already.

@dmitshur
Copy link
Member

dmitshur commented Apr 24, 2018

Tested locally, not seeing any issues. Merging and bumping version.

Thanks again!

@dmitshur dmitshur merged commit 423bf76 into gopherjs:master Apr 24, 2018
dmitshur added a commit that referenced this pull request Apr 24, 2018
This version uses a new package for export data. See PR #803.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants