Skip to content

Best practice for distributing Go packages that make use of GopherJS at runtime? #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dmitshur opened this issue Sep 29, 2014 · 16 comments · Fixed by shurcooL/Go-Package-Store#18

Comments

@dmitshur
Copy link
Member

One blocking issue I'm running into with shurcooL/Go-Package-Store#18 right now is this.

It imports .../gopherjs/build (indirectly) and compiles the frontend .go code to JavaScript when serving html.

That works fine. The problem is with distributing the root package.

Normally, when you do go get -u some/package, go get will recursively get all of its dependencies.

So, currently to install Go Package Store (with javascript, not GopherJS):

go get -u github.com/shurcooL/Go-Package-Store

However, since the .go code for GopherJS is hidden behind a // +build js tag, the dependencies of the go code meant to be compiled by GopherJS compiler are not fetched.

I tried to do an import with blank identifier to force go get to get dependencies, but that causes compile errors as the meant-for-GopherJS has init() and global variables that are executed and cause panics, since the DOM isn't available.

My best runner-up alternative solution right now is to change the installation steps to:

go get -u github.com/shurcooL/Go-Package-Store
go get -u -d -tags=js github.com/shurcooL/Go-Package-Store/...

But that's twice the steps and pretty unidiomatic...

I'm looking for suggestions for better ways, if anyone can share some ideas.

(Note that I don't want to include the generated javascript inside the repository; I'd really prefer to have it compiled on demand from source when the process starts. It's not very slow, and it makes the code cleaner. But of course that's a viable alternative; to bake in the generated javascript instead of generating it at runtime... This is something go generate may come in handy for.)

@neelance
Copy link
Member

Have you tried to remove // +build js from script.go and then install with

go get -v github.com/shurcooL/Go-Package-Store/...

?

@dmitshur
Copy link
Member Author

That would fetch the dependencies, but generate errors when go get tries to install the meant-for-GopherJS Go package that assumes the DOM is available. See:

I tried to do an import with blank identifier to force go get to get dependencies, but that causes compile errors as the meant-for-GopherJS has init() and global variables that are executed and cause panics, since the DOM isn't available.

Edit: Actually, I realized those errors are runtime, not compile time.

Still, it would compile the meant-for-GopherJS Go package and place it in the user's $GOPATH/bin, which is less than desirable. Especially since that binary will simply panic if you try to run it. :/

@neelance
Copy link
Member

Still, it would compile the meant-for-GopherJS Go package and place it in the user's $GOPATH/bin, which is less than desirable. Especially since that binary will simply panic if you try to run it. :/

Yes, that's true. Maybe the easiest solution would be to fix the panics in init()? Where exactly does it crash?

@dmitshur
Copy link
Member Author

It panics because I do silly things like this, for shortness:

var document = dom.GetWindow().Document()

Which succeeds when dom is actually present in the frontend, but otherwise I'm getting a nil pointer dereference panic.

Maybe the easiest solution would be to fix the panics in init()?

That will work, albeit force more verbosity onto the frontend-targetted .go files (which are already more verbose than the pure JavaScript equivalents for simple things due to imports, package clause, main func declaration). But I'm not sure I'd be quite happy with that solution and not sure if it can scale to solve it for larger projects. It can perhaps work in the interim, but I'm still looking for a better solution.

@neelance
Copy link
Member

neelance commented Nov 2, 2014

What about hiding only the problematic lines behind a // +build js wall? Keeping the rest compatible with plain Go also has the advantage that you can provide unit tests for some parts and test them without GopherJS.

@dmitshur
Copy link
Member Author

dmitshur commented Nov 2, 2014

What about hiding only the problematic lines behind a // +build js wall?

Interesting idea. I will play with it. Thanks!

@dmitshur
Copy link
Member Author

dmitshur commented Nov 2, 2014

Actually, that won't work to solve the original problem, since the dependencies that are hidden behind // +build js build tag will not be fetched by the go tool.

I'm starting to warm up to the idea of just having this step as part of the installation instructions:

go get -u -d -tags=js github.com/shurcooL/Go-Package-Store/...

It does exactly what's needed, guaranteed to work, and doesn't bloat the generated backend Go code with meaningless imports of frontend packages that aren't actually used in backend.

Also, I'm looking at potentially shifting to pre-compiling the frontend dependencies and embedding them into the backend as compressed binary data via a go generate step that makes use of go-bindata. That way, when an end user go gets my package, it will have everything ready to run. But for development, I will still have the benefit of hot reload of frontend resources via an alternative dev code path.

@neelance
Copy link
Member

neelance commented Nov 2, 2014

Actually, that won't work to solve the original problem, since the dependencies that are hidden behind // +build js build tag will not be fetched by the go tool.

Of course you'll still need all dependencies in files that don't have // +build js. I'm suggesting to only hide the func init()s.

@neelance
Copy link
Member

Did it work?

@dmitshur
Copy link
Member Author

I haven't tried that approach. I don't think it's a good idea to import frontend Go code into the backend, even if it doesn't do anything, just because it would bloat up the size and doesn't feel right.

My current best bet is to try using go generate together with go-bindata and move generating GopherJS code (as compressed blob) at go generate time rather than at runtime. It will be some time before I get time to try it, though.

@neelance
Copy link
Member

Okay. Can we close this issue?

@dmitshur
Copy link
Member Author

You can if you want to, but I don't consider this issue solved yet.

As far as I can tell, at this moment there is no good way to distribute a Go application with the frontend component written in Go and have the frontend dependencies be acquired as part of go get. I haven't been able to merge shurcooL/Go-Package-Store#18 because of it, and I really want to.

Note that it's not a problem if you're willing to compile the Go to JavaScript and include that as part of your backend Go package, but that was something I wanted to avoid for now if possible.

I still think there's a potential solution via go generate turning the generated JavaScript into a .go file containing the JavaScript, compressed. But it will be some time before I can implement it and see if it works well.

It would be great if other people found and suggested other ideas that we haven't considered yet. I think solving this challenge is will be one of few remaining milestones to make GopherJS-powered frontends more flexible and available for wider audiences.

@dmitshur
Copy link
Member Author

dmitshur commented Jan 1, 2015

Made some progress on this today, I was able to use a go:generate directive to statically build GopherJS packages into .js files for distribution.

It's still not as clean as I want it to be, because it adds a non-compressed .js file to the repo.

Next steps will be to try using go-bindata to achieve that.

@dmitshur
Copy link
Member Author

Made significant progress on this today, and I think I'm very close to being able to (rightfully) close this issue. I'll post updates when it's complete.

@neelance
Copy link
Member

Cool! I'm curious to see what you come up with.

@dmitshur
Copy link
Member Author

I've finally put everything together, cleaned it up (for the most part) and committed. The solution can be seen in the updated https://github.com/shurcooL/Go-Package-Store/pull/18/files.

I will merge that PR soon and it will close this issue. Thank you for the patience, as it took me a long time to resolve this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants