Skip to content

Using native JS libs for improved compression performance #577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
flimzy opened this issue Jan 27, 2017 · 2 comments
Open

Using native JS libs for improved compression performance #577

flimzy opened this issue Jan 27, 2017 · 2 comments

Comments

@flimzy
Copy link
Member

flimzy commented Jan 27, 2017

The GopherJS app I'm working on presently depends on Gzip/zlib for a few operations, which are fairly slow in GopherJS at present.

Following in the spirit of #554 and #558, I decided to benchmark using a native JS library for this task, and I have experienced significant performance improvements. I chose the pako library, for it's chunked operation, which makes streaming support possible, but there may be other better libraries to consider. When I refer to pako from here on, assume that I mean pako, or any other lib deemed more appropriate.

My initial benchmarks look promising, with a ~2.25x - 28.7x improvement, with the greater benefit being observed for large inputs. Note these tests are only on 'deflate' operations, and on simple HTML inputs, but it seems sufficient for a proof-of-concept.

BenchmarkStandardZlib1b             2000            574000 ns/op
BenchmarkStandardZlib1kb             500           4282000 ns/op
BenchmarkStandardZlib1mb               1        1145000000 ns/op
BenchmarkPakoZlib1b                 5000            254400 ns/op
BenchmarkPakoZlib1kb                5000            306400 ns/op
BenchmarkPakoZlib1mb                  30          41266666 ns/op

You can see my code here.

How might GopherJS take advantage of this?

Three options come to mind:

  1. Ignore this. A drop-in replacement for zlib/gzip/whatever, could be created as a separate package.

This is obviously the least intrusive, but also the least likely to reap benefits--especially for anyone using a third-party library which depends on compression algorithms, as it would require hacking vendored libraries to take advantage.

  1. Make GopherJS depend on pako.

This seems a bit ugly and evil to me. It would probably make it impossible to install gopherjs using only go tools.(I suppose that's not true, if we were to vendor any 3rd party libs; but that could pose other drawbacks.) So I'm all but completely discounting this option; I mention it only for completeness sake.

  1. Use a polyfill.

The standard compression lib could detect whether $global.pako is defined, and if so, attempt to use it, falling back to the default implementation. This seems quite like the Crypto API solution, the only difference is that pako isn't part of the standard JavaScript Web API.

This would still allow users to load pako before their compiled GopherJS code, and reap the benefits of the improved performance, while not bloating the compiled code (or minimally so).

So my question is, would the polyfill option be of interest to the GopherJS project? I would be interested in working on a PR toward that end, if it's considered worth-while. But there may well be concerns I have not considered. Or maybe we want to see #558 merged before another polyfill library is attempted.

@flimzy flimzy changed the title Using native JS libs for improved performance Using native JS libs for improved compression performance Jan 27, 2017
@dmitshur
Copy link
Member

Do I understand it right that pako is a library for doing de/compression, written in JavaScript, but optimized to be really fast JavaScript? This is not a C++-implementation of de/compression that is a part of the browser and exposed via some JavaScript APIs, the way webcrypto is?

@flimzy
Copy link
Member Author

flimzy commented Jan 27, 2017

Do I understand it right that pako is a library for doing de/compression, written in JavaScript, but optimized to be really fast JavaScript? This is not a C++-implementation of de/compression that is a part of the browser and exposed via some JavaScript APIs, the way webcrypto is?

You understand correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants