Update everything #45

lovasoa · 2014-04-28T11:00:21Z

I updated old code and added new feature.

More recent version of SQLite (3.8.4)
Compiled to asm.js (should be faster, at least on firefox)
Changed API. Results now have the form [{'columns':[], values:[]}]
Improved GUI of the demo. It now has :
- syntax highlighting
- nice HTML tables to display results
- ability to load and save sqlite database files

If you want to merge only some changes, I can make several distinct pull requests.

…f 'emmake make'

kripken · 2014-04-29T18:23:02Z

Makefile

@@ -1,20 +1,27 @@
 # Need $(EMSCRIPTEN), for example run with        emmake make

-EMCC=$(EMSCRIPTEN)/emcc -s RESERVED_FUNCTION_POINTERS=2 -O2 --closure 1 -s ASM_JS=0
+EMSCRIPTEN?=/usr/bin
+EMCC=EMCC_FAST_COMPILER=1 $(EMSCRIPTEN)/emcc -s RESERVED_FUNCTION_POINTERS=2 -s ALIASING_FUNCTION_POINTERS=1 -s OUTLINING_LIMIT=100 -O3


In current emscripten, no need for EMCC_FAST_COMPILER=1 (on by default). And ALIASING_FUNCTION_POINTERS=1 is the default so that should not be needed either.

Why is there the outlining limit?

kripken · 2014-04-29T18:23:27Z

Thanks for submitting this!

lovasoa · 2014-04-29T18:47:18Z

You’re welcome :)
I will remove EMCC_FAST_COMPILER and ALIASING_FUNCTION_POINTERS
I added OUTLINING_LIMIT because I got a warning ("Output contains some very large functions [...]"). But I still get it, and firefox still takes several hundreds of ms to compile the code. I can't figure out what value to give to OUTLINING_LIMIT...

ispedals · 2014-04-29T21:25:24Z

For what it's worth, this also fixes non-ASCII TEXT values being improperly decoded.

Compile with a more recent version of emscripten.

…s for firefox to compile the asm.js code.

Should be compatible with some older browser versions.

kripken · 2014-04-30T20:47:38Z

An outlining limit of 500 is going to be very bad for throughput, check that.

lovasoa · 2014-04-30T21:00:17Z

But even with that, sqlite3_exec doesn't seem to be outlined, and is
signaled by firefox as being slow to compile. Is there a way to specify
which functions to outline ?

kripken · 2014-05-02T21:27:27Z

No, just by size right now.

Outlining always makes things slower, and is bad for code size too. Perhaps we should ship an outlined and a non-outlined version, so people can choose either fast startup or fast throughout.

lovasoa · 2014-05-03T08:28:22Z

Yes, maybe we should do that.
But I don't know how to compile the quick startup version, since even with a low OUTLINING_LIMIT, firefox takes more than 2 seconds to compile the code, and complains about _sqlite3VdbeExec compiling slowly.

…o it.

lovasoa · 2014-05-03T20:52:01Z

I added an outlining benchmark: http://lovasoa.github.io/sql.js/test/outlining/benchmark.html
(the results may be affected by garbage collection, so you may have to run it several times to get significant results)
I will delete it when it's not needed anymore.
I was a bit surprised by the results: outlining doesn't seem to have any significant impact on the speed of the compiled asm.js code. However, apart from that, it has only negative consequences, as you stated. That can be seen by running the test in chrome.

For now, I chose an OUTLINING_LIMIT of 6500. With that, it takes around 3 seconds for Firefox to compile the code.

I think the only solution to create the fast startup version, is to not to use the amalgamation of sqlite, but the base source code (http://www.sqlite.org/amalgamation.html). What do you think about it?

lovasoa · 2014-05-03T22:28:16Z

Or maybe we could just ship a non-asmjs version, that wouldn't have the compilation overhead...

…tlining. Generate a non-asmjs version with even shorter loading times. Performance is now very good in both Firefox and Chrome with the asm.js version. With the non-asm version, firefox can take a few hundred ms to execute complex queries

lovasoa · 2014-05-05T08:58:52Z

Do you think it can be merged now?

lovasoa · 2014-05-28T18:57:43Z

@kripken ?

…ts failed

Closes #13

Factorize code and produce more descriptive errors

Release notes of sqlite itself are here: http://sqlite.org/releaselog/3_8_5.html

lovasoa · 2014-06-11T09:38:15Z

Hello !

Don't keep track of finalized statements.

kripken · 2014-06-11T23:52:56Z

Sorry, this keeps dropping off my radar...

kripken · 2014-06-11T23:54:14Z

I added you as a collaborator now. Again, sorry - my other emails keep drowning out this one and I miss it. I just don't have the time to focus on this project myself.

lovasoa · 2014-06-12T09:45:34Z

Thanks!

* Update SQLite to 3.8.5 * Change the API * Add support for prepared statements * Add support for BLOBs * Compile to asm.js * Many other things, see the commits

lovasoa added 11 commits April 23, 2014 19:29

Set to /usr/bin if not previously set. Allows to run 'make' instead o…

71d17fb

…f 'emmake make'

Correct printf format

46cff2d

clean. Use clang

6c237cf

Remove warnings

78d60a7

Correct dependencies

e2b768b

Better GUI

347786e

Reflect code changes

fb064ee

change title

2c951d9

Add informations about whats new in the fork

f651f75

Add ability to save and load a database.

67a14ec

extend list of new features

a8bdf14

kripken reviewed Apr 29, 2014
View reviewed changes

lovasoa added 5 commits April 29, 2014 23:34

Update Makefile according to @kripken advices.

5ee0316

Compile with a more recent version of emscripten.

Whoops! I forgot to remove my nasty hack to the Makefile.

f5a11ac

Separate javascript code from HTML

6d6e4bd

Add an outlining_limit again. Otherwise, it would take several second…

2a65161

…s for firefox to compile the asm.js code.

More simple function to generate HTML tables.

a7cc65a

Should be compatible with some older browser versions.

lovasoa added 2 commits May 3, 2014 16:48

Add an outlining performance benchmark

6949388

Update the benchmarks. Choose the default OUTLINING_LIMIT according t…

1d514c7

…o it.

lovasoa added 9 commits May 25, 2014 18:39

Add db.each

9ff31e8

Remove useless console.log

07042f1

worker: Don't throw an error when id is missing

f5a20f2

Update the compiled worker file

0190b08

Add an example of how to write a database to the disk in node

5857536

Add download links

fc3f619

Update HTML example

2f20c02

Update node example

060ebf3

Increment version

bca3458

lovasoa added 13 commits June 3, 2014 11:52

Add @hankinsoft to the authors

3ff4f49

Call postMessage without second argument if using transferable elemen…

5bb3690

…ts failed

Fix a bug in internet explorer.

aa448b3

Closes #13

bump version

3738933

Add debug file, as suggested in #15

5275839

Improve error handling.

de92cea

Factorize code and produce more descriptive errors

Update version number

a909703

Update compiled files

44e183c

Correct headings level

f460869

Add link & reformat downloads

7a08327

Use lowercase keywords

6bf6684

Update version

1d1292d

Update sqlite to 3.8.5

999fd41

Release notes of sqlite itself are here: http://sqlite.org/releaselog/3_8_5.html

Optimize memory management.

7356d05

Don't keep track of finalized statements.

lovasoa added a commit that referenced this pull request Jun 12, 2014

Merge pull request #45 from lovasoa/master

55b6177

* Update SQLite to 3.8.5 * Change the API * Add support for prepared statements * Add support for BLOBs * Compile to asm.js * Many other things, see the commits

lovasoa merged commit 55b6177 into sql-js:master Jun 12, 2014

Update everything #45

Update everything #45

Uh oh!

Conversation

lovasoa commented Apr 28, 2014

Uh oh!

kripken Apr 29, 2014

Choose a reason for hiding this comment

Uh oh!

kripken commented Apr 29, 2014

Uh oh!

lovasoa commented Apr 29, 2014

Uh oh!

ispedals commented Apr 29, 2014

Uh oh!

kripken commented Apr 30, 2014

Uh oh!

lovasoa commented Apr 30, 2014

Uh oh!

kripken commented May 2, 2014

Uh oh!

lovasoa commented May 3, 2014

Uh oh!

lovasoa commented May 3, 2014

Uh oh!

lovasoa commented May 3, 2014

Uh oh!

lovasoa commented May 5, 2014

Uh oh!

lovasoa commented May 28, 2014

Uh oh!

lovasoa commented Jun 11, 2014

Uh oh!

kripken commented Jun 11, 2014

Uh oh!

kripken commented Jun 11, 2014

Uh oh!

lovasoa commented Jun 12, 2014

Uh oh!

Uh oh!