The build is now green. The code should be good to go. I have started some Readme documentation of the compilation scheme, to complement the code comments which are more local. I'll continue in the coming days.

@gzm0 I expect reviewing this will take time (like, weeks or more). Beside the compilation scheme, are there things you would like to see and that would help you review?

I'm already marking you as reviewer, but it's fine if you wait for me to provide everything you will need before starting, obviously.

Also general note about future edits: from now on I will refrain from force-pushing on this branch. I will only add commits, even if they are wip addressing comments (but hopefully each passing the CI regardless). We'll rebase when all the reviewing process over.

Beside the compilation scheme, are there things you would like to see and that would help you review?

I think for now, this is OK. My plan was to first focus on the "outer structure" anyways, so for this, I do not need to know a lot (anything?) about the compilation scheme.

One thing that I'd like to get out of this PR is to get a better understanding how reviews will work going forward in this repository between @sjrd, @tanishiking and me. Basically, I'd like to understand if we need more "formal" structure to ensure that:

• Everybody stays in the loop about changes relevant to their area of expertise.
• We do not waste resources by having multiple people look at changes (I have a feeling this already happens sometimes between @sjrd and me when we have external contributions).

In any case though, I think we'll be better equipped at the end of the review of this PR to answer this question. I just wanted to put it out there, so we do not forget.

from now on I will refrain from force-pushing on this branch

FWIW: For me Github's "viewed" state is likely going to be much more important than individual commits. So for example, feel free to amend the README.md comment as much as you'd like until it is ready for review.

One thing @sjrd you might be able to provide that helps me, is a "suggested order of review" for the two new packages. (I was assuming webassembly should go first, but maybe you have a different, more informed suggestion).

One thing @sjrd you might be able to provide that helps me, is a "suggested order of review" for the two new packages. (I was assuming webassembly should go first, but maybe you have a different, more informed suggestion).

Good point. I would suggest the following:

In webassembly:

1. Start with the set Identities, Types, Modules, Instructions. They define our core immutable model of WebAssembly.
2. ModuleBuilder
3. FunctionBuilder, but ignore the body of def switch as well as def localDeadCodeEliminationOfInstrs (you can come back to those later after getting more comfortable with straightforward Wasm codegen)

This should be enough of webassembly to understand the rest. BinaryWriter and TextWriter can be reviewed independently of everything else. They can be good "lighter review" to do when you don't have much time to commit.

In wasmemitter:

1. Start with EmbeddedConstants and SpecialNames (they are trivial but some of their comments give context)
2. Review TypeTransformer
3. Read the doc comments of VarGen.genFieldID.typeData and its type members
4. Read CoreWasmLib.genNewArrayOfThisClass to get familiar with a) our "DSL" to emit Wasm instructions and b) what working with Wasm instructions looks like. I suggest this one because a) it is a direct equivalent of newArrayOfThisClass in CoreJSLib, b) it manipulates virtually no larger concept of our compilation scheme and c) it is non-trivial nevertheless.
5. Read LoaderContent once (or the generated __loader.js file for a project like the helloworld) to get an idea of what functions exist in there.
6. Read the headers of the other def genX methods of CoreWasmLib (not their implementation), to also get an idea of what helper functions exist.
7. At this point you should have enough overall context to get to get to the meat, so:
8. Read the "API" of WasmContext and WasmContext.ClassInfo.
9. Review FunctionEmitter. Keep genApplyWithDispatch for the end. Only do the "HERE BE DRAGONS" part if you feel adventurous.
10. Review SWasmGen
11. ClassEmitter, but ignore the monster genCreateJSClassFunction
12. Emitter
13. CoreWasmLib
14. WasmContext
15. Preprocessor
16. and finally what's left in no particular order

FYI, I'll mark comments I made as resolved once I feel the discussion is resolved. @tanishiking @sjrd do you feel we need to agree on a scheme on how to resolve comments? (e.g. opener only resolves?)

I think opener resolves, unless it's a typo or something like that.

I believe I have addressed everything so far.

• The createJSError fix was integrated in its original commit, to keep being a stand-alone.
• I now squashed the change to deriving LinkedClasses into the big commit, since that actively reduces the diff.
• The rest is in small separate commits for now.

I think I'm done with the README at this point. There are many more things I could write about, but they are more local, so comments in the code should cover them well. I focused on things that have a global impact across the backend.

Well ... it turns out we didn't need Phase 2 of the bucket assignment algorithm at all! I added a few commits that should address the latest comments on that algorithm.

@tanishiking Could you double-check what I did there?

Here is a gist with the bucket assignments computed for the helloworld and for the test suite:
https://gist.github.com/sjrd/0c10d4cdf43e5a8796ed6d0619edfb34
It's interesting to note that there are barely more buckets for the full test suite than for the helloworld. We basically get 40 buckets because of the huge mess that is the collections hierarchy, but not much more beyond that.

(also the recent changes did not alter the number of buckets allocated for the test suite -- I did not check the helloworld)

And I addressed the remaining comments that were not resolved yet, AFAICT.

Thank you so much @gzm0 for this epic review!