Skip to content

Support for Emscripten #2618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 16 commits into from
Closed

Conversation

matthewelse
Copy link

@matthewelse matthewelse commented Nov 10, 2016

This is an initial port of MicroPython to compile to JS with Emscripten. It includes a basic REPL application written in JS for testing, and could easily be extended to run in a browser using jq-console.

It probably needs testing more thoroughly, and additional features such as using JavaScript's GC might be interesting future work.

See previous efforts here: #1561 #888

(Some of the) remaining work to do:

  • Importing modules (e.g. setting up Python path etc.)

cc: @piranna @jaustin

@matthewelse
Copy link
Author

matthewelse commented Nov 10, 2016

There are a couple of changes that I made which need to be modified to prevent them breaking other builds, but I'll have a look at those tomorrow.

Copy link

@piranna piranna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need it I could take a look on it since I have more experience with that. When merged next step would be to configure the repo and CI server to generate and host the prebuild images as releases.

"name": "micropython",
"version": "1.8.6",
"description": "micropython compiled to ASM.js with emscripten",
"main": "main.js",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nain file must point to one that can be require()d (importable), use bin field and rename the file to server.js instead, this would allow to exec it with npm start too :-)

"main": "main.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"install": "make"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use prebuild-install module to install prebuild versions of the module.

@dpgeorge
Copy link
Member

Thanks @matthewelse, this is good progress! But please take note of #2602.

@piranna
Copy link

piranna commented Nov 11, 2016

Maybe we could work on @matthewelse fork for the Javascript things (npm, repl...) until main micropython one gets cleaned and later merge both of them... :-) I could clearly work on that area, not only because I proposed the original idea but also I have been working a lot lately with Node.js and know how it and the npm ecosystem works... :-)

@piranna
Copy link

piranna commented Nov 11, 2016

Later it would be just a matter of update the URLs to point to the official repo :-P

@pfalcon
Copy link
Contributor

pfalcon commented Nov 11, 2016

@matthewelse : If you're serious about getting this merged upstream eventually, you would read https://github.com/micropython/micropython/wiki/ContributorGuidelines , especially part around "When you make commits for a change, please follow format of existing commit messages (use "git log" to review them)."

@matthewelse
Copy link
Author

@pfalcon I've reworded the commit messages to follow the format that seems to be usually followed - once people are happy to merge this, I'll squash the commits down, since there's quite a bit of redundancy

@pfalcon
Copy link
Contributor

pfalcon commented Nov 11, 2016

Sounds good!

@piranna
Copy link

piranna commented Nov 11, 2016

Why the name of micropython-js for the CLI? Is it not a (C)Python compatible REPL? I think we should move it to make it a replacement if we want it to make an alternative to use the system install CPython binary...

@matthewelse
Copy link
Author

@piranna - I'm not sure what you mean, are you saying that we should call it python or micropython or something like that?

@piranna
Copy link

piranna commented Nov 11, 2016

@piranna - I'm not sure what you mean, are you saying that we should call it python or micropython or something like that?

Yes, that's it. In fact, is you use {"bin": "repl.js"} it will automatically use the module name (micropython) from the package.json file as the name of the CLI binary. Also I would add some of the most common cpython command line arguments, like -c and -m and the general behaviour of cpython binary for executing scripts.

On a side note, since the binary filename doesn't matter, I usually name it server.js so it can be executed with npm start, since it's its default value.

@matthewelse
Copy link
Author

I think a small amount of work is needed to set up the Python path and that sort of thing to get importing modules working, but it shouldn't be too bad.

@piranna
Copy link

piranna commented Nov 11, 2016

I think a small amount of work is needed to set up the Python path and that sort of thing to get importing modules working, but it shouldn't be too bad.

If we could import both Python and Node.js modules with the same import syntax that would be awesome... :-D Although maybe this would need some help/changes of the runtime... The main problem with that would be that Node.js allow to export directly an object while Python doesn't... :-/

@matthewelse
Copy link
Author

matthewelse commented Nov 11, 2016

If we could import both Python and Node.js modules with the same import syntax that would be awesome...

Agreed this would be awesome - perhaps we should consider that in another issue/PR, and keep this one primarily for doing the initial implementation of the port so it doesn't get too big.

@piranna
Copy link

piranna commented Nov 11, 2016

perhaps we should consider that in another issue/PR, and keep this one primarily for doing the initial implementation of the port so it doesn't get too big.

Agree, by the moment being able to import Python modules would be enough.

@matthewelse
Copy link
Author

@piranna I've now implemented basic support for importing modules (for testing purposes), though I think there are better ways of approaching this than the way I've done it currently. Namely, I think we can improve the implementation by abstracting away the mp_import_stat and mp_lexer_new_from_file to the JavaScript layer, so that different JS environments can have their own implementation (the implementation I've provided currently works for node.js, but wouldn't make sense in the browser for example).

@piranna
Copy link

piranna commented Nov 11, 2016

For Python modules your approach makes sense, and the same your idea about abstract it to allow it to work on browsers. For Node.js modules you can use the require() function instead, it should be easy to addapt your code to add support for it since you are already embeding Javascript code. Maybe something like browserify or brfs could be used to add support for the browser without complicating too much the implementation...

Come on, it's almost there! :-P

@piranna
Copy link

piranna commented Nov 11, 2016

I think the problem of Node.js modules allowing to export objects or functions is not a problem at all here, since the only difference would be that the import statement would return a Function or whatever other object instead of a module object, but also this code would only makes sense if it later would run on a Node.js/Javascript environment since you already are using a npm module... isn't it?

@matthewelse
Copy link
Author

matthewelse commented Nov 11, 2016

@piranna that sounds about right, though I'm not sure it'll be that straight-forward to map a JS object straight over to micropython

@piranna
Copy link

piranna commented Nov 11, 2016

@piranna that sounds about right, though I'm not sure it'll be that straight-forward to map a JS object straight over to micropython

It's not about mapping the objects, but just being able to access to the API of the modules. For simple use cases like JSON or simple function calls a call to require() would be just enough, later we could see about how to do the proper mapping/conversion between the returned values and the methods calls.

@matthewelse
Copy link
Author

matthewelse commented Nov 13, 2016

@piranna @pfalcon - just wondering whether you had any thoughts about things we should aim to include in this PR before we aim to get it merged. Any ideas?

@pfalcon
Copy link
Contributor

pfalcon commented Nov 13, 2016

I can only point to @dpgeorge's comment above - it make take a few months to get merged, sorry about that. Please spend this time productively with @piranna and other interested parties. There was a bounty somewhere, if you're interested, contact the sponsor to get the exact details what he wants to be done to claim it, etc. And please keep patches clean and conforming to the existing codestyle ;-).

@piranna
Copy link

piranna commented Nov 13, 2016

I would add all the cpython cli arguments, and with a low priority support
to import npm modules as if they were python ones.

El 13/11/2016 21:48, "Paul Sokolovsky" notifications@github.com escribió:

I can only point to @dpgeorge https://github.com/dpgeorge's comment above

  • it make take a few months to get merged, sorry about that. Please spend
    this time productively with @piranna https://github.com/piranna and other
    interested parties. There was a bounty somewhere, if you're interested,
    contact the sponsor to get the exact details what he wants to be done to
    claim it, etc. And please keep patches clean and conforming to the existing
    codestyle ;-).


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2618 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAgfvnbj4Qm_LtRwKIrdzbHWUFbmdu3yks5q93eOgaJpZM4KvOcA
.

@alanjds
Copy link

alanjds commented Nov 14, 2016

@matthewelse This is very nice. Thanks for applying for the #888 bounty. I will try it as soon as we have a browser version.

@piranna
Copy link

piranna commented Jan 5, 2017

How is the status of that?

#!/usr/bin/env node

var repl = require('repl');
var mpy = require('./main.js');
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's now importable, this line can be var mpy = require('.'); instead, that's cleaner.

@kkimdev
Copy link

kkimdev commented May 19, 2017

@matthewelse Hi, what's the status of this?

@dpgeorge
Copy link
Member

Thanks @matthewelse and others involved here for the effort. Sorry it didn't get merged, but was superseded by #3575 which was merged in 7d675f3

@dpgeorge dpgeorge closed this Mar 13, 2019
tannewt added a commit to tannewt/circuitpython that referenced this pull request Feb 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants